Is it possible to combine AWS CodeBuild and CodePipeline to build described CI workflow?

CodeBuild now directly supports building GitHub pull requests (without Lambda intermediate step), if you're looking to simply run a build as part of the PR. For running more steps with CodePipeline as part of a PR, you'll still need to set up some scaffolding as the other answers suggest. https://aws.amazon.com/about-aws/whats-new/2017/09/aws-codebuild-now-supports-building-github-pull-requests/

CodePipeline does support basic, fully-managed integrations with both GitHub and CodeBuild, as listed in Product and Service Integrations with AWS CodePipeline. With these integrations, it is possible to use CodeBuild with CodePipeline to run a build of a commit when a commit is pushed to a branch on GitHub. See Use AWS CodePipeline with AWS CodeBuild to Run Builds for details on integrating CodeBuild with CodePipeline as a Build action provider, and see the Four-Stage Pipeline Tutorial for details on integrating Github with CodePipeline as a Source action provider.

Currently, the Pull Request feature in Github is not supported in the official CodePipeline integration, you did not overlook anything. For an interesting AWS-ecosystem open source project (not yet v1.0) that does support GitHub Pull Request integration (though not yet CodePipeline), you might want to check out LambCI.

It looks like this can be done somewhat manually by using Lambda and S3 - https://aws.amazon.com/blogs/devops/integrating-git-with-aws-codepipeline/

Webhooks notify a remote service by issuing an HTTP POST when a commit is pushed to the repository. AWS Lambda receives the HTTP POST through Amazon API Gateway, and then downloads a copy of the repository. It places a zipped copy of the repository into a versioned S3 bucket. AWS CodePipeline can then use the zip file in S3 as a source; the pipeline will be triggered whenever the Git repository is updated.