Serverless Framework: ways to achieve full "infrastructure as code"?

I agree that documentation on this would make an excellent pull request here.

You're correct that serverless is using CloudFormation under the hood. The framework does expose the underlying CloudFormation machinery to you, by way of the resources key of your serverless.yml.

I think the intent of the framework is that you would put the rest of these resources (Cognito stuff, S3, etc.) in the resources: section of your serverless.yml file, using regular old CloudFormation syntax.

For example, this file will create a DynamoDB table and S3 bucket, in addition to the serverless function:

service: aws-nodejs # NOTE: update this with your service name
provider:
  name: aws
  runtime: nodejs6.10
functions:
  hello:
    handler: handler.deletecustomer
    events:
      - http:
          path: /deletecustomer
          method: post
          cors: true
resources:
  Resources:
    tablenotes:
      Type: AWS::DynamoDB::Table
      Properties:
        AttributeDefinitions:
          - AttributeName: noteId
            AttributeType: S
          - AttributeName: userId
            AttributeType: S
        KeySchema:
          - AttributeName: userId
            KeyType: HASH
          - AttributeName: noteId
            KeyType: RANGE
        ProvisionedThroughput:
          ReadCapacityUnits: '5'
          WriteCapacityUnits: '5'
    mysamplebucket:
      Type: AWS::S3::Bucket
      Properties:
        WebsiteConfiguration:
          IndexDocument: index.html
          ErrorDocument: error.html
        AccessControl: Private
        VersioningConfiguration:
          Status: Suspended

If you're new to CloudFormation, I'd also recommend taking a peek at CloudFormer.