How to auto deploying git repositories with submodules on AWS?

After banging my head against this all day, I've found a simple solution (for Code Pipeline) that doesn't require any SSH key juggling in the buildspec. I am using Bitbucket but I would think this would work for other providers. I'm also cloning my submodule via https, I'm not sure if that's a requirement or not.

1. Configure your source to do a full clone of the repository. This will pass along the git metadata that you need.

2. Configure your build role to add a customer-managed UseConnection permission to give your build action access to the credentials you configured for your source. Documentation from AWS here: https://docs.aws.amazon.com/codepipeline/latest/userguide/troubleshooting.html#codebuild-role-connections

3. Set up your env to include git-credential-helper: yes and clone the submodule in your buildspec.yml:

And that's it! Submodule will be available for build, and without having to do a bunch of key configuration for every submodule you want to use.

Maybe a good addition to the documentation if this ends up being useful for people.

Edit: Codebuild now has a "submodules" flag https://docs.aws.amazon.com/codebuild/latest/APIReference/API_GitSubmodulesConfig.html

Here's what worked for me

We're going to reinitialize the git repository and then trigger a submodule clone during the build phase of our deploy, essentially patching in support for submodules in codepipeline / codebuild

• Generate a new SSH key for your github account, if using an organization you may want to create a deploy user
• Store this ssh key in your aws parameter store using aws ssm put-parameter --name build_ssh_key --type String --value "$(cat id_rsa)" ideally use SecureString instead of String but the guide I was following simply used string so I'm not sure if the commandline will require any extra params
• Go into IAM and grant your CodePipeline user read access to your paramstore, I just granted read access to SSM

Then make your buildspec.yml look like the following:

version: 0.2

env:
  parameter-store:
    build_ssh_key: "build_ssh_key"

phases:
  install:
    commands:
      - mkdir -p ~/.ssh
      - echo "$build_ssh_key" > ~/.ssh/id_rsa
      - chmod 600 ~/.ssh/id_rsa
      - ssh-keygen -F github.com || ssh-keyscan github.com >>~/.ssh/known_hosts
      - git config --global url."[email protected]:".insteadOf "https://github.com/"
      - git init
      - git remote add origin <Your Repo url here using the git protocol>
      - git fetch
      - git checkout -t origin/master
      - git submodule init
      - git submodule update --recursive
  build:
    commands:
      - echo '...replace with real build commands...'

artifacts:
  files:
    - '**/*'