Getting GitLab CI to clone private repositories

I'm posting this as an answer since others weren't completely clear and/or detailed IMHO

Starting from GitLab 8.12+, assuming the submodule repo is in the same server as the one requesting it, you can now:

1. Set up the repo with git submodules as usual (git submodule add git@somewhere:folder/mysubmodule.git)

2. Modify your .gitmodules file as follows

    [submodule "mysubmodule"]
      path = mysubmodule
      url = ../../group/mysubmodule.git
   

   where ../../group/mysubmodule.git is a relative path from your repository to the submodule's one.

3. Add the following lines to gitlab-ci.yml

    variables:
      GIT_SUBMODULE_STRATEGY: recursive
   

   to instruct the runner to fetch all submodules before the build.

Caveat: if your runner seems to ignore the GIT_SUBMODULE_STRATEGY directive, you should probably consider updating it.

(source: https://docs.gitlab.com/ce/ci/git_submodules.html)

See also other solutions:

• git submodule permission (see Marco A.'s answer)
• job token and override repo in git config (see a544jh's answer)

---

Here a full howto with SSH keys:

General Design

• generating a pair of SSH keys
• adding the private one as a secure environment variable of your project
• making the private one available to your test scripts on GitLab-CI
• adding the public one as a deploy key on each of your private dependencies

Generating a pair of public and private SSH keys

Generate a pair of public and private SSH keys without passphrase:

ssh-keygen -b 4096 -C "<name of your project>" -N "" -f /tmp/name_of_your_project.key

Adding the private SSH key to your project

You need to add the key as a secure environment variable to your project as following:

• browse https://<gitlab_host>/<group>/<project_name>/variables
• click on "Add a variable"
• fill the text field Key with SSH_PRIVATE_KEY
• fill the text field Value with the private SSH key itself
• click on "Save changes"

Exposing the private SSH key to your test scripts

In order to make your private key available to your test scripts you need to add the following to your .gitlab-ci.yml file:

before_script:
  # install ssh-agent
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  # run ssh-agent
  - eval $(ssh-agent -s)
  # add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  # disable host key checking (NOTE: makes you susceptible to man-in-the-middle attacks)
  # WARNING: use only in docker container, if you use it with shell you will overwrite your user's ssh config
  - mkdir -p ~/.ssh
  - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config

Code Snippet comes from GitLab documentation

Adding the public SSH key as a deploy key to all your private dependencies

You need to register the public SSH key as deploy key to all your private dependencies as following:

• browse https://<gitlab_host>/<group>/<dependency_name>/deploy_keys
• click on "New deploy key"
• fill the text field Title with the name of your project
• fill the text field Key with the public SSH key itself
• click on "Create deploy key"

If you don't want to fiddle around with ssh keys or submodules, you can override the repo in git's configuration to authenticate with the job token instead (in gitlab-ci.yml):

before_script:
  - git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com/group/repo.git".insteadOf [email protected]:group/repo.git