Gitlab CI in multiple platforms simultaneously
Check out the latest GitLab release 11.5 (Nov. 2018) with:
Parallel attribute for faster pipelines
The speed of pipelines is an important factor for any team, and running tests or other parallelizable tasks tends to take a big chunk of the time for any build.
Adding this new keyword gives teams the ability to simply parallelize tests, allowing everyone to accelerate their software delivery process.
To use this feature, simply set the parallel attribute to how many copies of the task you’d like to split it into, and GitLab will handle the work of automatically creating the appropriate number of jobs running your task.
As commented by March, this is not an exact fit for the OP (which is about running the same pipeline in multiple architectures/systems parallel to each other).
See documentation and issue.
parallel
allows you to configure how many instances of a job to run in parallel.
This value has to be greater than or equal to two (2) and less or equal than 50.This creates
N
instances of the same job that run in parallel.
They’re named sequentially fromjob_name
1/N tojob_name
N/N.test: script: rspec parallel: 5
GitLab uses "runners" to execute CI jobs. Runners are installed wherever you want to run a CI job, so if you want to run on multiple architectures then you will need to install runners on systems for each architecture. Runner install documentation can be found here:
https://docs.gitlab.com/runner/install/index.html
For Linux-based jobs it is common to use Docker for job execution - this doesn't give architectural flexibility, but it does allow you to test on different flavors and with different software using containerisation. For other architectures you may need to install runners yourself, or use other peoples shared runners.
While you are installing the runner software there are some keys steps:
you have the opportunity to link each runner to your GitLab project, which means it will show up in the runners list under Project > Settings > CI/CD.
you will have the opportunity to assign "tags" to the runners. Tags can be used to help identify a runner or group of runners by an arbitrary name (e.g. you could add "Windows x86_64" as a tag, or "Windows" and "x86_64" tags). These tags can be used in jobs to select a runner.
Once you have your runners installed you can get editing your .gitlab-ci.yml
file.
GitLab CI files are broken up into "stages". Jobs in each stage can run in parallel. Stage names are defined at the top of the file.
stages:
- build
- deploy
Each CI job can be attached to a stage by using the stage:
entry:
build job:
stage: build
script:
- echo "I am a build stage job"
In your case you will need to create multiple jobs for each architecture you want to build for. Attaching these to the same stage will allow them to run in parallel.
To control where each job runs you have two main mechanisms:
Tags - tags allow you to pin a job to a runner tag. You can specify multiple tags using the
tags:
entry which forms an AND list (e.g.win
tag ANDx86_64
tag). When that job runs GitLab will find a runner that has all the required tags, and run the job there.Image - When running on Docker / Kubernetes you can specify a docker image to use for the runner. To use a docker image you first need to specify a runner that can run docker images (e.g. a docker-in-docker or kubernetes runner), which might, for example, be tagged with
docker
orkubernetes
. Then you use theimage:
entry to specify the docker image.
Here's an example showing both tags and images:
build win x86_64:
stage: build
tags:
- win
- x86_64
script:
- echo "I am a build stage job for win x86_64"
build win 32:
stage: build
tags:
- win
- 32-bit
script:
- echo "I am a build stage job for win 32"
build debian:
stage: build
tags:
- docker
image: debian:stretch
script:
- echo "I am a build stage job for debian, running on docker using debian:stretch image"
There is currently no support for dynamic jobs, or running one job on multiple runners / architectures, so this involves a bit of manual effort. On the positive side it makes GitLab CI files easy to read, and easy to see what will run during CI execution.