How to test ansible roles?
Solution
In order to test ansible roles with multiple versions of ansible, python and various Linux flavors we can use
- molecule for our ansible role functionality
- docker as our abstraction layer on which we run the target system for our ansible role
- tox to setup generic virtualenvs and test our various combinations without side effects
- travis to automate it all
This will be quite a long/detailed answer. You can check out an example ansible role with the whole setup here
- https://github.com/ckaserer/ansible-role-example
- https://travis-ci.com/ckaserer/ansible-role-example
Step 1: test ansible role with molecule
Molecule docs: https://molecule.readthedocs.io/en/stable/
Fixes Issue: 1) no official ansible images
I could create ansible images for every distro I would like to test as jeff geerling describes in his blog posts.
The clear downside of this approach: The images will need maintenance (eventually)
However, with molecule, we can combine base images with a Dockerfile.j2 (Jinja2 Template) to create the images with minimal requirements to run ansible. Through this approach, we are now able to use the official linux distro images from docker hub and do not need to maintain a docker repo for each linux distro and the various versions.
Here the important bits in molecule.yml
platforms:
- name: instance-${TOX_ENVNAME}
image: ${MOLECULE_DISTRO:-'centos:7'}
command: /sbin/init
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
The default dockerfile.j2 from molecule is already good, but I have a few additions
# Molecule managed
{% if item.registry is defined %}
FROM {{ item.registry.url }}/{{ item.image }}
{% else %}
FROM {{ item.image }}
{% endif %}
{% if item.env is defined %}
{% for var, value in item.env.items() %}
{% if value %}
ENV {{ var }} {{ value }}
{% endif %}
{% endfor %}
{% endif %}
RUN if [ $(command -v apt-get) ]; then apt-get update && apt-get install -y python sudo bash ca-certificates iproute2 && apt-get clean; \
elif [ $(command -v zypper) ]; then zypper refresh && zypper install -y python sudo bash python-xml iproute2 && zypper clean -a; \
elif [ $(command -v apk) ]; then apk update && apk add --no-cache python sudo bash ca-certificates; \
elif [ $(command -v xbps-install) ]; then xbps-install -Syu && xbps-install -y python sudo bash ca-certificates iproute2 && xbps-remove -O; \
elif [ $(command -v swupd) ]; then swupd bundle-add python3-basic sudo iproute2; \
elif [ $(command -v dnf) ] && cat /etc/os-release | grep -q '^NAME=Fedora'; then dnf makecache && dnf --assumeyes install python sudo python-devel python*-dnf bash iproute && dnf clean all; \
elif [ $(command -v dnf) ] && cat /etc/os-release | grep -q '^NAME="CentOS Linux"' ; then dnf makecache && dnf --assumeyes install python36 sudo platform-python-devel python*-dnf bash iproute && dnf clean all && ln -s /usr/bin/python3 /usr/bin/python; \
elif [ $(command -v yum) ]; then yum makecache fast && yum install -y python sudo yum-plugin-ovl bash iproute && sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf && yum clean all; \
fi
# Centos:8 + ansible 2.7 failed with error: "The module failed to execute correctly, you probably need to set the interpreter"
# Solution: ln -s /usr/bin/python3 /usr/bin/python
By default, this will test the role with centos:7. However, we can set the environment variable MOLECULE_DISTRO to whichever image we would like to test and run it via
MOLECULE_DISTRO=ubuntu:18.04 molecule test
Summary
We use official distro images from docker hub to test our ansible role via molecule.
The files used in this step
- molecule.yml https://github.com/ckaserer/ansible-role-example/blob/master/molecule/default/molecule.yml
- Dockerfile.j2 https://github.com/ckaserer/ansible-role-example/blob/master/molecule/default/Dockerfile.j2
Source
- https://molecule.readthedocs.io/en/stable/examples.html#systemd-container
- https://www.jeffgeerling.com/blog/2019/how-i-test-ansible-configuration-on-7-different-oses-docker
- https://www.jeffgeerling.com/blog/2018/testing-your-ansible-roles-molecule
Step 2: Check compatibility for your role ( python version X ansible version X linux distros )
Fixes Issue 2: how to best test a role for ansible version compatibility?
Let's use tox to create virtual environments to avoid side effects while testing various compatibility scenarios.
Here the important bits in tox.ini
[tox]
minversion = 3.7
envlist = py{3}-ansible{latest,29,28}-{ alpinelatest,alpine310,alpine39,alpine38, centoslatest,centos8,centos7, debianlatest,debian10,debian9,debian8, fedoralatest,fedora30,fedora29,fedora28, ubuntulatest,ubuntu2004,ubuntu1904,ubuntu1804,ubuntu1604 }
# only test currently supported ansible versions
# https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#release-status
skipsdist = true
[base]
passenv = *
deps =
-rrequirements.txt
ansible25: ansible==2.5
...
ansiblelatest: ansible
commands =
molecule test
setenv =
TOX_ENVNAME={envname}
MOLECULE_EPHEMERAL_DIRECTORY=/tmp/{envname}
[testenv]
passenv =
{[base]passenv}
deps =
{[base]deps}
commands =
{[base]commands}
setenv =
...
centoslatest: MOLECULE_DISTRO="centos:latest"
centos8: MOLECULE_DISTRO="centos:8"
centos7: MOLECULE_DISTRO="centos:7"
...
{[base]setenv}
The entirety of requirements.txt
docker
molecule
by simply executing
tox
it will create virtual envs for each compatibility combination defined in tox.ini by
envlist = py{3}-ansible{latest,29,28}-{ alpinelatest,alpine310,alpine39,alpine38, centoslatest,centos8,centos7, debianlatest,debian10,debian9,debian8, fedoralatest,fedora30,fedora29,fedora28, ubuntulatest,ubuntu2004,ubuntu1904,ubuntu1804,ubuntu1604 }
which translates in this particular case to: python3 x ansible version x linux distro
Great! We have created tests for compatibility checks with the added benefit of always testing with ansible latest to notice breaking changes early on.
The files used in this step
- tox.ini https://github.com/ckaserer/ansible-role-example/blob/master/tox.ini
- requirements.txt https://github.com/ckaserer/ansible-role-example/blob/master/requirements.txt
Source
- https://tox.readthedocs.io/en/latest/
- https://molecule.readthedocs.io/en/stable/testing.html#tox
Step 3: CI with travis
Running the checks locally is good, running in a CI tool is great. So let's do that.
For this purpose following bits in the .travis.yml are important
---
version: ~> 1.0
os: linux
language: python
python:
- "3.8"
- "3.7"
- "3.6"
- "3.5"
services: docker
cache:
pip: true
directories:
- .tox
install:
- pip install tox-travis
env:
jobs:
# ansible:latest - check for breaking changes
...
- TOX_DISTRO="centoslatest" TOX_ANSIBLE="latest"
- TOX_DISTRO="centos8" TOX_ANSIBLE="latest"
- TOX_DISTRO="centos7" TOX_ANSIBLE="latest"
...
# ansible: check version compatibility
# only test currently supported ansible versions
#
https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#release-status
- TOX_DISTRO="centoslatest" TOX_ANSIBLE="{29,28}"
- TOX_DISTRO="centos8" TOX_ANSIBLE="{29,28}"
- TOX_DISTRO="centos7" TOX_ANSIBLE="{29,28}"
...
script:
- tox -e $(echo py${TRAVIS_PYTHON_VERSION} | tr -d .)-ansible${TOX_ANSIBLE}-${TOX_DISTRO}
# remove logs/pycache before caching .tox folder
- |
rm -r .tox/py*/log/*
find . -type f -name '*.py[co]' -delete -o -type d -name __pycache__ -delete
First we have specified language: python to run builds with multiple versions of python as defined in the python: list.
We need docker, so we add it via services: docker.
The test will take quite some time, let's cache pip and our virtenv created by tox with
cache:
pip: true
directories:
- .tox
We need tox...
install:
- pip install tox-travis
And finally, we define all our test cases
env:
jobs:
# ansible:latest - check for breaking changes
...
- TOX_DISTRO="centoslatest" TOX_ANSIBLE="latest"
...
Note that I have separate jobs for latest and the distinct versions. That is on purpose. I would like to easily see what broke. Is it version compatibility or an upcoming change located in ansible's latest release.
The files used in this step
- https://github.com/ckaserer/ansible-role-example/blob/master/.travis.yml
Source
- https://docs.travis-ci.com/user/caching/
Bonus: run tox in parallel
You can run the tests in parallel (e.g. 3 test simultaneous) by executing
tox -p 3
However, this will not give the output from molecule. You can enable that with
tox -p 3 -o true
The obvious downside to this approach is the pain of figuring out which line belongs to which process in the parallel execution.
Source
- https://tox.readthedocs.io/en/latest/example/basic.html#parallel-mode
No real answer here, but some ideas :
Ansible Silo might have fitted, but no commit for a year.
And it's not exactly what you're looking for, but Ansible Runner is meant to be a fit for the "run ansible" use case. And it's a part of Ansible Tower / AWS, so it should last.
Runner is intended to be most useful as part of automation and tooling that needs to invoke Ansible and consume its results.
They do mention executing from a container here
The design of Ansible Runner makes it especially suitable for controlling the execution of Ansible from within a container for single-purpose automation workflows
But the issue for you is that the official ansible/ansible-runner container is tagged after ansible-runner version, and ansible itself is installed through pip install ansible at container build time.