Ansible testing with Molecule

(Updated: )
8 minute read
configuration-management
,

When I first started using ansible, I did not know about molecule. It was a bit daunting to start a role from scratch and trying to develop it without having the ability to test it. Then a co-worker of mine told me about molecule and everything changed.

I do not have any of the tools I need installed on this machine, so I will go through, step by step, how I set up ansible and molecule on any new machine I come across for writing ansible roles.

Requirements

What we are trying to achieve in this post, is a working ansible role that can be tested inside a docker container. To be able to achieve that, we need to install docker on the system. Follow the instructions on installing docker found on the docker website.

Good Practices

First thing’s first. Let’s start by making sure that we have python installed properly on the system.

 $ python --version
 Python 3.7.1

Because in this case I have python3 installed, I can create a virtualenv easier without the use of external tools.

 # Create the directory to work with
 $ mkdir -p sandbox/test-roles
 # Navigate to the directory
 $ cd sandbox/test-roles/
 # Create the virtualenv
 ~/sandbox/test-roles $ python -m venv .ansible-venv
 # Activate the virtualenv
 ~/sandbox/test-roles $ source .ansible-venv/bin/activate
 # Check that your virtualenv activated properly
 (.ansible-venv) ~/sandbox/test-roles $ which python
 /home/elijah/sandbox/test-roles/.ansible-venv/bin/python

At this point, we can install the required dependencies.

 $ pip install ansible molecule docker
 Collecting ansible
   Downloading https://files.pythonhosted.org/packages/56/fb/b661ae256c5e4a5c42859860f59f9a1a0b82fbc481306b30e3c5159d519d/ansible-2.7.5.tar.gz (11.8MB)
     100% |████████████████████████████████| 11.8MB 3.8MB/s
 Collecting molecule
   Downloading https://files.pythonhosted.org/packages/84/97/e5764079cb7942d0fa68b832cb9948274abb42b72d9b7fe4a214e7943786/molecule-2.19.0-py3-none-any.whl (180kB)
     100% |████████████████████████████████| 184kB 2.2MB/s

 ...

 Successfully built ansible ansible-lint anyconfig cerberus psutil click-completion tabulate tree-format pathspec future pycparser arrow
 Installing collected packages: MarkupSafe, jinja2, PyYAML, six, pycparser, cffi, pynacl, idna, asn1crypto, cryptography, bcrypt, paramiko, ansible, pbr, git-url-parse, monotonic, fasteners, click, colorama, sh, python-gilt, ansible-lint, pathspec, yamllint, anyconfig, cerberus, psutil, more-itertools, py, attrs, pluggy, atomicwrites, pytest, testinfra, ptyprocess, pexpect, click-completion, tabulate, future, chardet, binaryornot, poyo, urllib3, certifi, requests, python-dateutil, arrow, jinja2-time, whichcraft, cookiecutter, tree-format, molecule, docker-pycreds, websocket-client, docker
 Successfully installed MarkupSafe-1.1.0 PyYAML-3.13 ansible-2.7.5 ansible-lint-3.4.23 anyconfig-0.9.7 arrow-0.13.0 asn1crypto-0.24.0 atomicwrites-1.2.1 attrs-18.2.0 bcrypt-3.1.5 binaryornot-0.4.4 cerberus-1.2 certifi-2018.11.29 cffi-1.11.5 chardet-3.0.4 click-6.7 click-completion-0.3.1 colorama-0.3.9 cookiecutter-1.6.0 cryptography-2.4.2 docker-3.7.0 docker-pycreds-0.4.0 fasteners-0.14.1 future-0.17.1 git-url-parse-1.1.0 idna-2.8 jinja2-2.10 jinja2-time-0.2.0 molecule-2.19.0 monotonic-1.5 more-itertools-5.0.0 paramiko-2.4.2 pathspec-0.5.9 pbr-4.1.0 pexpect-4.6.0 pluggy-0.8.1 poyo-0.4.2 psutil-5.4.6 ptyprocess-0.6.0 py-1.7.0 pycparser-2.19 pynacl-1.3.0 pytest-4.1.0 python-dateutil-2.7.5 python-gilt-1.2.1 requests-2.21.0 sh-1.12.14 six-1.11.0 tabulate-0.8.2 testinfra-1.16.0 tree-format-0.1.2 urllib3-1.24.1 websocket-client-0.54.0 whichcraft-0.5.2 yamllint-1.11.1

Creating your first ansible role

Once all the steps above are complete, we can start by creating our first ansible role.

 $ molecule init role -r example-role
 --> Initializing new role example-role...
 Initialized role in /home/elijah/sandbox/test-roles/example-role successfully.

 $ tree example-role/
 example-role/
 ├── defaults
 │   └── main.yml
 ├── handlers
 │   └── main.yml
 ├── meta
 │   └── main.yml
 ├── molecule
 │   └── default
 │       ├── Dockerfile.j2
 │       ├── INSTALL.rst
 │       ├── molecule.yml
 │       ├── playbook.yml
 │       └── tests
 │           ├── __pycache__
 │           │   └── test_default.cpython-37.pyc
 │           └── test_default.py
 ├── README.md
 ├── tasks
 │   └── main.yml
 └── vars
     └── main.yml

 9 directories, 12 files

You can find what each directory is for and how ansible works by visiting docs.ansible.com.

meta/main.yml

The meta file needs to modified and filled with information about the role. This is not a required file to modify if you are keeping this for yourself, for example. But it is a good idea to have as much information as possible if this is going to be released. In my case, I don’t need any fanciness as this is just sample code.

---
galaxy_info:
  author: Elia el Lazkani
  description: This is an example ansible role to showcase molecule at work
  license: license (BDS-2)
  min_ansible_version: 2.7
  galaxy_tags: []
dependencies: []

tasks/main.yml

This is where the magic is set in motion. Tasks are the smallest entities in a role that do small and idempotent actions. Let’s write a few simple tasks to create a user and install a service.

---
# Create the user example
- name: Create 'example' user
  user:
    name: example
    comment: Example user
    shell: /bin/bash
    state: present
    create_home: yes
    home: /home/example

# Install nginx
- name: Install nginx
  apt:
    name: nginx
    state: present
    update_cache: yes
  notify: Restart nginx

handlers/main.yml

If you noticed, we are notifying a handler to be called after installing nginx. All handlers notified will run after all the tasks complete and each handler will only run once. This is a good way to make sure that you don’t restart nginx multiple times if you call the handler more than once.

---
# Handler to restart nginx
- name: Restart nginx
  service:
    name: nginx
    state: restarted

molecule/default/molecule.yml

It’s time to configure molecule to do what we need. We need to start an ubuntu docker container, so we need to specify that in the molecule YAML file. All we need to do is change the image line to specify that we want an ubuntu:bionic image.

---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: instance
    image: ubuntu:bionic
provisioner:
  name: ansible
  lint:
    name: ansible-lint
scenario:
  name: default
verifier:
  name: testinfra
  lint:
    name: flake8

molecule/default/playbook.yml

This is the playbook that molecule will run. Make sure that you have all the steps that you need here. I will keep this as is.

---
- name: Converge
  hosts: all
  roles:
    - role: example-role

First Role Pass

This is time to test our role and see what’s going on.

 (.ansible-role) ~/sandbox/test-roles/example-role/ $ molecule converge
 --> Validating schema /home/elijah/sandbox/test-roles/example-role/molecule/default/molecule.yml.
 Validation completed successfully.
 --> Test matrix

 └── default
     ├── dependency
     ├── create
     ├── prepare
     └── converge

 --> Scenario: 'default'
 --> Action: 'dependency'
 Skipping, missing the requirements file.
 --> Scenario: 'default'
 --> Action: 'create'

     PLAY [Create] ******************************************************************

     TASK [Log into a Docker registry] **********************************************
     skipping: [localhost] => (item=None)

     TASK [Create Dockerfiles from image names] *************************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Discover local Docker images] ********************************************
     ok: [localhost] => (item=None)
     ok: [localhost]

     TASK [Build an Ansible compatible image] ***************************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Create docker network(s)] ************************************************

     TASK [Create molecule instance(s)] *********************************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Wait for instance(s) creation to complete] *******************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     PLAY RECAP *********************************************************************
     localhost                  : ok=5    changed=4    unreachable=0    failed=0


 --> Scenario: 'default'
 --> Action: 'prepare'
 Skipping, prepare playbook not configured.
 --> Scenario: 'default'
 --> Action: 'converge'

     PLAY [Converge] ****************************************************************

     TASK [Gathering Facts] *********************************************************
     ok: [instance]

     TASK [example-role : Create 'example' user] ************************************
     changed: [instance]

     TASK [example-role : Install nginx] ********************************************
     changed: [instance]

     RUNNING HANDLER [example-role : Restart nginx] *********************************
     changed: [instance]

     PLAY RECAP *********************************************************************
     instance                   : ok=4    changed=3    unreachable=0    failed=0

It looks like the converge step succeeded.

Writing Tests

It is always a good practice to write unittests when you’re writing code. Ansible roles should not be an exception. Molecule offers a way to run tests, which you can think of as unittest, to make sure that what the role gives you is what you were expecting. This helps future development of the role and keeps you from falling in previously solved traps.

molecule/default/tests/test_default.py

Molecule leverages the testinfra project to run its tests. You can use other tools if you so wish, and there are many. In this example we will be using testinfra.

import os

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')


def test_hosts_file(host):
    f = host.file('/etc/hosts')

    assert f.exists
    assert f.user == 'root'
    assert f.group == 'root'


def test_user_created(host):
    user = host.user("example")
    assert user.name == "example"
    assert user.home == "/home/example"


def test_user_home_exists(host):
    user_home = host.file("/home/example")
    assert user_home.exists
    assert user_home.is_directory


def test_nginx_is_installed(host):
    nginx = host.package("nginx")
    assert nginx.is_installed


def test_nginx_running_and_enabled(host):
    nginx = host.service("nginx")
    assert nginx.is_running

warning

Uncomment truthy: disable in .yamllint found at the base of the role.

 (.ansible_venv) ~/sandbox/test-roles/example-role $ molecule test
 --> Validating schema /home/elijah/sandbox/test-roles/example-role/molecule/default/molecule.yml.
 Validation completed successfully.
 --> Test matrix

 └── default
     ├── lint
     ├── destroy
     ├── dependency
     ├── syntax
     ├── create
     ├── prepare
     ├── converge
     ├── idempotence
     ├── side_effect
     ├── verify
     └── destroy

 --> Scenario: 'default'
 --> Action: 'lint'
 --> Executing Yamllint on files found in /home/elijah/sandbox/test-roles/example-role/...
 Lint completed successfully.
 --> Executing Flake8 on files found in /home/elijah/sandbox/test-roles/example-role/molecule/default/tests/...
 /home/elijah/.virtualenvs/world/lib/python3.7/site-packages/pycodestyle.py:113: FutureWarning: Possible nested set at position 1
   EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
 Lint completed successfully.
 --> Executing Ansible Lint on /home/elijah/sandbox/test-roles/example-role/molecule/default/playbook.yml...
 Lint completed successfully.
 --> Scenario: 'default'
 --> Action: 'destroy'

     PLAY [Destroy] *****************************************************************

     TASK [Destroy molecule instance(s)] ********************************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Wait for instance(s) deletion to complete] *******************************
     ok: [localhost] => (item=None)
     ok: [localhost]

     TASK [Delete docker network(s)] ************************************************

     PLAY RECAP *********************************************************************
     localhost                  : ok=2    changed=1    unreachable=0    failed=0


 --> Scenario: 'default'
 --> Action: 'dependency'
 Skipping, missing the requirements file.
 --> Scenario: 'default'
 --> Action: 'syntax'

     playbook: /home/elijah/sandbox/test-roles/example-role/molecule/default/playbook.yml

 --> Scenario: 'default'
 --> Action: 'create'

     PLAY [Create] ******************************************************************

     TASK [Log into a Docker registry] **********************************************
     skipping: [localhost] => (item=None)

     TASK [Create Dockerfiles from image names] *************************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Discover local Docker images] ********************************************
     ok: [localhost] => (item=None)
     ok: [localhost]

     TASK [Build an Ansible compatible image] ***************************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Create docker network(s)] ************************************************

     TASK [Create molecule instance(s)] *********************************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Wait for instance(s) creation to complete] *******************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     PLAY RECAP *********************************************************************
     localhost                  : ok=5    changed=4    unreachable=0    failed=0


 --> Scenario: 'default'
 --> Action: 'prepare'
 Skipping, prepare playbook not configured.
 --> Scenario: 'default'
 --> Action: 'converge'

     PLAY [Converge] ****************************************************************

     TASK [Gathering Facts] *********************************************************
     ok: [instance]

     TASK [example-role : Create 'example' user] ************************************
     changed: [instance]

     TASK [example-role : Install nginx] ********************************************
     changed: [instance]

     RUNNING HANDLER [example-role : Restart nginx] *********************************
     changed: [instance]

     PLAY RECAP *********************************************************************
     instance                   : ok=4    changed=3    unreachable=0    failed=0


 --> Scenario: 'default'
 --> Action: 'idempotence'
 Idempotence completed successfully.
 --> Scenario: 'default'
 --> Action: 'side_effect'
 Skipping, side effect playbook not configured.
 --> Scenario: 'default'
 --> Action: 'verify'
 --> Executing Testinfra tests found in /home/elijah/sandbox/test-roles/example-role/molecule/default/tests/...
     ============================= test session starts ==============================
     platform linux -- Python 3.7.1, pytest-4.1.0, py-1.7.0, pluggy-0.8.1
     rootdir: /home/elijah/sandbox/test-roles/example-role/molecule/default, inifile:
     plugins: testinfra-1.16.0
 collected 5 items

     tests/test_default.py .....                                              [100%]

     =============================== warnings summary ===============================

 ...

     ==================== 5 passed, 7 warnings in 27.37 seconds =====================
 Verifier completed successfully.
 --> Scenario: 'default'
 --> Action: 'destroy'

     PLAY [Destroy] *****************************************************************

     TASK [Destroy molecule instance(s)] ********************************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Wait for instance(s) deletion to complete] *******************************
     changed: [localhost] => (item=None)
     changed: [localhost]

     TASK [Delete docker network(s)] ************************************************

     PLAY RECAP *********************************************************************
     localhost                  : ok=2    changed=2    unreachable=0    failed=0

I have a few warning messages (that’s likely because I am using python 3.7 and some of the libraries still don’t fully support the new standards released with it) but all my tests passed.

Conclusion

Molecule is a great tool to test ansible roles quickly and while developing them. It also comes bundled with a bunch of other features from different projects that will test all aspects of your ansible code. I suggest you start using it when writing new ansible roles.