The most exciting phrase to hear in science, the one that heralds discoveries, is not ‘Eureka!’ but ‘Now that’s funny…’ – Isaac Asimov
Why we test
At Oteemo, we believe in automating to the maximum degree possible, ensuring consistent, versionable and robust infrastructure and application platforms.
In order to achieve this, we need to be able to describe our infrastructure as code, but also we must have a high level of confidence that the code we write meets the requirements to which we are developing. We like to be able to fail fast and be able to refactor our code without fear.
To automate our infrastructure, we generally look towards Ansible. In the past, to test Ansible, I would use Test Kitchen. Test Kitchen, while being an awesome tool for Chef, was not necessarily ideal for Ansible. One of the reasons we like Ansible is that it arguably puts a greater importance on idempotence than some of the other confuguration management tools. Test Kitchen, while there are work arounds, does not perform an idempotence test. In addition, since Ansible is done in python, it is not very convenient to pull in all of the ruby dependencies that Test Kitchen requires and I find it to be an easier sell to existing operations and engineering teams that are accustomed to hand-jamming golden images when we focus on one language.
Enter Molecule
The Molecule has been out since 2015 and is a testing aid written with Ansible in mind. It allows us to test our roles locally, prior to committing them. It allows us to test against multiple instances, operating systems, and virtualization providers.
Getting Started
$ # If you do not already have Ansible installed
$ pip install ansible
$ # install molecule
$ pip install molecule
Note 1: As of this writing, molecule 1.25.0 is the stable version. If you want to install the release candidate (2.0.0rc7) append --pre
to the install command
Note 2: When installing on a mac, you probably need to preface the above pip commands with sudo -H
Using Molecule
When initializing molecule for either an existing role or a new role, a few decisions should be made:
- Driver
- Vagrant (default)
- Docker
- OpenStack
- Providor
- Virtualbox (default)
- libvirt
- VMWare Fusion
- Parallels
- Verifier
- Testinfra (default)
- Serverspec
- Goss (beta)
For this post, I am going to stick with the defaults. To follow along, be sure to have Vagrant and Virtualbox installed. You will need to install the python package for your driver as well:
$ pip install python-vagrant
Hello World
The Hello World in configuration management land seems to be the ntp install. We can use molecule to scaffold the role for us.
Initialize
$ molecule init --role ntp
This gives us:
$ tree ntp
ntp
├── README.md
├── defaults
│ └── main.yml
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── molecule.yml
├── playbook.yml
├── tasks
│ └── main.yml
├── tests
│ ├── test_default.py
│ └── test_default.pyc
└── vars
└── main.yml
First, let’s look at the contents of molecule.yml.
dependency:
name: galaxy
driver:
name: vagrant
vagrant:
platforms:
- name: ubuntu/trusty64
box: trusty64
box_url: https://vagrantcloud.com/ubuntu/boxes/trusty64/versions/14.04/providers/virtualbox.box
providers:
- name: virtualbox
type: virtualbox
options:
memory: 512
cpus: 2
instances:
- name: ntp
ansible_groups:
- group1
verifier:
name: testinfra
The dependency section allows us to provide role dependencies with ansible-galaxy. we will not be doing that in this tutorial. The other sections should be self explanatory. Feel free to check out the docs if it is not.
To start out, I am going to make a modification to the verifier section:
verifier:
name: testinfra
options:
v: True
sudo: True
The v is for verbosity. This will have Testinfra tell us the tests that run. The sudo flag runs the tests will elevated permissions. While this is not always necessary, I often include it.
Now let’s run it to make sure it works. Then we will start to develop the role.
$ # Make sure you are in the same directory as molecule.yml
$ molecule test
If this is your first foray in to vagrant, or you have never used this particular box before then the first run will take some time as you will have to download the box. Subsequent runs will be quicker.
After vagrant spins up the box, you should see:
Molecule destroy any existing instances
--> Destroying instances...
Molecule runs an ansible syntax check
--> Checking playbook's syntax...
playbook: playbook.yml
Molecule creates the instances
--> Creating instances...
Bringing machine 'ntp' up with 'virtualbox' provider...
==> ntp: Cloning VM...
... [Vagrant output omitted]
==> ntp: Machine not provisioned because `--no-provision` is specified.
Molecule runs the provisioner
--> Starting Ansible Run...
PLAY [all] *********************************************************************
TASK [Gathering Facts] *********************************************************
ok: [ntp]
PLAY RECAP *********************************************************************
ntp : ok=1 changed=0 unreachable=0 failed=0
Molecule runs an idempotence check
--> Idempotence test in progress (can take a few minutes)...
--> Starting Ansible Run...
Idempotence test passed.
Molecule runs linters on ansible and the tests
--> Executing ansible-lint...
--> Executing flake8 on *.py files found in tests/...
Molecule runs the tests
--> Executing testinfra tests found in tests/...
============================= test session starts ==============================
platform darwin -- Python 2.7.12, pytest-3.1.2, py-1.4.34, pluggy-0.4.0 -- /usr/local/opt/python/bin/python2.7
cachedir: .cache
rootdir: /Users/smcgowan/ws/oteemo/ntp, inifile:
plugins: testinfra-1.5.5
collected 1 itemss
tests/test_default.py::test_hosts_file[ansible://ntp] PASSED
=============================== warnings summary ===============================
None
Module already imported so can not be re-written: testinfra
-- Docs: http://doc.pytest.org/en/latest/warnings.html
===================== 1 passed, 1 warnings in 2.47 seconds =====================
Molecule destroys the instance (if everything passed)
--> Destroying instances...
==> ntp: Forcing shutdown of VM...
==> ntp: Destroying VM and associated drives...
You can perform any of these steps individually. Use molecule --help
to see the options.
Install ntp
Now that we have molecule working and the role scaffolded, we can create some tests for the ntp role. The general strategy is to create the tests and run them to go red and then to write the ansible code that passes the tests to go green.
# tests/test_default.py
import testinfra.utils.ansible_runner
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
'.molecule/ansible_inventory').get_hosts('all')
def test_ntp_is_installed(Package)
p = Package('ntp')
assert p.is_installed
Now run molecule test
. This should not pass, since ntp is not installed. Now you are red. To go green, update your role to install the package:
---
# tasks/main.yml
- name: ensure ntp package is installed
package:
name: ntp
state: present
tags: [ntp]
Run molecule test
again, and we should pass!
collected 1 itemss
tests/test_default.py::test_ntp_is_installed[ansible://ntp] PASSED
=============================== warnings summary ===============================
None
Module already imported so can not be re-written: testinfra
-- Docs: http://doc.pytest.org/en/latest/warnings.html
===================== 1 passed, 1 warnings in 2.18 seconds =====================
--> Destroying instances...
==> ntp: Forcing shutdown of VM...
==> ntp: Destroying VM and associated drives...
Hooray!
Start/enable the service
We want to guarantee that ntp is running, so let’s add the test to tests/test_default.py and run molecule test.
import testinfra.utils.ansible_runner
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
'.molecule/ansible_inventory').get_hosts('all')
def test_ntp_is_installed(Package):
p = Package('ntp')
assert p.is_installed
def test_ntp_is_started_and_enabled(Service):
s = Service('ntp')
assert s.is_running
assert s.is_enabled
Ubuntu automatically starts and enables the service, so we do not have anymore work to do. We are still green!
Multiple platforms
Let’s make our role work across multiple platforms. In molecule.yml, update the platforms section:
vagrant:
platforms:
- name: ubuntu/trusty64
box: trusty64
box_url: https://vagrantcloud.com/ubuntu/boxes/trusty64/versions/14.04/providers/virtualbox.box
- name: bento/centos-7.3
box: bento/centos-7.3
Unfortunately, molecule does not yet handle multiple platforms in the same run yet. Molecule v2 handles scenarios is still a release candidate. For now, we will use the –platform option.
$ molecule test --platform bento/centos-7.3
This will fail.
tests/test_default.py::test_ntp_is_installed[ansible://ntp] PASSED
tests/test_default.py::test_ntp_is_started_and_enabled[ansible://ntp] FAILED
=================================== FAILURES ===================================
________________ test_ntp_is_started_and_enabled[ansible://ntp] ________________
Service = <class 'testinfra.modules.base.systemdservice'="">
def test_ntp_is_started_and_enabled(Service):
s = Service('ntp')
> assert s.is_running
E assert False
E + where False = .is_running
tests/test_default.py:16: AssertionError
We are now red again because in CentOS land, the service name is ntpd. We will need to update the test in tests/test_default.py.
def test_ntp_is_started_and_enabled(Service, SystemInfo):
distro = SystemInfo.distribution
if distro == 'ubuntu':
svc = 'ntp'
elif distro == 'centos':
svc = 'ntpd'
s = Service(svc)
assert s.is_running
assert s.is_enabled
We can now run the tests again.
$ molecule test --platform ubuntu/trusty64
The Ubuntu test is still green.
$ molecule test --platform bento/centos-7.3
The CentOS test is still red!!! This is because CentOS does not make the assumption that just because you installed a package that you want the service up and running (also because CentOS 7 uses chronyd by default, so we should probably make sure it is disabled if we want to use ntpd). We will have to update tasks/main.yml to get this to pass.
---
# tasks file for ntp
- name: ensure ntp package is installed
package:
name: ntp
state: present
tags: [ntp]
- name: set ntp service name
set_fact:
ntp_service: "{{ (ansible_os_family == 'Debian') | ternary('ntp','ntpd') }}"
tags: [ntp]
- name: ensure ntp is running and enabled
service:
name: "{{ ntp_service }}"
state: started
enabled: yes
tags: [ntp]
Now if we run both the tests they are green. Yay!
The end
Clearly we could do more things with this role. For example we could configure ntp, and perhaps I will add to the role when molecule v2 is officially released; however, this should get you started testing your infrastructure with molecule.
You can follow the code here