When you use Ansible to provision hosts, running a task on a set of hosts in parallel is not a problem. You simply use the `serial` keyword in a playbook. See: http://docs.ansible.com/ansible/latest/playbooks_delegation.html#rolling-update-batch-size
What about when you want to run a task several times in parallel, either on each host, or on a single host? A common use case is when you want to provision infrastructure at a cloud provider, such as Amazon Web Services (AWS). In that case, you normally run the task on localhost and the task just calls out to the cloud provider API.
The reason you might want to do these calls in parallel is because creating cloud resources sometimes takes a long time. AWS EC2 instances and RDS instances are examples of some resources which take a long time to create. If you want to create several, you can use a loop and wait for each to finish, but that takes a long time. As a alternative, you can create them all in parallel.
Here is a example Ansible play of how you can run any Ansible task in parallel and wait for them all to finish.
---
- name: Run tasks in parallel
hosts: localhost
connection: local
gather_facts: no
tasks:
- name: Pretend to create instances
command: "sleep {{ item }}" # Instead of calling a long running operation at a cloud provider, we just sleep.
with_items:
- 6
- 8
- 7
register: _create_instances
async: 600 # Maximum runtime in seconds. Adjust as needed.
poll: 0 # Fire and continue (never poll)
- name: Wait for creation to finish
async_status:
jid: "{{ item.ansible_job_id }}"
register: _jobs
until: _jobs.finished
delay: 5 # Check every 5 seconds. Adjust as you like.
retries: 10 # Retry up to 10 times. Adjust as needed.
with_items: "{{ _create_instances.results }}"
The output will be:
PLAY [Run tasks in parallel] **********************************************************************************************************
TASK [Pretend to create instances] ****************************************************************************************************
changed: [localhost] => (item=6)
changed: [localhost] => (item=8)
changed: [localhost] => (item=7)
TASK [Wait for creation to finish] ****************************************************************************************************
FAILED - RETRYING: Wait for creation to finish (10 retries left).
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'957883465297.7487', 'item': 6, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/957883465297.7487'})
FAILED - RETRYING: Wait for creation to finish (10 retries left).
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'450659566541.7513', 'item': 8, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/450659566541.7513'})
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'763185500456.7538', 'item': 7, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/763185500456.7538'})
PLAY RECAP ****************************************************************************************************************************
localhost : ok=3 changed=3 unreachable=0 failed=0
See the example code at GitHub.
One of the plays runs the tasks in parallel, and one runs then in sequence, for comparison.
In the pursuit to automate testing to create faster feedback loops and build quality in, some teams forget the value of manual testing. In my experience, without manual testing (as well) we are toast.
How should you structure your Ansible variables in playbooks calling cloud modules? Ansible has extensive support for variables at different levels (there are 21 different levels!).
