Containers and IaC
Ansible
Inventory and variable precedence, idempotent task patterns, check mode, and debugging a play that does the wrong thing.
Cheatsheet #
| Task | Command |
|---|---|
| Dry run with diffs | ansible-playbook site.yml --check --diff |
| One host only | ansible-playbook site.yml --limit web-01 |
| Syntax check | ansible-playbook site.yml --syntax-check |
| Which hosts would run | ansible-playbook site.yml --list-hosts |
| Which tasks would run | ansible-playbook site.yml --list-tasks |
| Start partway through | ansible-playbook site.yml --start-at-task='Install nginx' |
| Step through | ansible-playbook site.yml --step |
| Ad-hoc command | ansible web -m command -a 'uptime' |
| Gather facts only | ansible web -m setup |
| One fact | ansible web -m setup -a 'filter=ansible_distribution*' |
| Verbose connection debug | ansible-playbook site.yml -vvvv |
| Extra variables | ansible-playbook site.yml -e env=prod -e @vars.yml |
| Inventory as resolved | ansible-inventory --list --yaml |
| Vault edit | ansible-vault edit group_vars/prod/secrets.yml |
| Lint | ansible-lint site.yml |
Dry run first #
--check runs every task in prediction mode and --diff prints the file changes it would make. Modules that shell out (command, shell) cannot predict, so they skip unless given check_mode: false.
ansible-playbook site.yml --check --diff --limit staging
ansible-playbook site.yml --check --diff --tags nginxchanged in check mode is a claim, not a promise: a task whose when depends on a fact that a skipped task would have set can report differently on the real run.
Inventory #
# inventory/prod.ini
[web]
web-[01:04].example.internal
[db]
db-01.example.internal ansible_host=10.0.5.11
[prod:children]
web
db
[prod:vars]
ansible_user=ops
ansible_python_interpreter=/usr/bin/python3# inventory/prod.yml — the YAML form, better for nested group vars
all:
children:
web:
hosts:
web-01.example.internal:
web-02.example.internal:
vars:
nginx_workers: 4ansible-inventory -i inventory/prod.yml --graph
ansible-inventory -i inventory/prod.yml --host web-01.example.internal
ansible all -i inventory/prod.yml -m ping --limit webDynamic inventory plugins (aws_ec2, k8s, azure_rm) replace static files for cloud fleets; the plugin config is a YAML file ending in the plugin’s name, such as aws_ec2.yml.
Variable precedence #
Later wins. The full list is long; these are the levels that actually come up:
| Source | Precedence |
|---|---|
Role defaults (defaults/main.yml) | Lowest — meant to be overridden |
| Inventory group vars | Low |
group_vars/all then group_vars/<group> | Low, more specific group wins |
host_vars/<host> | Above group vars |
Play vars: | Above inventory |
Task vars: | Above play |
Role vars/main.yml | High — hard to override, use sparingly |
include_vars | High, at the point it runs |
--extra-vars / -e | Highest, always wins |
Put tunables in defaults/, environment facts in group_vars/, and reserve vars/ for constants the role’s own tasks depend on.
Playbook shape #
- name: Configure web tier
hosts: web
become: true
gather_facts: true
serial: "25%" # rolling, a quarter of the group at a time
max_fail_percentage: 0
vars:
nginx_version: "1.27.*"
pre_tasks:
- name: Drain from the load balancer
ansible.builtin.uri:
url: "http://lb.internal/drain/{{ inventory_hostname }}"
method: POST
delegate_to: localhost
roles:
- role: nginx
tags: [nginx]
tasks:
- name: Deploy configuration
ansible.builtin.template:
src: nginx.conf.j2
dest: /etc/nginx/nginx.conf
owner: root
mode: "0644"
validate: nginx -t -c %s # never install a config that fails its own check
notify: reload nginx
handlers:
- name: reload nginx
ansible.builtin.service:
name: nginx
state: reloadedserial plus max_fail_percentage: 0 is the rollout safety net: the first batch that fails stops the play before it reaches the rest of the fleet.
Handlers run once, at the end of the play, only if notified. meta: flush_handlers forces them earlier when a later task depends on the restart.
Writing idempotent tasks #
Use the module that describes state, not the command that changes it. command and shell always report changed unless you tell them how to know better.
- name: Package present
ansible.builtin.package:
name: "nginx={{ nginx_version }}"
state: present
- name: Line in a config file
ansible.builtin.lineinfile:
path: /etc/security/limits.conf
regexp: '^\*\s+nofile'
line: '* nofile 65535'
- name: Run a migration exactly once
ansible.builtin.command: /opt/app/bin/migrate
args:
creates: /var/lib/app/.migrated # skip if this exists
- name: Shell with an explicit change test
ansible.builtin.shell: /usr/local/bin/sync-data
register: sync
changed_when: "'updated' in sync.stdout"
failed_when: sync.rc not in [0, 2]| Guard | Effect |
|---|---|
creates: / removes: | Skip when a path exists or does not |
changed_when: | Define what counts as a change |
failed_when: | Define what counts as a failure |
check_mode: false | Run a read-only command even under --check |
run_once: true | Execute on the first host only |
delegate_to: | Run this task somewhere else (load balancer, database host) |
until:/retries:/delay: | Wait for convergence |
throttle: | Limit concurrency for a rate-limited API |
Templates and facts #
# nginx.conf.j2
worker_processes {{ ansible_processor_vcpus }};
upstream app {
{% for host in groups['app'] %}
server {{ hostvars[host]['ansible_default_ipv4']['address'] }}:8080;
{% endfor %}
}ansible web-01 -m setup | jq '.ansible_facts | keys'
ansible web-01 -m setup -a 'filter=ansible_mounts'
ansible-playbook site.yml -e '{"debug_facts": true}' --tags facts- name: What is this value, actually
ansible.builtin.debug:
var: nginx_workers
verbosity: 1 # only with -vFact gathering costs a round trip per host; gather_facts: false plus setup with gather_subset where facts are actually needed is the usual optimisation on large fleets.
Secrets #
ansible-vault create group_vars/prod/secrets.yml
ansible-vault edit group_vars/prod/secrets.yml
ansible-vault encrypt_string 's3cret' --name db_password
ansible-playbook site.yml --vault-password-file ~/.vault-passVault-encrypted values decrypt into memory during a run and can leak through debug, -vvv output and failed-task dumps. Mark sensitive tasks no_log: true.
Troubleshooting #
| Symptom | Cause |
|---|---|
UNREACHABLE | SSH, not Ansible: test with ssh -v, check ansible_user and key |
| Task always reports changed | command/shell without creates or changed_when |
| Variable is not what you expect | Precedence — dump it with debug: var= at the point of use |
| Works ad-hoc, fails in a play | Different become, environment or interpreter |
The module failed to execute correctly | Wrong Python on the target — set ansible_python_interpreter |
| Handler never runs | The notifying task reported ok, not changed |
| Slow runs | Fact gathering, no pipelining, low forks |
# ansible.cfg
[defaults]
inventory = inventory/prod.yml
forks = 25
host_key_checking = True
stdout_callback = yaml
callbacks_enabled = profile_tasks
[ssh_connection]
pipelining = True
ssh_args = -o ControlMaster=auto -o ControlPersist=60spipelining = True removes a file transfer per task and is the single biggest speed-up; it requires requiretty to be off in sudoers, which is the default on modern distributions.
Oneliners #
# Reachability across the fleet
ansible all -m ping -o | grep -v SUCCESS
# Which hosts are running which OS version
ansible all -m setup -a 'filter=ansible_distribution_version' -o | sed 's/ | SUCCESS.*"ansible_distribution_version": "/ /;s/".*//'
# Check a package version everywhere
ansible web -m command -a 'nginx -v' -o
# Run a playbook against one host and show only changes
ansible-playbook site.yml --limit web-01 --diff | grep -E '^changed|^--- |^\+\+\+ '
# Tasks sorted by duration (needs profile_tasks)
ansible-playbook site.yml 2>&1 | grep -E '^[a-z_]+ -+ [0-9.]+s' | sort -k3 -rn | head
# Find every task that uses shell or command
grep -rnE 'ansible\.builtin\.(shell|command)|^\s+(shell|command):' roles/ | grep -v creates
# Which variables a group resolves to
ansible-inventory --host web-01.example.internal | jq 'keys'
# Encrypt an existing plaintext vars file in place
ansible-vault encrypt group_vars/prod/secrets.yml
# Confirm no secrets are readable in the repository
grep -rLn '\$ANSIBLE_VAULT' group_vars/*/secrets.yml
# Apply a single role without a playbook file
ansible web -m include_role -a 'name=nginx' --become
# Restart a service in batches of two with a pause
ansible-playbook restart.yml --forks 2 -e 'pause_seconds=10'