Ansible as remote executor in a Puppet environment - codecentric AG Blog

:

When you are using Puppet you might know this problem:
How can I execute arbitrary commands on all or some of my Puppet nodes?

In this article, I explain how you can do so with Ansible. Ansible it another configuration management tool like Puppet and Chef. My colleague Daniel Schneller gives a great introduction in his article “Ansible, simple yet powerful automation”. There are already a lot of comparisons of Puppet and Ansible available for example [1] and [2]. In this post I focus on the combination of Ansible and Puppet.

An often encountered problem with Puppet is that it does not provide a remote executor by itself but relies on mcollective. Ansible’s advantage over Puppet is that it does not need an agent on each server. It uses ssh to execute commands on remote hosts. In order for this to work, Ansible needs an inventory enumerating all the servers it shall manage. Taking a look at the Ansible documentation, an inventory is usually a file. The simplest form is just a list of hostnames:

web01.example.com
web02.example.com
db01.example.com
db02.example.com
...

web01.example.com web02.example.com db01.example.com db02.example.com ...

In order to facilitate Ansible to run Puppet commands, we need to create such an inventory. As Puppet knows all the servers it manages, it can help us to create the inventory. Puppet uses certificates to secure the communication between Puppet agent and server. It stores these certificates on its master and you can access this information. For example, the output of puppet cert list --all looks like this:

+ "web01.example.com" (SHA256) 11:AB:C5:A3:7D:7B:90:40:57:FC:03:97:D7:73:07:E3:F8:D1:F5:81:3C:80:C0:4A:B2:F4:69:A7:BC:85:C8:59
+ "web02.example.com" (SHA256) 9A:84:F1:14:19:6B:BC:65:4D:29:C9:31:C1:2A:6A:BD:DC:7A:DD:28:09:42:C1:23:59:4F:9C:31:DF:DC:3B:D5
+ "db01.example.com"  (SHA256) D2:00:26:F8:3F:9B:19:CD:1F:AB:D2:74:1A:93:35:11:9C:AB:1E:C2:50:4A:72:A7:81:82:5E:83:AB:3C:50:EF
+ "db01.example.com"  (SHA256) FD:A2:F1:F1:5D:0C:1A:6B:02:3A:66:FA:51:C3:DF:88:2B:83:95:D8:BB:4C:11:CB:50:82:BF:C8:CA:26:3B:90
...

+ "web01.example.com" (SHA256) 11:AB:C5:A3:7D:7B:90:40:57:FC:03:97:D7:73:07:E3:F8:D1:F5:81:3C:80:C0:4A:B2:F4:69:A7:BC:85:C8:59 + "web02.example.com" (SHA256) 9A:84:F1:14:19:6B:BC:65:4D:29:C9:31:C1:2A:6A:BD:DC:7A:DD:28:09:42:C1:23:59:4F:9C:31:DF:DC:3B:D5 + "db01.example.com"  (SHA256) D2:00:26:F8:3F:9B:19:CD:1F:AB:D2:74:1A:93:35:11:9C:AB:1E:C2:50:4A:72:A7:81:82:5E:83:AB:3C:50:EF + "db01.example.com"  (SHA256) FD:A2:F1:F1:5D:0C:1A:6B:02:3A:66:FA:51:C3:DF:88:2B:83:95:D8:BB:4C:11:CB:50:82:BF:C8:CA:26:3B:90 ...

There you see a list of all managed hosts.

The next step is to convert the output of puppet cert to an Ansible inventory. The simplest way is to convert the output into a corresponding Ansible inventory You can do this with a bash command:

puppet cert list --all | egrep "^\+.*" | awk-F"\"" '{ print $2 }' > /etc/ansible/puppet_hosts

puppet cert list --all | egrep "^\+.*" | awk-F"\"" '{ print $2 }' > /etc/ansible/puppet_hosts

You can use the created file as inventory for Ansible:

ansible “all” -i /etc/ansible/puppet_hosts -a "uptime"

ansible “all” -i /etc/ansible/puppet_hosts -a "uptime"

This command will produce an output like this:

web01.example.com | success | rc=0 >>
15:27:00 up 238 days, 21:20,  1 user,  load average: 0.58, 0.60, 0.63
 
web02.example.com | success | rc=0 >>
15:27:00 up 176 days, 22:07,  1 user,  load average: 0.83, 0.74, 0.73
 
db01.example.com | success | rc=0 >>
15:27:00 up 310 days,  5:51,  1 user,  load average: 0.00, 0.00, 0.00
 
db02.example.com | success | rc=0 >>
15:27:00 up 210 days, 10 min,  1 user,  load average: 1.19, 0.89, 0.77
…

web01.example.com | success | rc=0 >> 15:27:00 up 238 days, 21:20,  1 user,  load average: 0.58, 0.60, 0.63web02.example.com | success | rc=0 >> 15:27:00 up 176 days, 22:07,  1 user,  load average: 0.83, 0.74, 0.73db01.example.com | success | rc=0 >> 15:27:00 up 310 days,  5:51,  1 user,  load average: 0.00, 0.00, 0.00db02.example.com | success | rc=0 >> 15:27:00 up 210 days, 10 min,  1 user,  load average: 1.19, 0.89, 0.77 …

Ansible calls such an inventory a static inventory. As your managed hosts will change over time you want to have something more dynamic. Therefore, Ansible also supports so called dynamic inventories. To use a dynamic inventory, just replace the inventory file from above with an executable, e.g. a script. The dynamic inventory script in our case looks like this:

#!/bin/bash
 
/usr/bin/puppet cert list --all > /dev/null 2>&1
if [ $? -ne 0 ]
then
  echo "Puppet error"
  exit 1
fi
 
HOSTS="{\n\t\"all\":\n\t\t[\n"
FIRST_HOST="true"
 
while read hostname
do
  # this way i don't get a trailing comma
  if [ "${FIRST_HOST}" == "true" ]
  then
    HOSTS="${HOSTS}\t\t\t\"${hostname}\""
    unset FIRST_HOST
  else
    HOSTS="${HOSTS}, \"${hostname}\""
  fi
done < <(/usr/bin/puppet cert list --all | egrep "^\+.*" | awk -F"\"" '{ print $2 }' | grep "example.com")
 
HOSTS="${HOSTS}\n\t\t]\n}"
 
echo -e "${HOSTS}"

#!/bin/bash/usr/bin/puppet cert list --all > /dev/null 2>&1 if [ $? -ne 0 ] then echo "Puppet error" exit 1 fiHOSTS="{\n\t\"all\":\n\t\t[\n" FIRST_HOST="true"while read hostname do # this way i don't get a trailing comma if [ "${FIRST_HOST}" == "true" ] then HOSTS="${HOSTS}\t\t\t\"${hostname}\"" unset FIRST_HOST else HOSTS="${HOSTS}, \"${hostname}\"" fi done < <(/usr/bin/puppet cert list --all | egrep "^\+.*" | awk -F"\"" '{ print $2 }' | grep "example.com")HOSTS="${HOSTS}\n\t\t]\n}"echo -e "${HOSTS}"

This script creates one long list of hosts with the help of Puppet and prints this list in Ansible compatible JSON to standard output:

{
  "all":
    [
       "web01.example.com", "web02.example.com", "db01.example.com", "db02.example.com", ...
    ]
}

{ "all": [ "web01.example.com", "web02.example.com", "db01.example.com", "db02.example.com", ... ] }

For a broader introduction to Dynamic Inventories and another example take a look at this post by my colleague Lukas Pustina. He describes how to use Ansible features like host facts and host groups with dynamic inventories.

With the dynamic inventory script, the command shown above changes to:

ansible “all” -i /usr/local/bin/puppet_dyn_hosts -a "uptime"

ansible “all” -i /usr/local/bin/puppet_dyn_hosts -a "uptime"

The Ansible command converts the current Puppet certificate list and executes the command on the selected hosts:

web01.example.com | success | rc=0 >>
15:27:00 up 238 days, 21:20,  1 user,  load average: 0.58, 0.60, 0.63
 
web02.example.com | success | rc=0 >>
15:27:00 up 176 days, 22:07,  1 user,  load average: 0.83, 0.74, 0.73
 
db01.example.com | success | rc=0 >>
15:27:00 up 310 days,  5:51,  1 user,  load average: 0.00, 0.00, 0.00
 
db02.example.com | success | rc=0 >>
15:27:00 up 210 days, 10 min,  1 user,  load average: 1.19, 0.89, 0.77
…

web01.example.com | success | rc=0 >> 15:27:00 up 238 days, 21:20,  1 user,  load average: 0.58, 0.60, 0.63web02.example.com | success | rc=0 >> 15:27:00 up 176 days, 22:07,  1 user,  load average: 0.83, 0.74, 0.73db01.example.com | success | rc=0 >> 15:27:00 up 310 days,  5:51,  1 user,  load average: 0.00, 0.00, 0.00db02.example.com | success | rc=0 >> 15:27:00 up 210 days, 10 min,  1 user,  load average: 1.19, 0.89, 0.77 …

This command is just an example. You can replace uptime with any other command. When you have a more complex task at hand, you can even write Ansible playbooks and execute them on the hosts.

Here is an example where you first stop a service, change file permissions and start the service again:

---
- name: Updates splunkforwarder
  hosts: all
  tasks:
  - name: Stop splunk to change uid
    sudo: yes
    service: name=splunk state=stopped

  - name: Change file permissions
    sudo: yes
    file: path=/opt/splunkforwarder owner=splunk group=splunk recurse=yes

  - name: Start splunk
    sudo: yes
    service: name=splunk state=started

--- - name: Updates splunkforwarder hosts: all tasks: - name: Stop splunk to change uid sudo: yes service: name=splunk state=stopped- name: Change file permissions sudo: yes file: path=/opt/splunkforwarder owner=splunk group=splunk recurse=yes- name: Start splunk sudo: yes service: name=splunk state=started

You can even integrate calls to Puppet agent into your Ansible playbooks, but this will be the topic of another post.

When you have more questions about how to combine Ansible and Puppet, do not hesitate to contact me.