"The Foreman" - Automatic installation of "The Foreman" with Puppet - codecentric AG Blog

:

This is the second blog post in a series about “The Foreman”, a complete lifecycle management tool for physical and virtual servers. In the first post we have learned what “The Foreman” is made for, how it works and what we can use it for.
Now with the second post we will start to get more hands on. The goal of this post is to show how to install “The Foreman” and supporting services like for example DNS, DHCP automatically through puppet on a bare-metal server.

The network setup used for this post is shown in figure 1. First of all, the network contains a VPN-Gateway connected to the internet. The backend network is secured through a DMZ and consists of two VLANs. On the VLAN 1, “The Foreman” will provide the DHCP-, DNS- and TFTP-Services. Through the network VLAN 2 it is ensured, that all machines can be accessed through the VPN.

Figure 1: Network setup

massem_foreman_4
To provision the Host A and B we need to install “The Foreman” on the one machine called “The Foreman” in figure 1. Therefore we have to do the following preparation work on this machine:

  • Installation of operating system.
  • Installation of prerequisites.
  • Get and run a provided Puppet Manifest to install “The Foreman”.

Now that we know the setup, lets prepare the machine we want to install “The Foreman” on. First of all, we have to install the Ubuntu 12.04 Server Edition. We decided to use Ubuntu 12.04 Server because it comes with long term support and is supported by “The Foreman”.

For this post we assume that you have already installed the OS and know some basics about Puppet. After we have finished the OS installation, we need to run a post installation script which you can see in listing 1. The script will install an openssh server for remote access as well as git. Further, the script creates a folder named git and clones a provided git repository. Thereafter, the script is going to copy a prepared network interface file, through which both network interfaces gets configured. Finally the script installs puppet and provides a prepared puppet configuration file, that will then be used to install the needed services like DHCP and DNS and “The Foreman”.

Listing 1:

#!/bin/bash
 
# Install an openssh server and git
cd $HOME
sudo apt-get install openssh-server git
mkdir -p git
cd git
 
# Checkout the git repository
if [ ! -d "$HOME/git/foreman-poc" ]; then
git clone https://github.com/codecentric/foreman-poc.git
fi
 
# Change the branch to bare-metal
cd foreman-poc
git checkout bare_metal
 
# Prepare the network interfaces
# You have to change this file depending on your network setup
sudo cp $HOME/git/foreman-poc/files/System/interfaces /etc/network/
 
# Get Puppet debian packages and install Puppet
wget https://apt.puppetlabs.com/puppetlabs-release-precise.deb
sudo dpkg -i puppetlabs-release-precise.deb
sudo apt-get update
sudo apt-get install --yes puppet
 
# Provide a new Puppet configuration, restart the service and get the standard libs
sudo cp $HOME/git/foreman-poc/files/System/puppet.conf /etc/puppet/
sudo service puppet restart
sudo puppet module install --force puppetlabs-stdlib
 
rm puppetlabs-release-precise.deb
 
# Reboot the system
sudo reboot

#!/bin/bash# Install an openssh server and git cd $HOME sudo apt-get install openssh-server git mkdir -p git cd git# Checkout the git repository if [ ! -d "$HOME/git/foreman-poc" ]; then git clone https://github.com/codecentric/foreman-poc.git fi# Change the branch to bare-metal cd foreman-poc git checkout bare_metal# Prepare the network interfaces # You have to change this file depending on your network setup sudo cp $HOME/git/foreman-poc/files/System/interfaces /etc/network/# Get Puppet debian packages and install Puppet wget https://apt.puppetlabs.com/puppetlabs-release-precise.deb sudo dpkg -i puppetlabs-release-precise.deb sudo apt-get update sudo apt-get install --yes puppet# Provide a new Puppet configuration, restart the service and get the standard libs sudo cp $HOME/git/foreman-poc/files/System/puppet.conf /etc/puppet/ sudo service puppet restart sudo puppet module install --force puppetlabs-stdlibrm puppetlabs-release-precise.deb# Reboot the system sudo reboot

The following commands are needed to download and execute the post installation script:

> wget https://github.com/codecentric/foreman-poc/blob/bare_metal/files/System/post-install.sh
> chmod +x post-install.sh
> sudo ./post-install.sh

Now we we are ready to install “The Foreman”. However, before we start a local puppet run, lets have a look on the the resources described inside the Puppet Manifest called ‘server.pp’.
We will have a look on some of the important parts of the preparation and installation itself. To get started with the “The Foreman” installation, we first of all need some sources and packages. Therefore we have to provide the apt key and apt sources to be ready to find and install “The Foreman” specific packages, see listing 2.

Listing 2:

# Function to create a apt key. If called with ensure => present the apt key will be added 
define aptkey($ensure, $apt_key_url = 'http://deb.theforeman.org') {
  case $ensure {
    'present': {
      exec { "apt-key present $name":
	command => "/usr/bin/wget -q $apt_key_url/$name -O -|/usr/bin/apt-key add -",
	unless  => "/usr/bin/apt-key list|/bin/grep -c $name",
      }
    }
    'absent': {
      exec { "apt-key absent $name":
	command => "/usr/bin/apt-key del $name",
	onlyif  => "/usr/bin/apt-key list|/bin/grep -c $name",
      }
    }
    default: {
      fail "Invalid 'ensure' value '$ensure' for apt::key"
    }
  }
}
 
# Creates a file for the apt source
file {'foremanlist':
	path	=> '/etc/apt/sources.list.d/foreman.list',
	ensure	=> present,
	mode	=> 0644,
	content	=> 'deb http://deb.theforeman.org/ precise 1.4'
}
 
# Calls the aptkey function above with name => ‘foreman.asc’ and ensure => present 
aptkey { 'foreman.asc':
	ensure	=> present
}
# Call apt-get update which requires the apt key and source file
exec { "apt-update":
	command	=> "/usr/bin/apt-get update",
	require	=> [
		Aptkey['foreman.asc'],
		File['foremanlist'],
	]
}

# Function to create a apt key. If called with ensure => present the apt key will be added define aptkey($ensure, $apt_key_url = 'http://deb.theforeman.org') { case $ensure { 'present': { exec { "apt-key present $name": command => "/usr/bin/wget -q $apt_key_url/$name -O -|/usr/bin/apt-key add -", unless => "/usr/bin/apt-key list|/bin/grep -c $name", } } 'absent': { exec { "apt-key absent $name": command => "/usr/bin/apt-key del $name", onlyif => "/usr/bin/apt-key list|/bin/grep -c $name", } } default: { fail "Invalid 'ensure' value '$ensure' for apt::key" } } }# Creates a file for the apt source file {'foremanlist': path => '/etc/apt/sources.list.d/foreman.list', ensure => present, mode => 0644, content => 'deb http://deb.theforeman.org/ precise 1.4' }# Calls the aptkey function above with name => ‘foreman.asc’ and ensure => present aptkey { 'foreman.asc': ensure => present } # Call apt-get update which requires the apt key and source file exec { "apt-update": command => "/usr/bin/apt-get update", require => [ Aptkey['foreman.asc'], File['foremanlist'], ] }

In a next step, we have to ensure that all needed packages are present – including the foreman-installer, bind9 for dns, isc-dhcp-server and gem, see listing 3.

Listing 3:

# Ensures that “The Foreman” installer is present
package { "foreman-installer":
	ensure	=> "installed",
	require	=> Exec['apt-update'],
}
# Ensures DNS-Server is present
package { "bind9":
	ensure	=> "installed",
	require	=> Exec['apt-update'],
}
# Ensures DHCP-Server is present
package { "isc-dhcp-server":
	ensure	=> "installed",
	require	=> Exec['apt-update'],
}
# Ensures Gem is present
package { "gem":
	ensure => "installed",
	require => Exec['apt-update'],
}

# Ensures that “The Foreman” installer is present package { "foreman-installer": ensure => "installed", require => Exec['apt-update'], } # Ensures DNS-Server is present package { "bind9": ensure => "installed", require => Exec['apt-update'], } # Ensures DHCP-Server is present package { "isc-dhcp-server": ensure => "installed", require => Exec['apt-update'], } # Ensures Gem is present package { "gem": ensure => "installed", require => Exec['apt-update'], }

Now that we have all packages at hand, lets have a detailed look at the DHCP and DNS configuration.
In listing 4, with the first resource we are placing a ‘rndc.key’ file into the config folder of the DNS-Server. This allows us secured server to server communication by providing the key also to the installation file used by “The Foreman”. The second resource creates a user ‘dhcpd’ and adds it to the group ‘bind’. Now we have to provide a new apparmor config file due to a missing write permission on the folder “etc/bind”, see the resource 4. Next, we have to add a line to the dhclient.conf, to ensure that the correct DNS-Server will be called by the server, resource 5. Finally we have to provide a modified proxydhcp.pp Manifest in which we added the correct path to the ‘rndc.key’ file.

As I am not an expert for linux administration, there are probably better solutions for setting up DNS and DHCP and I would like to see them. So feel free to add a comment below. However the described solution worked for us and we have a stable DHCP-Server which updates the DNS entries.

Listing 4:

# Placing the keyfile
file { "/etc/bind/rndc.key":
	ensure	=> present,
	source	=> "/home/server/git/foreman-poc/files/BIND/rndc.key",
	owner	=> root,
	group	=> bind,
	mode	=> 640,
	require	=> Package["bind9"],
}
# Adding user 'dhcpd' to group 'bind', as this users needs to read the keyfile
user { "dhcpd":
	ensure	=> present,
	groups	=> ['bind'],
	require => [
		Package["isc-dhcp-server"],
		Package["bind9"],
	],
}
# Workaround that DHCP can read the keyfile
# Replace existing DHCPd-apparmor configuration
service { "apparmor":
    ensure  => "running",
    enable  => "true",
}
file { "/etc/apparmor.d/usr.sbin.dhcpd":
	notify  => Service["apparmor"],
	ensure	=> present,
	owner	=> root,
	group	=> root,
	mode	=> 644,
	source	=> "/home/server/git/foreman-poc/files/DHCP/apparmor_usr.sbin.dhcpd",
	require => Package["isc-dhcp-server"],
}
# Dhclient: prepend DNS-server
file_line { 'dhclient':
	path	=> '/etc/dhcp/dhclient.conf',
	line	=> 'prepend domain-name-servers 172.16.0.2;',
	match	=> "prepend domain-name-servers",
}
# Modifying foreman-installer to support DDNS
file { "/usr/share/foreman-installer/modules/foreman_proxy/manifests/proxydhcp.pp":
	ensure	=> present,
	source	=> "/vagrant/files/DHCP/proxydhcp.pp",
	owner	=> root,
	group	=> root,
	mode	=> 644,
	require	=> Package["foreman-installer"],
}

# Placing the keyfile file { "/etc/bind/rndc.key": ensure => present, source => "/home/server/git/foreman-poc/files/BIND/rndc.key", owner => root, group => bind, mode => 640, require => Package["bind9"], } # Adding user 'dhcpd' to group 'bind', as this users needs to read the keyfile user { "dhcpd": ensure => present, groups => ['bind'], require => [ Package["isc-dhcp-server"], Package["bind9"], ], } # Workaround that DHCP can read the keyfile # Replace existing DHCPd-apparmor configuration service { "apparmor": ensure => "running", enable => "true", } file { "/etc/apparmor.d/usr.sbin.dhcpd": notify => Service["apparmor"], ensure => present, owner => root, group => root, mode => 644, source => "/home/server/git/foreman-poc/files/DHCP/apparmor_usr.sbin.dhcpd", require => Package["isc-dhcp-server"], } # Dhclient: prepend DNS-server file_line { 'dhclient': path => '/etc/dhcp/dhclient.conf', line => 'prepend domain-name-servers 172.16.0.2;', match => "prepend domain-name-servers", } # Modifying foreman-installer to support DDNS file { "/usr/share/foreman-installer/modules/foreman_proxy/manifests/proxydhcp.pp": ensure => present, source => "/vagrant/files/DHCP/proxydhcp.pp", owner => root, group => root, mode => 644, require => Package["foreman-installer"], }

Finally we need two more things for the installation of “The Foreman”. First, the installation itself should be unattended. Therefore we need a file called ‘answers.yaml’ that includes the answers to question that are normally asked during the installation process. You can see the first resource in listing 5 how to provide the file. Second, the last resource in listing 5 does nothing else than just start the installation process itself.

Listing 5:

# Options for foreman-installer
file { "/usr/share/foreman-installer/config/answers.yaml":
	ensure	=> present,
	source	=> "/vagrant/files/Foreman/answers.yaml",
	owner	=> root,
	group	=> root,
	mode	=> 600,
	require	=> Package["foreman-installer"],
}
# Installation of foreman
exec { 'foreman-installer':
	command	=> "/usr/bin/foreman-installer",
	timeout => 0,
	require => [
		Package["bind9"],
		File['/usr/share/foreman-installer/modules/foreman_proxy/manifests/proxydhcp.pp'],
		File['/usr/share/foreman-installer/config/answers.yaml'],
		File["/etc/bind/rndc.key"],
	],
}

# Options for foreman-installer file { "/usr/share/foreman-installer/config/answers.yaml": ensure => present, source => "/vagrant/files/Foreman/answers.yaml", owner => root, group => root, mode => 600, require => Package["foreman-installer"], } # Installation of foreman exec { 'foreman-installer': command => "/usr/bin/foreman-installer", timeout => 0, require => [ Package["bind9"], File['/usr/share/foreman-installer/modules/foreman_proxy/manifests/proxydhcp.pp'], File['/usr/share/foreman-installer/config/answers.yaml'], File["/etc/bind/rndc.key"], ], }

As I mentioned before, for the installation to run unattended, a file called ‘answers.yaml’ is needed. In listing 6 you can see the full ‘answers.yaml’ file used by our installation. The file tells the installation process to install “The Foreman” with a custom environment called ‘cloudbox’, line 3. Environments are used to group Puppet Modules for different kind of hosts. Further environments could be for example production, test or development.
Foreman-Proxies will be installed for TFTP, DHCP and DNS. These Foreman-Proxies will be installed on top of our before installed TFTP-, DNS-, and DHCP-Servers. While installing the Foreman-Proxies we can configure many details like the rndc-key and secret. Finally the listing 6 shows, that we are installing the Puppet-Master with the same environment ‘cloudbox’ as we did before.

Listing 6:

---
foreman:
  environment: cloudbox
  custom_repo: true
  oauth_consumer_key: Ls6P7vd3sfv8QZviRTNnPUX2k5RPhTnn
  oauth_consumer_secret: Uq2Hwyp7kHuSB7YGU3beMXcTKuyA9VaD
foreman_proxy:
  custom_repo: true
  puppetrun: true
  tftp: true
  tftp_servername: 172.16.0.2
  dhcp: true
  dhcp_managed: true
  dhcp_interface: eth2
  dhcp_gateway: 172.16.0.2
  dhcp_range: 172.16.0.16 172.16.0.255
  dhcp_nameservers: 172.16.0.2
  dhcp_key_name: rndc-key
  dhcp_key_secret: bQR3x3fquV+YjZ+aChpfJQ==
  dns: true
  dns_interface: eth2
  dns_zone: local.cloud
  dns_reverse: 0.16.172.in-addr.arpa
  dns_server: 172.16.0.2
  dns_forwarders: 8.8.8.8
  foreman_base_url: http://server.local.cloud
  oauth_consumer_key: Ls6P7vd3sfv8QZviRTNnPUX2k5RPhTnn
  oauth_consumer_secret: Uq2Hwyp7kHuSB7YGU3beMXcTKuyA9VaD
puppet:
  server: true
  server_environments:
      - cloudbox

--- foreman: environment: cloudbox custom_repo: true oauth_consumer_key: Ls6P7vd3sfv8QZviRTNnPUX2k5RPhTnn oauth_consumer_secret: Uq2Hwyp7kHuSB7YGU3beMXcTKuyA9VaD foreman_proxy: custom_repo: true puppetrun: true tftp: true tftp_servername: 172.16.0.2 dhcp: true dhcp_managed: true dhcp_interface: eth2 dhcp_gateway: 172.16.0.2 dhcp_range: 172.16.0.16 172.16.0.255 dhcp_nameservers: 172.16.0.2 dhcp_key_name: rndc-key dhcp_key_secret: bQR3x3fquV+YjZ+aChpfJQ== dns: true dns_interface: eth2 dns_zone: local.cloud dns_reverse: 0.16.172.in-addr.arpa dns_server: 172.16.0.2 dns_forwarders: 8.8.8.8 foreman_base_url: http://server.local.cloud oauth_consumer_key: Ls6P7vd3sfv8QZviRTNnPUX2k5RPhTnn oauth_consumer_secret: Uq2Hwyp7kHuSB7YGU3beMXcTKuyA9VaD puppet: server: true server_environments: - cloudbox

If you like to have a look on the final Puppet Manifest, with the full installation for example the TFTP-Server and net boot images which will be provided for the provisioning, you can check it out using following Link.

After the puppet run succeeded, we have to find out our local IP-address (ifconfig) and open a web browser on our local machine. Now enter the IP-address and don’t forget to put https:// in front. The final address should look like this (https://yourlocalipaddress) and if everything worked out correct you should be able to see the login screen. Enter admin as username and changeme as password and you should be able to see the figure 2.

Figure 2: “The Foreman”: Start screen

Auswahl_074
Now we have a Foreman-Server, fully installed and configured with DHCP-, DNS-, TFTP- and PXE-Services, ready to use.

So, thats it for this blog post. I hope you enjoyed the second post. Now lets recap what we have done up to now. On a bare-metal machine we have installed an OS. Afterwards, we downloaded a post installation script which prepared our machine to run puppet and git. The preparation script has cloned a repository from github and started a local puppet run. This puppet run has automatically installed a DNS-, DHCP- and TFTP-Server as well as “The Foreman” itself. We had a closer look on the Puppet Manifest for the installation of “The Foreman” itself and the installation was again unattended through the use of a file called ‘answers.yaml’. Finally we checked if the installation was successful.
If you enjoyed what you have seen up to now. Get a feeling for “The Foremans” web user interface and remember about the chapters “Provisioning setup” and “Provision a host” from the first blog post and try to provision a host on your own. You should also check out the following blog post, which describes the Automatic Provisioning of a Hadoop Cluster on Bare Metal with The Foreman and Puppet. I will continue the blog series with a third post in which I will show how to use “The Foreman” API to configure “The Foreman” automatically.

Authors

Felix Massem and Jan-Frederic Markert