Skip to content

Building Puppet

Jump to bottom
aborkar-ibm edited this page Oct 31, 2019 · 71 revisions

Building Puppet

Below versions of Puppet are available in respective distributions at the time of creation of these build instructions:

  • RHEL 6.10 has 3.6.2
  • RHEL 7.5 has 3.6.2
  • RHEL 7.6 has 3.6.2
  • Ubuntu 16.04 has 3.8.5
  • Ubuntu 18.04 has 5.4.0

The instructions provided below specify the steps to build Puppet 5.5.2 on Linux on IBM Z for the following distributions

  • RHEL (6.10, 7.5, 7.6, 8.0)
  • SLES (12 SP4, 15, 15 SP1)
  • Ubuntu (16.04, 18.04)

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.

  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build using script

If you want to build puppet using manual steps, go to STEP 2.

Use the following commands to build puppet using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/Puppet/5.5.2/build_puppet.sh

# Build Puppet
bash build_puppet.sh -s [master/agent]

If the build completes successfully for master installation, go to STEP 2.8 and for agent installation, go to step 3.7. In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

Step 2: Puppet Master Installation

2.1) Install following dependencies

  • RHEL (6.10, 7.5, 7.6, 8.0)

    sudo yum install -y gcc-c++ readline-devel tar openssl unzip libyaml-devel PackageKit-cron openssl-devel make git wget sqlite-devel glibc-common hostname

  • RHEL 8.0

    sudo yum update -y
sudo yum install -y gcc-c++ readline-devel tar openssl unzip libyaml PackageKit-cron openssl-devel make git wget sqlite-devel glibc-common hostname procps-ng diffutils glibc-langpack-en

  • SLES (12 SP4, 15, 15 SP1)

    sudo zypper install -y gcc-c++ readline-devel tar openssl unzip libopenssl-devel make git wget sqlite-devel glibc-locale cron net-tools curl
    • Build Openssl 1.0.2k 
      cd /<source_root>/
wget https://www.openssl.org/source/old/1.0.2/openssl-1.0.2k.tar.gz
tar zxf openssl-1.0.2k.tar.gz
cd openssl-1.0.2k
./config --prefix=/usr --openssldir=/etc/ssl --libdir=lib shared zlib-dynamic
make
sudo make install

  • Ubuntu (16.04)

    sudo apt-get update
sudo apt-get install -y g++ libreadline6 libreadline6-dev tar openssl unzip libyaml-dev libssl-dev make git wget libsqlite3-dev libc6-dev cron locales ruby ruby-dev

  • Ubuntu (18.04)

    sudo apt-get update
sudo apt-get install -y g++ libreadline7 libreadline-dev tar make git wget libsqlite3-dev libc6-dev cron locales unzip libyaml-dev zlibc zlib1g-dev zlib1g libxml2-dev libgdbm-dev openssl1.0 libssl1.0-dev

2.2) Download and install Ruby

  • RHEL (6.10, 7.5, 7.6, 8.0)

    cd /<source_root>/
wget http://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.5.tar.gz
tar -xvf ruby-2.4.5.tar.gz
cd ruby-2.4.5
./configure && make && sudo -E make install

  • SLES (12 SP4, 15, 15 SP1)

    cd /<source_root>/
wget http://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.5.tar.gz
tar -xvf ruby-2.4.5.tar.gz
cd ruby-2.4.5
./configure LDFLAGS='-L/<source_root>/openssl-1.0.2k' --with-openssl-include=/<source_root>/openssl-1.0.2k/include --with-openssl-dir=/usr/
make && sudo -E make install

  • Ubuntu (18.04)

    cd /<source_root>/
wget http://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.1.tar.gz
tar -xvf ruby-2.4.1.tar.gz
cd ruby-2.4.1
./configure && make && sudo -E make install

2.3) Install bundler

  • RHEL (6.10, 7.5, 7.6, 8.0), SLES (12 SP4, 15, 15 SP1) and Ubuntu (18.04)

    cd /<source_root>/
sudo /usr/local/bin/gem install bundler rake-compiler

  • Ubuntu (16.04)

    cd /<source_root>/
sudo /usr/bin/gem install bundler rake-compiler

2.4) Install Puppet

  • RHEL (6.10, 7.5, 7.6, 8.0), SLES (12 SP4, 15, 15 SP1) and Ubuntu (18.04)
    cd /<source_root>/
    sudo /usr/local/bin/gem install puppet -v 5.5.2
  • Ubuntu (16.04)
    cd /<source_root>/
    sudo /usr/bin/gem install puppet -v 5.5.2

2.5) Locate the $confdir by command

Note: Please make sure the directory /usr/local/lib has sufficient read and execute permissions to run the above command. Incase it doesn't, run sudo chmod 755 /usr/local/lib to give the necessary permissions. Make sure that /usr/local/bin is available in PATH environment variable.

    confdir=`puppet master --configprint confdir`
    echo $confdir

The output gives the directory. If such directory does not exist, create one.

    mkdir -p $confdir

2.6) Create necessary directories and files in $confdir

    mkdir $confdir/modules
    mkdir $confdir/manifests
    cd $confdir
    touch puppet.conf
    wget https://raw.githubusercontent.com/puppetlabs/puppet/master/conf/auth.conf
    mkdir -p $confdir/opt/puppetlabs/puppet
    mkdir -p $confdir/var/log/puppetlabs

2.7) Create "puppet" user and group

    sudo useradd -d /home/puppet -m -s /bin/bash puppet
    sudo /usr/local/bin/puppet resource group puppet ensure=present

Note: Set a user specified password for puppet user .Running sudo passwd puppet will prompt for new password

2.8) Add the following parameters to $confdir/puppet.conf (assuming hostname of the master machine is master.myhost.com)

Note: Hostname can found by running the command hostname -f

    [main]
          logdir = $confdir/var/log/puppetlabs
          basemodulepath = $confdir/modules
          server = master.myhost.com
          user  = puppet
          group = puppet
          pluginsync = false
     [master]
          certname = master.myhost.com
          autosign = true

2.9) The Puppet master runs on TCP port 8140. This port needs to be open on your master firewall (and any intervening firewalls and network devices), and your agent must be able to route and connect to the master. To do this, you need to have an appropriate firewall rule on your master, such as the following rule for the Netfilter firewall

    sudo iptables -A INPUT -p tcp -m state --state NEW --dport 8140 -j ACCEPT

Step 3: Puppet Agent Installation

3.1) Install following dependencies

  • RHEL (6.10, 7.5, 7.6)

    sudo yum install -y gcc-c++ readline-devel tar openssl unzip libyaml-devel PackageKit-cron openssl-devel make git wget sqlite-devel glibc-common hostname

  • RHEL 8.0

    sudo yum update -y
sudo yum install -y gcc-c++ readline-devel tar openssl unzip libyaml PackageKit-cron openssl-devel make git wget sqlite-devel glibc-common hostname procps-ng diffutils glibc-langpack-en

  • SLES (12 SP4, 15, 15 SP1)

    sudo zypper install -y gcc-c++ readline-devel tar openssl unzip libopenssl-devel make git wget sqlite-devel glibc-locale cron net-tools curl
    • Build Openssl 1.0.2k 
      cd /<source_root>/
wget https://www.openssl.org/source/old/1.0.2/openssl-1.0.2k.tar.gz
tar zxf openssl-1.0.2k.tar.gz
cd openssl-1.0.2k
./config --prefix=/usr --openssldir=/etc/ssl --libdir=lib shared zlib-dynamic
make
sudo make install

  • Ubuntu (16.04)

    sudo apt-get update
sudo apt-get install -y g++ libreadline6 libreadline6-dev tar openssl unzip libyaml-dev libssl-dev make git wget libsqlite3-dev libc6-dev cron locales ruby ruby-dev

  • Ubuntu (18.04)

    sudo apt-get update
sudo apt-get install -y g++ libreadline7 libreadline-dev tar make git wget libsqlite3-dev libc6-dev cron locales unzip libyaml-dev zlibc zlib1g-dev zlib1g libxml2-dev libgdbm-dev openssl1.0 libssl1.0-dev

3.2) Download and install Ruby

  • RHEL (6.10, 7.5, 7.6, 8.0)

    cd /<source_root>/
wget http://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.5.tar.gz
tar -xvf ruby-2.4.5.tar.gz
cd ruby-2.4.5
./configure && make && sudo -E make install

  • SLES (12 SP4, 15, 15 SP1)

    cd /<source_root>/
wget http://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.5.tar.gz
tar -xvf ruby-2.4.5.tar.gz
cd ruby-2.4.5
./configure LDFLAGS='-L/<source_root>/openssl-1.0.2k' --with-openssl-include=/<source_root>/openssl-1.0.2k/include --with-openssl-dir=/usr/
make && sudo -E make install

  • Ubuntu (18.04)

    cd /<source_root>/
wget http://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.1.tar.gz
tar -xvf ruby-2.4.1.tar.gz
cd ruby-2.4.1
./configure && make && sudo -E make install

3.3) Install bundler

  • RHEL (6.10, 7.5, 7.6, 8.0), SLES (12 SP4, 15, 15 SP1) and Ubuntu (18.04)

    cd /<source_root>/
sudo /usr/local/bin/gem install bundler rake-compiler

  • Ubuntu (16.04)

    cd /<source_root>/
sudo /usr/bin/gem install bundler rake-compiler

3.4) Install Puppet

  • RHEL (6.10, 7.5, 7.6 8.0), SLES (12 SP4, 15, 15 SP1) and Ubuntu (18.04)
    cd /<source_root>/
    sudo /usr/local/bin/gem install puppet -v 5.5.2
  • Ubuntu (16.04)
    cd /<source_root>/
    sudo /usr/bin/gem install puppet -v 5.5.2

3.5) Locate the $confdir by command

Note: Please make sure the directory /usr/local/lib has sufficent read and execute permissions to run the above command. Incase it doesn't, run sudo chmod 755 /usr/local/lib to give the necessary permissions. Make sure that /usr/local/bin is available in PATH environment variable.

    confdir=`puppet agent --configprint confdir`
    echo $confdir

The output gives the directory. If such directory does not exist, create one.

    mkdir -p $confdir

3.6) Create necessary directories and files in $confdir

    cd $confdir
    mkdir -p $confdir/opt/puppetlabs/puppet
    mkdir -p $confdir/var/log/puppetlabs
    touch puppet.conf

3.7) Add the following parameters to $confdir/puppet.conf (assuming hostname of the master machine is master.myhost.com and hostname of the agent machine is agent.myhost.com)

Note: Hostname can found by running the command hostname -f

    [main]
          logdir =  $confdir/var/log/puppetlabs
          basemodulepath = /etc/puppetlabs/puppet/modules
          server = master.myhost.com
          user  = puppet
          group = puppet
          pluginsync = false
     [agent]
          certname = agent.myhost.com
          report = true
          pluginsync = false

3.8) Add an entry in /etc/hosts file with ipaddress and hostname of master node

     sudo vi /etc/hosts
     <master ipaddress> <master hostname>

Step 4: Connecting the Master and Agent for the first time

4.1) Run the master application on master machine (assuming with hostname master.myhost.com)

    puppet master --verbose --no-daemonize

The --verbose option outputs verbose logging and the --no-daemonize option keeps the daemon in the foreground and redirects output to standard output. You can also add the --debug option to produce more verbose debug output from the daemon.

4.2) On the agent application (assuming the hostname of the agent is agent.myhost.com)

    puppet agent --test

Note: The following errors might be seen after execution of the above step

Info: Retrieving pluginfacts
Error: /File[/opt/puppetlabs/puppet/cache/facts.d]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///pluginfacts
Info: Retrieving plugin
Error: /File[/opt/puppetlabs/puppet/cache/lib]: Could not evaluate: Could not retrieve information from environment production source(s) puppet://master.myhost.com/plugins

This is because you don't have any plugins to syn yet, and the pluginsyn property is set to be true by default. So solutions are:

1) Disable the setting in the agent's 'puppet.conf' file by setting  pluginsyn=false. Or 

2) Create at least one plugin ( By running puppet module install saz-sudo  on master)

Step 5: Testing

For testing, run the tests from the source code on Master machine.

5.1) Switch user to puppet, clone Puppet git repository in /home/puppet and execute "bundle install" to install the required gems

  • Download and install Ruby, bundler for Ubuntu (16.04) 
    cd /<source_root>/
wget http://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.1.tar.gz
tar -xvf ruby-2.4.1.tar.gz
cd ruby-2.4.1
./configure && make && sudo -E make install
sudo /usr/local/bin/gem install bundler
  • Clone Puppet git repository 
    su puppet
cd /home/puppet
git clone git://github.com/puppetlabs/puppet
cd puppet
git checkout 5.5.2
bundle install --path .bundle/gems/

5.2) Running the test cases

Few testcases need to be executed as root user and others as puppet user.

5.2.1) Execute testcases as root user:

* Unit testcases, except ssl, face, indirector, network related testcases, should be executed as root user.
* The integration testcases for provider and type should be executed as root user.
* On SLES 15,15 SP1, the unit and integration tests for provider and type are failing. This is a known issue, for more information refer this.
* Note: Run the below commands as root user.You can switch to root user by running exit, if you are currently switched to puppet user.

  • Create a shell script
    For example,rootuser_tests.sh 
	cd /home/puppet/puppet  
	touch rootuser_tests.sh  
	chmod +x rootuser_tests.sh
  • Add the following content to the shell script
	#!/bin/bash
	set -e
		
	echo "Running Unit testcases as root user"
	declare -a unittests1
	unittests1=$(ls spec/unit|egrep -v "ssl|face|indirector|network")
	unittest_list1=($unittests1)
	for i in "${unittest_list1[@]}"
	do
	  bundle exec rspec "spec/unit/$i"
	done
		
	echo "Running Integration testcases as root user"
	declare -a integration1
	integration1=$(ls spec/integration|egrep "provider|type")
	integration_list1=($integration1)
	for j in "${integration_list1[@]}"
	do
	  bundle exec rspec --exclude-pattern ./spec/integration/provider/service/systemd_spec.rb "spec/integration/$j"
	done

  • Run the shell script

    • Ubuntu (16.04, 18.04)
    locale-gen "en_US.UTF-8"
export LC_ALL="en_US.UTF8"
./rootuser_tests.sh
    • RHEL (6.10, 7.5, 7.6, 8.0) and SLES (12 SP4, 15, 15 SP1)
    export LC_ALL="en_US.UTF8"
./rootuser_tests.sh
5.2.2) Execute testcases as puppet user

* ssl, face, indirector, network related unit testcases should be executed as puppet user.
* The integration testcases except provider and type related testcases should be executed as puppet user.
* On SLES 15,15 SP1, the unit and integration tests for provider and type are failing. This is a known issue, for more information refer this.
* data_binding_spec.rb file is not executed as it does not involve any testcases to be invoked directly.

  • Create a shell script
    For example puppetuser_tests.sh 
	cd /home/puppet/puppet
	touch puppetuser_tests.sh
	chmod +x puppetuser_tests.sh
  • Add the following content to the script
	#!/bin/bash
	set -e
		
	echo "Running Unit testcases as puppet user"
	declare -a unittests2
	unittests2=$(ls spec/unit|egrep "ssl|face|indirector|network")
	unittest_list2=($unittests2)
	for i in "${unittest_list2[@]}"
	do
		bundle exec rspec "spec/unit/$i"
	done
		
	echo "Running Integration testcases as puppet user"
	declare -a integration2
	integration2=$(ls spec/integration|egrep -v "data_binding_spec.rb|provider|type")
	integration_list2=($integration2)
	for j in "${integration_list2[@]}"
	do
	  bundle exec rspec "spec/integration/$j"
	done
  • Switch user to puppet
	su puppet
  • Run the shell script
	./puppetuser_tests.sh

Note: In case of below test case failure create a softlink as sudo ln -s /bin/touch /usr/bin/touch.

1) Puppet::Transaction when filtering with tags should propagate events correctly from a tagged container when running with tags
   Failure/Error: expect(Puppet::FileSystem.exist?(file1)).to be_truthy

     expected: truthy value
          got: false
   # ./spec/integration/transaction_spec.rb:371:in `block (3 levels) in <top (required)>'

For more details refer: https://tickets.puppetlabs.com/browse/PUP-8708

References:

https://puppetlabs.com/

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally