Code samples for the Practical Devops book

These are the code samples for the book “Practical Devops” by Joakim Verona published by Packt Publishing.

The books home page: https://www.packtpub.com/networking-and-servers/practical-devops

In most cases you will need the book to make the most out of the exercises.

Please ensure that you have the latest version of these exercises before continuing.

To get the code samples from Github do:

git clone https://github.com/jave/practicaldevops.git

And to keep the samples updated,

cd practicaldevops
git pull https://github.com/jave/practicaldevops.git

ch3 How Architecture affects DevOps

Liquibase hello-world

cd ch3/liquibase-helloworld
mvn liquibase:update

Manual installation

dnf install postgresql
dnf install nginx
cd ch3/crm1
lein build
lein run

ch4 Everything is code

Docker intermission

These instructions are for Fedora, but they are similar for other distributions such as Ubuntu.

To make sure Docker is working properly, see the following documentation for Fedora.

https://docs.docker.com/v1.5/installation/fedora/

• For fedora 21 and later do:

dnf  -y install docker

• docker-io was renamed to docker from Fedora 21, so use “docker-io” on older red hat derivates, “docker” on newer
• Use a sudo capable user to run docker commands, or the root user
• You can also add a docker group with rights to use the docker socket needed to communicate with the docker daemon.

This approach is described here https://docs.docker.com/v1.5/installation/fedora/

In summary:

$ sudo groupadd docker
$ sudo chown root:docker /var/run/docker.sock
$ sudo usermod -a -G docker $USERNAME

• You might need “setenforce 0” to start docker. The comand will disable selinux, which has security implications. Use this only on a test machine.
• To start and enable docker on reboot:

sudo systemctl start docker
sudo systemctl enable docker

To verify that docker works:

sudo docker run -i -t fedora /bin/bash

For some exercises you need to have docker-compose installed first.

On Fedora 23 you can do:

dnf install docker-compose

In earlier versions you needed to download docker-compose manually.

setting up a basic git server

bare repo:

cd /opt/git 
mkdir project.git
cd project.git
git init --bare

• Now try cloning, making changes, and pushing to the server

Gerrit

Run a Gerrit container:

docker run -d -p 8080:8080 -p 29418:29418 openfrontier/gerrit

On the host machine you can now install the supporting git-review package:

sudo dnf install git-review

Rebase your commits on top of the commits in the remote repository:

git pull --rebase origin master

Interactively edit the history, possibly squashing commits together to make a more readable history:

git rebase -i origin/master

Gitlab

Now create a directory for gitlab, and fetch the compose file:

mkdir gitlab 
cd gitlab 
wget https://raw.githubusercontent.com/sameersbn/docker-gitlab/master/docker-compose.yml

Now start the gitlab stack.

docker-compose up

When the containers are up and running, access the web ui:

http://loaclhost:10080

and enter the following credentials:

• username: root
• password: 5iveL!fe

ch5 Build the code

Create a “freestyle” class job in Jenkins that runs the “fortune” command.

First install Jenkins.

dnf install jenkins

Then follow the instruction in the book to configure the job.

Cheating with FPM

To install fpm:

yum install rubygems
yum install ruby
yum install ruby-devel
gem install fpm

Package this shell script:

#!/bin/sh
echo 'Hello World!'

chmod a+x usr/local/bin/hello.sh
fpm -s dir -t rpm -n hello-world -v 1 -C installdir usr

rpm -qivp hello-world.rpm
rpm -ivh hello-world.rpm

Build servers, Jenkins in particular

dnf install jenkins

systemctl start jenkins

ch6 Test the code

A Junit example

cd ch6/hello-junit
mvn install

Arquilian

There is an arquillian hello-world in the Arquillian documentation.

git clone https://github.com/aslakknutsen/arquillian-example-helloworld.git
cd arquillian-example-helloworld
mvn install

Automated acceptance testing

There are two implementations, one with annotations, and one with Lambda notation.

While the lambda notation is easier to read than the annotation syntax, cucumbers lambda notation is fairly new and can be problematic to get to work depending on your Java implementation.

To run the annotation based example:

cd ch6/hello-cucumber6
mvn clean test

To run the lambda based example:

cd ch6/hello-cucumber8
mvn clean test

A complete test automation scenario

hello-selenium-world

Hello selenium world is a minimal selenium example that should open a firefox browser window and ask google ‘hello world’. You should see a list of search matches for ‘hello world’.

It is useful to check that this example runs before testing other examples. To run it:

cd ch6/hello-selenium
mvn test

Running the usermanager example manually

You will need Leiningen, http://leiningen.org/ https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein

ch6/usermanager
lein run

Running the automated test

autotest_v1/bin/autotest.sh

Handling the tricky dependencies with Docker

docker run -d -p 4444:4444 --name selenium-hub selenium/hub
docker run -d --link selenium-hub:hub selenium/node-firefox

ch7 Deploying the code

Executing code on the client

salt -E '.*' cmd.run 'ls -l'

Puppet master, Puppet agent

rfkrocktk/puppet is a convenient docker image for exploring puppet.

• https://hub.docker.com/r/rfkrocktk/puppet/ this is the agent
• https://hub.docker.com/r/rfkrocktk/puppetmaster/ this is the master

docker --name dockerduck --hostname dockerduck -e PUPPETMASTER_TCP_HOST=ultramaster.example.com \
    -v /var/lib/docker/dockercontainer/puppet/ssl:/var/lib/puppet/ssl rfkrocktk/puppet

Ansible

FROM williamyeh/ansible:centos7

docker run -v `pwd`/ansible:/ansible  -it <hash> bash
cd /ansible
ansible-playbook -i inventory playbook.yml    --connection=local --sudo

A docker container which supports systemd:

FROM fedora
RUN yum -y update; yum clean all
RUN yum install  ansible sudo
RUN systemctl mask systemd-remount-fs.service dev-hugepages.mount \
sys-fs-fuse-connections.mount \
systemd-logind.service getty.target console-getty.service
RUN cp /usr/lib/systemd/system/dbus.service /etc/systemd/system/;\
sed -i 's/OOMScoreAdjust=-900//' /etc/systemd/system/dbus.service

VOLUME ["/sys/fs/cgroup", "/run", "/tmp"]
ENV container=docker

CMD ["/usr/sbin/init"]

To run the new container:

docker run -it --rm -v /sys/fs/cgroup:/sys/fs/cgroup:ro  -v `pwd`/ansible:/ansible <hash>

Connect to the container:

docker exec -it <hash> bash

A slightly more advanced exercise:

---
- hosts: localhost
  vars:
    http_port: 80
    max_clients: 200
  remote_user: root
  tasks:
  - name: ensure apache is at the latest version
    yum: name=httpd state=latest
  - name: write the apache config file
    template: src=/srv/httpd.j2 dest=/etc/httpd.conf
    notify:
    - restart apache
  - name: ensure apache is running (and enable it at boot)
    service: name=httpd state=started enabled=yes
  handlers:
    - name: restart apache
      service: name=httpd state=restarted

Deploying with Chef

Start a clean container for the exercise:

docker run -it ubuntu bash

Install Chef in the container:

curl -L https://www.opscode.com/chef/install.sh | bash

Verify the chef-solo was installed:

chef-solo -v

Fetch and unpack a pre-rolled chef configuration:

curl -L  http://github.com/opscode/chef-repo/tarball/master -o master.tgz
tar -zxf master.tgz
mv chef-repo* chef-repo
rm master.tgz

Create a configuration file for chef:

mkdir .chef
echo "cookbook_path [ '/root/chef-repo/cookbooks' ]" > .chef/knife.rb

Now create a template:

knife cookbook create phpapp

Deploying with Saltstack

Start a Saltstack container:

docker run -i -t --name=saltdocker_master_1 -h master -p 4505 -p 4506 \
   -p 8080 -p 8081 -e SALT_NAME=master -e SALT_USE=master \
   -v `pwd`/srv/salt:/srv/salt:rw jacksoncage/salt

Start a shell inside the Saltstack container:

docker exec -i -t saltdocker_master_1 bash

Salt state to install httpd:

top.sls:
base:
  '*':
    - webserver

webserver.sls:
apache2:               # ID declaration
  pkg:                # state declaration
    - installed       # function declaration

Run this command to ensure the desired state:

salt-call --local state.highstate -l debug

Vagrant

yum install 'vagrant*'

To use Vagrants Virtualbox driver, you need to set up Virtualbox according to your distribution.

Create a virtual machine with Vagrant from a recipy:

vagrant init hashicorp/precise32

Try starting the machine:

vagrant up

You can now ssh to the machine:

vagrant ssh

Add this to the Vagrant file:

Vagrant.configure("2") do |config|
  config.vm.box = "hashicorp/precise32"
  config.vm.provision :shell, path: "bootstrap.sh"
end

And create a bootstrap.sh file that will install Apache httpd:

#!/usr/bin/env bash
apt-get update
apt-get install -y apache2

ch8 Monitoring the code

Nagios

Start a Nagios container:

docker run -e     NAGIOSADMIN_USER=nagiosadmin -e NAGIOSAMDIN_PASS=nagios  -p 80:30000 cpuguy83/nagios 

Start a second container to monitor:

docker run -p 30001:80 nginx

A docker compose file for the scenario:

nagios:
  image: mt-nagios 
  build:
    - mt-nagios
  ports:
   -  80:30000 
  environment:
    - NAGIOSADMIN_USER=nagiosadmin
    - NAGIOSAMDIN_PASS=nagios
  volumes:
   ./nagios:/etc/nagios   
nginx:
  image: nginx

Configuration files for the Nagios example:

define host {
    name        regular-host
    use         linux-server
    register       0
    max_check_attempts   5
}

define host{
    use             regular-host
    host_name       client1
    address         192.168.200.15
    contact_groups  admins
    notes           test client1
}

hostgroups.cfg

define hostgroup {
    hostgroup_name  test-group
    alias           Test Servers
    members         client1
}

services.cfg
define service {
    use                     generic-service
    hostgroup_name          test-group
    service_description     PING
    check_command           check_ping!200.0,20%!600.0,60%
}

An example mail configuration:

define contact{
    contact_name                    matangle-admin
    use                             generic-contact
    alias                           Nagios Admin
    email                           [email protected]
}

define contactgroup{
    contactgroup_name       admins
    alias                   Nagios Administrators
    members                 matange-admin
}

Munin

docker run -p 30005:80 lrivallain/munin:latest

Running commands in the munin container:

docker exec -it <hash> bash
su - munin --shell=/bin/bash
/usr/share/munin/munin-update 

If you are having trouble running munin-update, try:

chown munin.munin /var/log/munin/munin-update.log

It may still take some time for the graphs to display.

This is the code for the munin plugin:

graph_title Load average
graph_vlabel load
load.label load

To emit data you simply print it to stdout.

printf "load.value "
cut -d' ' -f2  /proc/loadavg

Here is an example script.

#!/bin/sh

case $1 in
   config)
        cat <<'EOM'
graph_title Load average
graph_vlabel load
load.label load
EOM
        exit 0;;
esac

printf "load.value "
cut -d' ' -f2  /proc/loadavg

Ganglia

To get help with the container:

docker run wookietreiber/ganglia --help

To run the Ganglia container:

docker run -p 30010:80 wookietreiber/ganglia

Graphite

Start Graphite:

docker run -it   -p 30020:80   -p 2003:2003   sitespeedio/graphite

Try the following url: http://localhost:30020/

Log handling

Start Kibana and Elasticsearch:

docker run -d elasticsearch &&
docker run --link some-elasticsearch:elasticsearch -d kibana

ch9 Issue Tracking

Bugzilla

docker run -p 6050:80 dklawren/docker-bugzilla 

Trac

docker run -d -p 6051:8080 barogi/trac:1.0.2

Redmine

docker run -d -p  6052:3000 redmine

The Gitlab issue tracker

Trying the Gitlab CLI:

GITLAB_API_PRIVATE_TOKEN=<token from your project>
GITLAB_API_ENDPOINT=http://gitlab.matangle.com:50003/api/v3
  gitlab help Issues

Jira

docker run -p 6053:8080 cptactionhank/atlassian-jira:latest

ch10 The Internet of Things and DevOps

NodeMCU

To get a newer firmware(please change the version to the latest available first):

wget https://github.com/nodemcu/nodemcu-firmware/releases/download/0.9.6-dev_20150704/nodemcu_integer_0.9.6-dev_20150704.bin

Get esptool:

git clone https://github.com/themadinventor/esptool.git

Install pyserial:

sudo dnf install pyserial

Burn the firmware:

sudo python ./esptool.py --port /dev/ttyUSB0 write_flash 0x00000 nodemcu_integer_0.9.6-dev_20150704.bin

You might need additional arguments:

sudo esptool.py --port=/dev/ttyUSB0 write_flash 0x0 nodemcu_integer_0.9.6-dev_20150704.bin  -fs 32m -fm dio -ff 40m

Do some tests to see that the connection is working:

sudo ./esptool.py read_mac
Connecting...
MAC: 18:fe:34:00:d7:21

sudo ./esptool.py flash_id
Connecting...
Manufacturer: e0
Device: 4016

Try the LED:

gpio.write(0, gpio.LOW)  -- turn led on

gpio.write(0, gpio.HIGH) -- turn led off

Blink the LED in a loop:

while 1 do                     -- loop forever
      gpio.write(0, gpio.HIGH) -- turn led off
      tmr.delay(1000000)       -- wait one second
      gpio.write(0, gpio.LOW)  -- turn led on
      tmr.delay(1000000)       -- wait one second
end

To connect to a wireless network.

wifi.setmode(wifi.STATION)
wifi.sta.config("SSID","password")

To see the IP we got:

print(wifi.sta.getip())

Connecting to a web server:

conn=net.createConnection(net.TCP, false) 
conn:on("receive", function(conn, pl) print(pl) end)
conn:connect(80,"121.41.33.127")
conn:send("GET / HTTP/1.1\r\nHost: www.nodemcu.com\r\n"
    .."Connection: keep-alive\r\nAccept: */*\r\n\r\n")

Timer:

tmr.alarm(1, 1000, 1, function() 
    print("hello world") 
end )

Stop the timer:

tmr.stop(1)