Managing Large Networks with Puppet

silas — Mon, 04 May 2009 00:00:00 GMT

Puppet is an open source configuration management tool written in Ruby. It allows a systems administrator to define how a system should be configured using Puppet's declarative language. Each Puppet client pulls its catalog at a regular interval and figures out how to make the catalog definitions true for the local operating system.

The Puppet Introduction uses the following diagram to show how Puppet works.

Currently Puppet is most useful when you have lots of nodes with similar setups. Unfortunately Puppet's declarative language is a bit weak when it comes to inheritance and users familiar with true object oriented systems will soon become frustrated (at least I did). I'm assuming this issue will be addressed in the future, but for now I'm going to tell you how I setup a 100+ node system while adhering to DRY principles.

A quick note, I'm assuming you already know how to use Puppet.

First lets start with the Puppetmaster configuration layout:

puppet/
 - fileserver.conf
 - manifests/
   - classes/
     - initialize.pp
   - nodes/
     - net/
       - example/
         - web01.pp   # web01.example.net
   - site.pp
   - templates.pp
 - modules/           # application specific modules
   - httpd/
     - files/         # static assets (ex: default HTML file)
     - manifests/     # application configuration logic
     - templates/     # dynamic configuration files (ex: httpd.conf)

The layout was adapted from recommendations in Pulling Strings with Puppet by James Turnbull.

When I was initially designing my Puppet setup the common practice was to define and initialize the various Puppet types and classes throughout the inheritance tree. So if you had a generic resolv.conf configuration you would include it at the top of the inheritance tree. This worked great until I needed to change one attribute of a class further down. I initially tackled this problem by hacking in if/case statements, but eventually it became unmanageable.

After a couple of rewrites I came up with the idea of wrapping the internals of each class in a conditional statement and initializing all classes at the end of each node.

The key components of my setup were:

import all modules in the site.pp manifest
wrap the code of each class in a conditional statement
define and extend attributes throughout the inheritance tree (including the class conditional)
include all classes at the end of each node

This lets you both arbitrarily redefine or extend class attributes (including the on/off state) throughout the inheritance tree.

I've created a simple example to show how this setup works. The main files are listed below (you'll want to go through each in order):

site.pp - import modules
templates.pp - define and extend class attributes
web01.pp - make use of defaults
ns01.pp - disable a class
initialize.pp - initialize all classes
client.pp - class implementation

Also note that I designed this setup about 8 months ago and I haven't kept as up-to-date on recent developments as I should. If there is a better way to do this please let me know.

Profile Management with Git and GitHub

silas — Sun, 08 Mar 2009 00:00:00 GMT

The following describes a simple way to manage you profile configuration files using GitHub.

Features

Centralized configuration management
Files live in their native locations (no symbolic linking)
Home directory is not a Git repository
All the power of git with a simple alias

Setup Repository

Log into GitHub and create a repository named config
Add your public keys to GitHub (if you haven't done so already)
Open a terminal and switch to your home directory
```
cd ~
```
Create a configuration directory
```
mkdir .config.git
```

Add the following alias to your current session and your .bash_profile

alias config='git --git-dir=$HOME/.config.git/ --work-tree=$HOME'
echo "alias config='git --git-dir=$HOME/.config.git/ --work-tree=$HOME'" >> .bash_profile

Add your .bash_profile to the configuration repository
```
config add .bash_profile
```
Commit the changes
```
config commit -m 'Initial commit'
```

Change the origin to GitHub

config remote add origin git@github.com:GITHUB_USERNAME/config.git

Push the changes
```
config push origin master
```

If you get an error when running config pull to the effect of You asked me to pull without... run the follow:

    echo -e '[branch "master"]\n  remote = origin\n  merge = refs/heads/master' >> ~/.config.git/config

Setup Configuration Management on a Different System

Add your public keys to GitHub (if you haven't done so already)
Switch to your home directory
```
cd ~
```
Backup your local configuration files, example:
```
mv .bash_profile .bash_profile.bk
```

Clone your configuration repository

git clone git@github.com:GITHUB_USERNAME/config.git config.git

Move the git metadata to ~/.config.git
```
mv config.git/.git .config.git
```
Enable dotglob
```
shopt -s dotglob
```
Move your configuration files to your home directory
```
mv -i config.git/* .
```
Delete the config.git directory
```
rmdir config.git
```
Logout and log back in

Basic Usage

config pull - get latest configuration changes
config add FILENAME - add a configuration file
config commit -a - save all configuration changes
config push - push configuration changes to GitHub
and any other config GIT_OPTION

You can see my configuration repository at http://github.com/silas/config.