Simple templating system using Bash

I often have to deploy several config files that are very similar. Things like Apache VirtualHost and PHP FPM pools. The solution to this kind of problem is to use something like Puppet or Chef that will apply a real template engine and much more like creating folders and stuff. However, this kind of solution is often lengthy to implement and prevents you from doing some quick editing on-the-fly.

Hence, for very simple needs, I started using simple scripts that would only replace variables and give me a basic template to start with. This is however not very flexible and needs to be adapted for each case. And so I did a templater that replaces variables with the value in the environment. It also supports defining default values and variable interpolation.

Example with Apache + FPM

{{LOG_DIR=/var/log/apache2}}
{{RUN_DIR=/var/run/php-fpm}}
{{FCGI=$RUN_DIR/$DOMAIN.fcgi}}
{{SOCKET=$RUN_DIR/$DOMAIN.sock}}
{{EMAIL=$USER@$DOMAIN}}
{{DOC_ROOT=/home/$USER/sites/$DOMAIN/htdocs}}
<VirtualHost *:80>
  ServerAdmin {{EMAIL}}
  ServerName {{DOMAIN}}
  ServerAlias www.{{DOMAIN}}

  DocumentRoot "{{DOC_ROOT}}"

  <Directory "{{DOC_ROOT}}">
    AllowOverride All
    Order allow,deny
    Allow From All
  </Directory>

  AddHandler php-script .php
  Action php-script /php5.fastcgi virtual
  Alias /php5.fastcgi {{FCGI}}
  FastCGIExternalServer {{FCGI}} -socket {{SOCKET}}

  LogLevel warn
  CustomLog {{LOG_DIR}}/{{DOMAIN}}.access.log combined
  ErrorLog {{LOG_DIR}}/{{DOMAIN}}.error.log
</VirtualHost>

Invocation

DOMAIN=cslavoie.com ./templater.sh examples/vhost-php.conf

Help

If you add the -h switch to the invocation, it will print all the variables and their current values

And the code is available on GitHub.

Reducing the load of all those social plugins

Some javascript libraries are big and they pose a stress on the browser when they are loaded all at the same time.

Most of these libraries suggest to load them in async.

Library examples

• Google Analytics
• Google Maps
• Google Plus
• Facebook Like
• Twitter button

Problems with existing solution

• A lot of ugly script tag with semi-minified code.
• Most of the library suggest the same trick so we see code duplication.
• You have to go copy-paste the code each time you start a new project.
• If you have multiple libraries, they will all fire at the same time, possibly causing a stress on the browser
• While the libraries are loading (usually with a lot of dependencies) the browser is sluggish and users probably want to read that article before clicking on 'Like'.

Solution

• Load each library with a function taking an id, a url and a delay.
• Allow each library to have a different delay.
• You may want Google Maps to load only 200ms after the page, but the Facebook Like can wait a bit more.
• Code on Github
• Working example on jsfiddle

Introducing Dotfiles Builder

Managing bashrc sucks

We all have our nice little bashrc that we are proud of. It tests for files, programs and terminal features, detect your OS version, builds a PATH, etc. For all of our OS and different setups, various solutions exist.

Keeping several versions

Pros:

• Ultimate fine-tuning
• Easy to understand
• Usually optimized for every setup

Cons:

• Very time consuming to manage
• Hard to “backport” new ideas

Keep a single unusable file with everything and edit accordingly

Pros:

• Easy to backport, you just need to rembember to do it
• Good performance
• Since you edit at each deployment, nice fine-tuning capability

Cons:

• The single file can become unbearably cluttered.
• You eventually end up managing several version.
• Tedious to edit at each deployment

Include several subfiles

Pros:

• Still have a lot fine-tuning capabilities
• If well constructed, can be easy to understand
• Easy to deploy new features

Cons:

• Hard to detect which file to include
• Multiplicates the number of files to manage
• Slow performance
• Until recently, this was my prefered method.

Wanted features

So, what does a good bashrc have?

Should have:

• Good performance. On a busy server, you really don't want to wait 5 seconds for your new terminal because your IO is sky rocketing.
• Reduce number of included files
• Reduce tests for environment features
• Reduce tests for program and files
• High flexibility
• Cross-OS compatible
• A lot of feature detection
• Ideally, configuration files
• Ease and speed of configuration
• It should not take more than a minute to setup a new bashrc
• If you need to specify your developer email, it would be nice to do it only once.

Yes, you read right, reduce tests AND do a lot of feature detection. You don't want to do Java specific configuration or set an empty variable if Java is not even installed, but you do want Java to be automatically detected.

Generating a bashrc

Let's face it, you will install or remove Java way less often then you will start a new shell. Why then test for Java at each new shell?

This is where I introduce the Dotfiles Builder. The script runs in Bash and outputs the wanted bashrc.

This way, instead of doing:

if [ -d "$HOME/bin" ]; then
  PATH="$HOME/bin:$PATH"
fi

You would do:

if [ -d "$HOME/bin" ]; then
  echo "PATH=\"$HOME/bin:$PATH\""
fi

And the result would simply be

PATH="$HOME/bin:$PATH"

But constructing PATH is a rather common task and you want to make sure the folder is not already on your PATH. Why not wrap it up ?

Take a look at the alpha version: https://github.com/lavoiesl/dotfiles-builder/
As well as the example output.

This is a very alpha version of the intended program, but I still want to share what I have and maybe get some feedback and collaborators along the way. Currently, it only generates a bashrc, but expect more to come.