Run a script with lowest priority

When doing low-priority tasks like backups are fixing permissions on a cronjob, it is a good idea to modify the niceness of the script. By using ionice and renice, you can ensure it won't get in the way of your important programs.

	#!/bin/sh
	# $$ represents the current PID
	# Set to lowest I/O priority
	ionice -c idle -p $$
	# Set to lowest CPU priority
	renice -n -20 -p $$ >/dev/null
	# Run a task
	date=$(date '+%s')
	tar czf /mnt/backups/$date.tar.gz /var/www

view raw cron.sh hosted with ❤ by GitHub

Choosing the best wireless router for a small office

At my office, we needed a new router. With more than 50 devices that are quite heavily using the network, we we crushing our basic personal router.

After some unsuccessful searches, I asked the question on Reddit and I worked from there.

Our setup:

• 20~25 wired computers
• 20~25 wireless computers
• ~25 smartphones

Our needs:

• ~500 GB download/month
• ~300 GB upload/month
• Google Apps (Docs, Spreadsheet, Gmail, etc.) – A lot of AJAX requests
• SSH – Needs very low latency
• Skype – High bandwidth usage
• Dropbox – Moderate bandwidth usage
• Sending and receiving >5GB files
• Syncing thousands of files over FTP

We had for routers:

• Dlink DIR-655 acting as the gateway: http://goo.gl/TXXs0[1]
• WRT54 acting as a router and a wifi AP, configured with DD-WRT: http://goo.gl/Wu89u[2]

Unfortunately, it seemed like the network was always slowing down for no reason and we needed to restart the router a couple time per day. We tried flashing the routers, switching them around and doing the updates, they just can’t support the load.

I knew I wanted some sort of QoS to prioritize usage. Ex: Skype > SSH > Browsers > FTP and at our location, good quality connections at a decent price is pretty hard to find so I was looking forward to a Dual-WAN setup.

Upload speed

When you are trying to improve your Internet connection for a lot of people, it will most likely be your upload speed that will be the bottleneck. For 30 heavy users, try to have at least 5mbps in upload

Network link speed negotiation

Router often use a non-standard algorithm to detect the speed of the link between the router and the modem, 10, 100 or 1000 mbps and sometimes, it can’t get it right. Fail to detect the current speed and you may end up talking at 10mbps to a modem trying to work at 60mbps. More often than not, you want this to be set at 100mbps.

AP

The router I chose has wireless antennas, but I you are 30 people on the same antennas, you will experience come slowdown. Consider installing some access-points over the place. You will have to wire them together, but it is much simpler than wiring everyone.

Router

I had heard good comments about Draytek and so Reddit confirmed it. They are pretty solid, easy to configure and a much cheaper alternative than their Cisco counterparts. I finally went for a Draytek 2920n. This is about 250$, supports balancing and failover with a second Internet connection and it is managed in about the same fashion as a common router. The big difference is that it  is much more powerful and can handle a traffic of a hundred devices without struggling. It was a godsend, we didn’t even need to upgrade the Internet connection, check it out.

Other options

As @devopstom suggested me, other great alternatives are Peplink, Netgear, HP and Meraki.

Meraki is especially cool with all the remote management, very easy management and automatic updates, but they were a bit too expensive for our needs.

Varnish 3 configuration

Shared caches are pretty strict when it comes to caching or not a request. This is good, otherwise it would cache requests that are not meant to be cached.

However, there is some things that can be done to improve cacheability:

1. Ignore Google Analytics cookies
2. Remove empty Cookie line
3. Normalize Accept-Encoding header
4. Allow replying with a stale response if the backend is slow

Bonus:

1. Remove some headers to reduce header size and hide some details about the server (security).
2. Add a debug header to help understand why a request is cached or not.

	#
	# Varnish 3
	#
	# Change this for your needs
	backend default {
	.host = "127.0.0.1";
	.port = "8080";
	}
	# Authorized hosts for PURGE requests
	acl purge {
	"localhost";
	"127.0.0.1";
	}
	# Below is a commented-out copy of the default VCL logic. If you
	# redefine any of these subroutines, the built-in logic will be
	# appended to your code.
	# sub vcl_recv {
	# if (req.restarts == 0) {
	# if (req.http.x-forwarded-for) {
	# set req.http.X-Forwarded-For =
	# req.http.X-Forwarded-For + ", " + client.ip;
	# } else {
	# set req.http.X-Forwarded-For = client.ip;
	# }
	# }
	# if (req.request != "GET" &&
	# req.request != "HEAD" &&
	# req.request != "PUT" &&
	# req.request != "POST" &&
	# req.request != "TRACE" &&
	# req.request != "OPTIONS" &&
	# req.request != "DELETE") {
	# /* Non-RFC2616 or CONNECT which is weird. */
	# return (pipe);
	# }
	# if (req.request != "GET" && req.request != "HEAD") {
	# /* We only deal with GET and HEAD by default */
	# return (pass);
	# }
	# if (req.http.Authorization || req.http.Cookie) {
	# /* Not cacheable by default */
	# return (pass);
	# }
	# return (lookup);
	# }
	sub vcl_recv {
	# Serve objects up to 2 minutes past their expiry if the backend is slow to respond.
	set req.grace = 120s;
	# Disable Varnish on some hosts
	# if (req.http.Host ~ "dev\.example\.com$) {
	# return (pass);
	# }
	# Ignore cookies for static files
	if (req.url ~ "\.(js|css|jpe?g|png|gif|tiff|avi|mov|mp3|ogg|wmv|wma|woff|ttf|otf|svg)") {
	unset req.http.cookie;
	}
	# Disable caching when user specifically asks for it
	if (req.http.Cache-Control ~ "no-cache" || req.http.Pragma ~ "no-cache") {
	return (pass);
	}
	# From http:#serverfault.com/questions/195654/how-to-cache-websites-using-varnish-php-and-cookies
	# Remove Google Analytics Cookies
	set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *(__)?utm[a-z0-9_]+=[^;]+;? *", "\1");
	# Enable purging, but only from authorized hosts
	if (req.request == "PURGE") {
	if (!client.ip ~ purge) {
	error 405 "Not allowed.";
	}
	return (lookup);
	}
	# normalize Accept-Encoding to reduce vary
	if (req.http.Accept-Encoding) {
	if (req.http.User-Agent ~ "MSIE 6") {
	unset req.http.Accept-Encoding;
	} elsif (req.http.Accept-Encoding ~ "gzip") {
	set req.http.Accept-Encoding = "gzip";
	} elsif (req.http.Accept-Encoding ~ "deflate") {
	set req.http.Accept-Encoding = "deflate";
	} else {
	unset req.http.Accept-Encoding;
	}
	}
	# Unset empty Cookie string
	if (req.http.Cookie ~ "^[\s;]*$") {
	unset req.http.Cookie;
	}
	}
	# sub vcl_pipe {
	# # Note that only the first request to the backend will have
	# # X-Forwarded-For set. If you use X-Forwarded-For and want to
	# # have it set for all requests, make sure to have:
	# # set bereq.http.connection = "close";
	# # here. It is not set by default as it might break some broken web
	# # applications, like IIS with NTLM authentication.
	# return (pipe);
	# }
	#
	# sub vcl_pass {
	# return (pass);
	# }
	sub vcl_pass {
	if (req.request == "PURGE") {
	error 502 "PURGE on a passed object";
	}
	}
	#
	# sub vcl_hash {
	# hash_data(req.url);
	# if (req.http.host) {
	# hash_data(req.http.host);
	# } else {
	# hash_data(server.ip);
	# }
	# return (hash);
	# }
	# sub vcl_hit {
	# return (deliver);
	# }
	#
	sub vcl_hit {
	if (req.request == "PURGE") {
	purge;
	error 200 "Purged";
	}
	}
	# sub vcl_miss {
	# return (fetch);
	# }
	sub vcl_miss {
	if (req.request == "PURGE") {
	purge;
	error 200 "Not in cache";
	}
	}
	#
	# sub vcl_fetch {
	# if (beresp.ttl <= 0s ||
	# beresp.http.Set-Cookie ||
	# beresp.http.Vary == "*") {
	# /*
	# * Mark as "Hit-For-Pass" for the next 2 minutes
	# */
	# set beresp.ttl = 120 s;
	# return (hit_for_pass);
	# }
	# return (deliver);
	# }
	sub vcl_fetch {
	# If backend is not responding, allow replying with a stale response.
	set beresp.grace = 120s;
	# Strip cookies for static files:
	if (req.url ~ "\.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|pdf|txt|tar|wav|bmp|rtf|js|flv|swf)$") {
	unset beresp.http.set-cookie;
	}
	# X-Cacheable is useful to debug the behaviour of Varnish
	# https://www.varnish-cache.org/trac/wiki/VCLExampleHitMissHeader
	if (beresp.http.Cache-Control ~ "private") {
	# You are respecting the Cache-Control=private header from the backend
	set beresp.http.X-Cacheable = "NO: Cache-Control=private";
	return (hit_for_pass);
	} elsif (beresp.http.Set-Cookie) {
	# You are respecting the Cache-Control=private header from the backend
	set beresp.http.X-Cacheable = "NO: Set-Cookie";
	return (hit_for_pass);
	} elseif (req.http.Cache-Control ~ "no-cache" || req.http.Pragma ~ "no-cache") {
	set beresp.http.X-Cacheable = "NO: Forced by user";
	return (hit_for_pass);
	#} elsif ( beresp.ttl < 1s ) {
	# # Even if no cache is specified, force a 10s cache.
	# # Be careful when using this, it may break some websites
	# set beresp.ttl = 10s;
	# set beresp.grace = 10s;
	# set beresp.http.X-Cacheable = "YES: Auto 10s";
	}
	}
	#
	# sub vcl_deliver {
	# return (deliver);
	# }
	sub vcl_deliver {
	# Remove some headers that are useless or may give security information
	remove resp.http.Age;
	remove resp.http.Via;
	remove resp.http.X-Powered-By;
	# Server is needed, so set something generic
	unset resp.http.Server;
	set resp.http.Server = "Webserver";
	}
	#
	# sub vcl_error {
	# set obj.http.Content-Type = "text/html; charset=utf-8";
	# set obj.http.Retry-After = "5";
	# synthetic {"
	# <?xml version="1.0" encoding="utf-8"?>
	# <!DOCTYPE html PUBLIC "-#W3C//DTD XHTML 1.0 Strict//EN"
	# "http:#www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
	# <html>
	# <head>
	# <title>"} + obj.status + " " + obj.response + {"</title>
	# </head>
	# <body>
	# <h1>Error "} + obj.status + " " + obj.response + {"</h1>
	# <p>"} + obj.response + {"</p>
	# <h3>Guru Meditation:</h3>
	# <p>XID: "} + req.xid + {"</p>
	# <hr>
	# <p>Varnish cache server</p>
	# </body>
	# </html>
	# "};
	# return (deliver);
	# }
	sub vcl_error {
	# Remove server for security reasons
	unset obj.http.Server;
	set obj.http.Server = "Webserver";
	}
	#
	# sub vcl_init {
	# return (ok);
	# }
	#
	# sub vcl_fini {
	# return (ok);
	# }

view raw default.vcl hosted with ❤ by GitHub

How to properly create and destroy a PHP session

When working with shared caches like Varnish or Nginx, cookies will kill everything you are trying to do.

The idea is that since a cookie can be used by the backend to modify the reply, like being logged in as a user, the shared does not take the chance and refuse to cache it. This behaviour can be modified, especially in the case of Google Analytics cookies, but for the PHP session cookie, you will typically want it.

However, it is important, for a useful usage of your shared cache, to only start a session when you really need it and destroy it when it is not needed anymore. PHP’s session_start specifically mentions that is does not unset the associated cookie.

So the idea is to start a session when you need it; for example, in a login page. This will send a header to the client, setting a cookie and the cookie will get sent back on every request. Therefore, in some global file, you can detect this cookie and reload the session. At last, on the logout, clear everything.

	<?php
	# global.php
	# In some global configuration file,
	# Start the session only if the cookie is present
	if (isset($_COOKIE[session_name()])) {
	session_start();
	}
	# In the same file or elsewhere, after the previous lines:
	if (isset($_SESSION['userid'])) {
	# If the session has been loaded and the userid is detected, set it
	$userid = $_SESSION['userid'];
	}

view raw global.php hosted with ❤ by GitHub

	<?php
	# login.php
	# We start a session if it not already started
	session_id() == '' && session_start();
	$_SESSION['userid'] = '1';

view raw login.php hosted with ❤ by GitHub

	<?php
	# logout.php
	session_start();
	# Destroy the session, this is for the current request
	session_destroy();
	# Ensure that the cookie is destructed by setting a date in the past
	setcookie(session_name(), false, time() - 3600);

view raw logout.php hosted with ❤ by GitHub

Generate Lorem Ipsum using loripsum.net API

loripsum.net describes itself as “The ‘lorem ipsum’ generator that doesn't suck”. It is the same text you are used to see over and over except that it is decorated with various HTML tags like <ul>, <b>, etc.

Seeing this, I thought it would be nice to use it to generate data fixtures that are meant to be used as blog posts, pages, comments, etc. I started a prototype for Wordpress, I should come up with a post about it soon enough, but in the mean time, I thought it would be valuable to share this little interface.

It is pretty straight forward and the options are documented in the code itself. Have fun!

	<?php
	/**
	* Generate rich Lorem Ipsum with various options
	* Uses the API of http://loripsum.net/
	* Features a generator made especially for Wordpress
	*
	* @link http://blog.lavoie.sl/2012/12/generate-lorem-ipsum-using-loripsumnet.html
	*/
	class LoremIpsumGenerator
	{
	protected static $default_options = array(
	'p' => 3, // number of paragraphs
	'l' => 'short', // paragraph length: short, medium, long, verylong
	'd' => 1, // Add <b> and <i> tags
	'a' => 0, // Add <a>
	'co' => 0, // Add <code> and <pre>
	'ul' => 0, // Add <ul>
	'ol' => 0, // Add <ol>
	'dl' => 0, // Add <dl>
	'bq' => 0, // Add <blockquote>
	'h' => 0, // Add <h1> through <h6>
	'ac' => 0, // Everything in ALL CAPS
	'pr' => 1, // Remove certain words like 'sex' or 'homo'
	);
	const URL = 'http://loripsum.net/generate.php';
	protected static function fetch(array $options)
	{
	$url = static::URL . '?' . http_build_query($options);
	return file_get_contents($url);
	}
	public static function generate(array $options = array())
	{
	$options = array_merge(static::$default_options, $options);
	$lorem = static::fetch($options);
	$lorem = str_replace("href='http://loripsum.net/' target='_blank'", 'href="#"', $lorem); // Changes links
	$lorem = str_replace(" cite='http://loripsum.net'", '', $lorem); // Remove cite attribute
	$lorem = preg_replace('/<(\/?)mark>/', '', $lorem); // Remove mark
	return $lorem;
	}
	public static function generateRich(array $options = array())
	{
	$options = array_merge(array(
	'p' => 7, // number of paragraphs
	'a' => 1, // Add <a>
	'ul' => 1, // Add <ul>
	'ol' => 1, // Add <ol>
	'bq' => 1, // Add <blockquote>
	'h' => 1, // Add <h1> through <h6>
	), $options);
	$lorem = static::generate($options);
	return $lorem;
	}
	public static function generateWordpressPost(array $options = array())
	{
	$lorem = static::generateRich($options);
	$lorem = preg_replace('/<(\/?)h1>/', '<$1b>', $lorem); // Replace h1 with b, not to conflict with page title
	$lorem = preg_replace('/<\/p>/s', "</p>\n<!--more-->", $lorem, 1); // Insert <!--more--> after first p
	return $lorem;
	}
	}

view raw LoremIpsumGenerator.php hosted with ❤ by GitHub