Using Edge Side Includes with Varnish to cache partial pages

Caching full pages with Varnish can be hard, most applications use sessions, which sets a Cookie, which makes Varnish ignore all caching.

When sessions are needed and full page cache is not available, you can resort to ESI (Edge Side Includes).

ESI has a markup language of its own, but the subset that Varnish supports is fairly simple: it is basically a placeholder that gets replaced by the referenced URL. They generate a subrequest that will mimic the original request, but for another URL. Using this system, you can cache parts of your website that do not change frequently or that are hard to generate. Since this is internal to Varnish, it will honour the cache system, including Cache-Control, If-Modified-Since, ETag, etc.

However, since the subrequest is built on top of the original, it will contain the original Cookie header, so we must ignore it.

The solution includes:

• The original script must add a "X-Esi" header to activate ESI parsing (performance).
• Cookies are removed from ESI requests unless "esi-cookies=1" is present is the URL.
• A "X-Esi-Level" header gets added when the current request is a ESI. Otherwise, it is removed.

Various scenarios where this technique can be used:

• A navigation menu with the current URL in parameter.
• A footer
• A user profile box (popup box) with the user id in parameter.
• Widgets of a sidebar

Since the X-Esi-Level header is enforced to be only present for ESI requests, you can trust it and safely ignore any security check as they would have already been done in the original request.

Here is the VCL used for Varnish and a simple example to illustrate the ?esi-cookies=1 trick.