My test environment:
- MacBook Pro (Late 2003, Retina, i7 2.66 Ghz)
- PCIe-based Flash Storage
- 2-4 virtuals machines using VMware Fusion 4, each with 2 GB of RAM.
- Ubuntu 13.10 server edition with PHP 5.5 and OPCache enabled
- GlusterFS running on all VMs with a volume in replica mode
- The volume was mounted using nodiratime,noatime using GlusterFS native driver (NFS was slower)
- siege -c 20 -r 5 http://localhost/foo # Cache warming
- siege -c 20 -r 100 http://localhost/foo # Actual test
I then compared the local filesystem (inside the VM) vs the Gluster volume using these setups:
- 2 nodes, 4 cores per node
- 2 nodes, 2 cores per node
- 4 nodes, 2 cores per node
All tests were ran 2-3 times while my computer was doing nothing and the results were very consistent.
|Local||2.91 s||9.92 s||5.39 s||6.07 s|
|Gluster||10.84 s||23.94 s||7.81 s||14.20 s|
|Local||5.41 s||19.14 s||9.67 s||11.41 s|
|Gluster||25.05 s||31.91 s||15.17 s||24.04 s|
|Local||5.57 s||19.6 s||9.79 s||11.65 s|
|Gluster||30.56 s||35.92 s||18.36 s||28.28 s|
|2 nodes, 4 cores||273 %||141 %||45 %||153 %|
|2 nodes, 2 cores||363 %||67 %||57 %||162 %|
|4 nodes, 2 cores||449 %||83 %||88 %||206 %|
|Average||361 %||97 %||63 %||174 %|
|2 nodes vs
|Local||3 %||2 %||1 %||2 %|
|Gluster||22 %||13 %||21 %||19 %|
|4 cores vs
|Local||86 %||93 %||79 %||86 %|
|Gluster||131 %||33 %||94 %||86 %|
- Red — Wordpress and Drupal have an acceptable loss in performance under Gluster, but Symfony is catastrophic.
- Blue — The local tests are slightly slower when using 4 nodes vs 2 nodes. This is normal, my computer had 4 VMs running.
- Green — The gluster tests are 20% slower on a 4 node setup because there is more communication between the nodes to keep them all in sync. 20% overhead for double the nodes isn’t that bad.
- Purple — The local tests are 85% quicker using 4 cores vs 2 cores. A bit under 100% is normal, there is always some overhead to parallel processing.
- Yellow — For the Gluster tests, Symfony and Drupal scale very well with the number of nodes, but Wordpress is stalling, I am not sure why.
I am still not sure why Symfony is so much slower on GlusterFS, but really, I can’t use it in production for the moment because I/O is already the weak point of my infrastructure. I am in the process of looking for a different hosting solution, maybe it will be better then.