Last night I (finally) updated my (ultra-simple) personal website to 6.0, and while I was at it I decided to try out some caching ideas that had been brewing in the back of my mind for a while.
Though all caching strategies have their pitfalls and drawbacks, the idea I wanted to try -- combining Boost and Memcache -- seems like a real live-wire. However, if it can be made to work, I think it might offer some significant advantages.
The origin for this idea comes from my experience with both modules. Boost's static file caching is the heavyweight champion for anonymous traffic as far as I've seen. It's essentially the poor-man's reverse proxy, preventing the http requests from even being handled by Drupal. This gives the best results I've seen yet in terms of handling big spikes (e.g. a slashdotting).
On the other hand, I've generally experienced Memcache as providing a tangible boost for logged-in users. My thinking is that because it places the load for cached application data (e.g. variables, views, menu tree, etc) into memory rather than on the SQL back-end, the data can be loaded more quickly, and this makes for snappier page execution times.
Running both of these systems seems like a workable idea. So far my site is running just fine, but again it's a very very simple use-case. While there's some redundant caching of pages -- e.g. on a fresh anon pageload, the page will end up cached in the database, memcached, and on the filesystem via boost -- that doesn't seem like a huge drawback. I'm curious if anyone has any experience here, thoughts to add, or holes to poke in this theory.
Below are some benchmarking numbers. I am laying siege to /node with 25 concurrent users in benchmarking mode for 1 minute, a pretty righteous surge of traffic (equivalent to close to a quarter-million visits in an hour). I am also watching top to observe server-load. This is all Drupal 6 on a mid-range VPS slice w/1gb of ram and APC enabled.
Initial System w/Regular Drupal Page Cache
Load under siege: 2.24
Siege result detail:
Transactions: 4135 hitsAvailability: 100.00 %Elapsed time: 59.52 secsData transferred: 119.89 MBResponse time: 0.35 secsTransaction rate: 69.47 trans/secThroughput: 2.01 MB/secConcurrency: 24.58Successful transactions: 4135Failed transactions: 0Longest transaction: 10.07Shortest transaction: 0.12
Memcache Enabled (one server w/100Mb of cache; simple simple)
Load: 2.91
Siege result detail:
Transactions: 3477 hitsAvailability: 100.00 %Elapsed time: 60.42 secsData transferred: 100.81 MBResponse time: 0.43 secsTransaction rate: 57.55 trans/secThroughput: 1.67 MB/secConcurrency: 24.74Successful transactions: 3477Failed transactions: 0Longest transaction: 28.22Shortest transaction: 0.11
As you can see, this is actually slightly worse. I think this is because of the extra overhead of running memcached and the fact that since my site is so simple and the requests are all for the same page, Drupal's normal/basic page-cache is actually more efficient w/its results already loading into the mysql query cache.
However, in a more complex configuration, this isn't often the case. Also, as I mentioned, memcached provides a speedup for logged-in users since Drupal doesn't need to hit the SQL to load cached application data like the menu tree, variables, block-cache, etc.
Boost
Load: 0.15
Siege result detail:
Transactions: 4527 hitsAvailability: 100.00 %Elapsed time: 60.44 secsData transferred: 131.67 MBResponse time: 0.32 secsTransaction rate: 74.90 trans/secThroughput: 2.18 MB/secConcurrency: 23.91Successful transactions: 4527Failed transactions: 0Longest transaction: 11.05Shortest transaction: 0.09
As you can see, Boost is pretty unbeatable when it comes to logged-out page caching. However, it doesn't help at all for logged-in users. I think by combining the two, we can get the best of both worlds.
High performance