John Gruber of Daring Fireball fame wondered “[…] why WP Super Cache (or some other caching solution) isn’t part of the default WordPress installation” in a recent blog post, so I thought I’d write a blog post explaining why.

First though, a bit of background. When you visit a WordPress-powered website, WordPress asks the database software for some details such as the post contents, details about the authors of the posts to be displayed, the comments for that post (if you’re in a single post view), and so forth. The database is really good at this kind of thing, but if you ask too much of it (too many queries), it will start slowing down and eventually start not replying in time before the webserver software (Apache, etc.) gives up on it. For the vast, vast majority of WordPress sites, this never happens. The database chugs happily along responding to queries and you never have any problems.

However if you get linked to from a popular site such as Daring Fireball, Digg, or Reddit, the high number of page requests will soon overload your database software. This is because most people host their sites on what’s called shared webhosting where you and a bunch of other people share a server. The server doesn’t have much spare CPU power as hosts want to make the most out of the servers. It’s a case of you get what you pay for.

To help avoid this, WordPress does actually have a built-in cache called the object cache. It was introduced way back in 2005 and it basically caches database query results. When displaying a post for example, WordPress usually asks the database for information about the author of that post (name, etc.). When another post is displayed by that same author further down the page, instead of asking the database for the information again, WordPress fetches this data out of it’s internal cache. This simple little feature greatly cuts down on the number of queries required to generate a webpage for a visitor.

However as soon as the page is done being generated, that object cache is discarded. Initially the object cache cached these little chunks of data to the filesystem so that they could be reused on subsequent pageviews. While great in theory, the concept turned out to be terrible in practice. You see WordPress has to be able to run on countless different types of server configurations. One common enough configuration that had trouble with this file-based type of caching was where all of your files were network based. Rather than your website’s files being stored on the server that was generating the webpage for the visitor, they were stored on a different server elsewhere in the datacenter. Having to fetch all of these little cache files over the network was a slow and resource intensive exercise. And that’s only one example of many configurations that the file-based caching had trouble with.

What’s worse is that many sites were actually slower with a persistent object cache enabled. Databases are actually quite fast (as they’re designed to store lots of little chunks of data), often faster than fetching that data from the filesystem and decoding it. So for the vast majority of sites, the ones that get relatively low traffic, having a persistent cache was a lot more trouble than it was worth. Just having a temporary object cache is enough.

While the same issues don’t really apply to WP Super Cache and other plugins total caching plugins because they caching each page to a single file instead of multiple little files, as with a persistent object cache, it simply isn’t needed for most sites due to lack of traffic. More importantly though, they require setup and there’s a tradeoff as your blog is no longer dynamic. Things like “latest comments” will be out of date for example.

So the WordPress developers have instead decided that the best course of action is to make it really easy to use a caching plugin, both behind the scenes and from the user perspective. You can install a plugin in just a couple of clicks directly from the admin interface.

Infact this is how WordPress development is done as a whole — include things that the majority of users will want and make it easy to install a plugin to do the rest. The core of WordPress should stay as slim as possible — bloat is bad!

I hope that clears up the decision why not to include persistent caching by default in WordPress. 🙂

TL;DR: Persistent caching isn’t needed for most sites and is a pain to get working in all hosting environments. Instead WordPress makes it really easy to install a caching plugin if you need it.