Caching in applications: quick guide to good practices (and the errors it hides)

Cache is one of the most seductive tools in software engineering. With little effort, you transform a slow operation into an instantaneous response. The system becomes faster, the infrastructure breathes, the user smiles. It looks like magic.

And it is precisely because it seems like magic that cache causes so much damage. It solves the performance problem so easily that people spread cache everywhere, without realizing that they are exchanging a visible problem, slowness, for an invisible problem, wrong data. And wrong data is much worse than slow data.

This is a quick guide to using cache well. The idea is not to exhaust the subject, but rather to give you the criteria that separate the cache that helps from the cache that becomes a trap. As the classic computing joke goes, there are only two hard problems: cache invalidation, naming things, and one-plus-one error.

What cache does, in a sentence

Caching is saving the result of an expensive operation to reuse it, instead of redoing the operation every time. You calculate it once, save it, and the next time you deliver the saved answer.

The "expensive" operation could be a heavy database query, a call to an external service, a complex calculation or the rendering of a page. The gain comes from not repeating work. When the same result is requested many times and changes little, caching is almost always a good idea.

The keyword is "changes little". That's where all the complexity lies, and that's where most people go wrong.

Good practice: cache what is read a lot and changes little

The ideal cache candidate has two traits: it is accessed frequently and changes rarely. Think about a list of product categories, the configuration of a system, a user's public profile. This type of data is read all the time and updated every now and then, perfect cache.

The bad cache candidate is the opposite: data that changes every instant or whose accuracy is critical in real time. The balance of a bank account, the stock available at the time of purchase, the price in an active negotiation, caching can give the user a number that is no longer true, with real consequences.

The practical question before caching anything: what happens if the user sees a value that is a few seconds or minutes out of date? If the answer is "no big deal", cache. If the answer is "a serious problem," think twice.

The real problem: invalidation

Putting data in the cache is trivial. The difficult part is knowing when to remove, or update. This is the invalidation problem, and it's where almost every cache bug is born.

There are two basic strategies, and both have their place. The first is time expiration: the data lives in the cache for a defined period and is then discarded. Simple, robust and sufficient for most cases. You accept that the data may be out of date for, say, five minutes, and go on with your life.

The second is event invalidation: when the data changes, you actively remove or update the cached version. It is more precise, but more fragile, it requires that every change in data remembers to notify the cache, and all it takes is a forgotten path for the user to see old information indefinitely.

Best practice for a quick guide: prefer timed expiration whenever tolerance allows. It's simpler, more resistant to human error, and avoids the class of bug where the cache "forgets" to update. Reserve event invalidation for when the accuracy truly justifies the complexity.

Good practice: define what happens when the cache fails

Cache is an extra layer, and extra layers fail. The cache server may go down, become unavailable, or slow. The question that many people forget to answer: what then, does the system stop?

Well-done caching is an optimization, not a dependency. If the cache disappears, the application must continue working, slower, perhaps, but working, searching for data from the original source. When the entire system crashes because the cache crashed, you don't have an optimization; has a single point of failure disguised as a performance improvement.

There is also a treacherous detail: when the cache empties at once, all requests hit the original source at the same time, and the overload can destroy precisely what the cache protected. It's a known effect, and it's worth designing the recovery with it in mind, so that the system reheats the cache without drowning.

Critical reflection: cache hiding the wrong problem

There is a use of caching that is both technically correct and strategically lazy: caching to hide a poorly made query. The query is slow because it is poorly written or the database is poorly modeled, and instead of correcting the cause, cache is thrown over it. It works, until the cache expires, until the non-cacheable case appears, until the problem moves.

Cache should accelerate what is already efficient, not make up what is inefficient. When you find yourself using cache to make something bearable that should be fixed, it's worth stopping and looking at the cause. The cache, in this case, is postponing a debt, not paying it.

There is also the cognitive cost. Each cache layer is one more thing where data might be out of date, one more place to investigate when something is strange. "Why is this user seeing old information?" is one of the most frustrating questions to debug, precisely because the cache is invisible until there is a problem. Too much cache turns a simple system into a versioning puzzle. Use sparingly, document where you are, and prefer fewer well-understood layers to many mysterious layers.

What remains

Cache is a powerful tool and is equally often misused. Used well, it leaves fast and cheap systems. It is misused, it delivers the wrong information, hides real problems and creates bugs that are difficult to track.

Good practices fit into a few lines: cache what is read a lot and changes little; prefer timed expiration to manual invalidation; ensure that the system survives without the cache; and never use caching to hide a problem that should be fixed. The rest is fine tuning.

If you are facing performance problems and considering caching as a solution, it is worth first understanding whether the bottleneck is actually from repeated reading or something deeper. On the blog there are other texts about backend, performance and architecture that delve deeper into these choices.