Cache hit rate: What it measures, what it doesn’t, and when to stop using it

Cache hit rate divides the requests served from cache by the total. However, the same metric measures different things in different systems, and doesn’t necessarily show latency. This article covers the practical questions first, then the parts that rarely get explained.

What cache hit rate measures, and why it means something different 

A cache hit rate measures how often requested data is found in cache rather than fetched from the underlying source. A cache hit is a request the cache serves directly. A cache miss is a request that has to go back to the origin, the database, or slower storage. The hit rate is the share of requests that were hits, and the miss rate is everything else.

However, different systems use caches differently. 

• A CPU cache hit rate describes nanosecond-scale memory access against main memory.

• A database buffer cache hit ratio describes pages served from memory rather than disk.

• A Redis hit ratio counts key lookups.

• A content delivery network (CDN) cache hit ratio counts content requests served from an edge server instead of the origin, and even there it splits again into request-hit-ratio and byte-hit-ratio, which disagree when a few large objects miss. 

Each of these is called a hit ratio, and each is measured against a different denominator.

This means hit rate is only meaningful in the context of the system that produced it. A 70% figure that means trouble in a static CDN cache may be fine for a database buffer cache under a write-heavy workload. Before reading a hit ratio as good or bad, it helps to know which cache, which requests, and which denominator produced it.

How to calculate cache hit rate

Divide the number of cache hits by the total number of cache requests, which is hits plus misses:

hit rate = hits ÷ (hits + misses)

A system that serves 90 requests from cache out of 100 total cache requests has a hit rate of 0.90, or 90% once multiplied by 100. The miss ratio is the complement: 10 misses out of 100 requests is a 10% cache miss ratio, and each of those is a cache miss that had to be answered by slower storage. Most tools report the value as a percentage, though some express it as a decimal between 0 and 1. 

What's a good cache hit rate? It depends.

There is no universal target, because a good cache hit ratio is a property of the workload, not the cache. Static content that changes rarely sustains high ratios. A well-optimized CDN cache serving mostly images and stylesheets often achieves hit ratios above 95%, with some deployments exceeding 99%, because the same cached content answers most requests. 

Many real-world CDNs sit lower, in the 80–95% range, once personalization, cache rules, and cookies enter the picture. In-memory caches fronting a database vary just as widely: many production teams investigate a sustained hit rate below roughly 80%, though the right threshold depends on the workload rather than any industry standard.

Dynamic and personalized workloads tell a different story. Content assembled per user, priced in real time, or invalidated on every write will show a lower hit rate, sometimes 20–60%, without anything being wrong. A checkout flow that needs the authoritative price cannot safely serve a cached value, so those cache requests are misses by design. Judging that system against a 95% benchmark would push an engineer to cache data that should never be cached.

A target is therefore only useful once the workload is understood. The more productive question is not "what number should I hit" but "what is this cache for, and what breaks if it misses?" 

Is your hit-rate number accurate?

Before optimizing a hit ratio, confirm that the metric measures what it seems to. Several common reporting methods produce figures that can be misleading. 

1. Cumulative counter. Redis reports keyspace_hits and keyspace_misses as totals accumulated since the server started. A ratio computed from them reflects the lifetime of the process, so a change made today is diluted by weeks of prior traffic and appears to do nothing. One reported case involved raising a PostgreSQL instance's shared_buffers from 0.5GB to 4GB and seeing the cache hit ratio sit unchanged six days later.¹ The configuration was fine, but because the counter had not been reset, it was reporting old data. Windowed measurement over a recent interval, taken after resetting the counters, shows current behavior.

2. Double buffering. A PostgreSQL buffer cache hit ratio counts pages found in the database's own shared buffers, but many operating systems also maintain a page cache beneath the database. A "miss" at the database layer may still be served from RAM by the OS, so the reported ratio understates how much data is in memory.² Raising shared_buffers may lower the reported ratio, though overall memory access barely changes.

3. Invalidation accounting. A cache that treats every invalidation or cache-tag check as a miss will report a lower hit rate than it delivers, even though traffic is served effectively. The same distortion appears at the CDN edge: A request that misses the first-tier cache but is served by a second tier is often still logged as a miss, so a multi-tier cache looks weaker than it performs.³

The upshot is to measure over a defined window, reset counters before testing a change, and confirm what the system counts as a hit before trusting the ratio.

Why a higher hit rate doesn’t always mean better performance

Raising the hit rate is usually treated as good, but not necessarily, for two reasons. 

The first is a measurement problem. When a hit rate becomes the target rather than a diagnostic, teams optimize the number instead of the outcome. Over-provisioning memory to push a ratio from 96% to 98% costs money while doing nothing for user response time. 

The second is more surprising, because it is a performance reversal rather than a reporting artifact. Research from Carnegie Mellon, Can Increasing the Hit Ratio Hurt Cache Throughput?, shows that for many caching algorithms, raising the hit ratio lowers throughput and worsens response time, due to contention on the hit path.⁴ Serving a hit is not free; it takes locks and updates metadata, and algorithms that work harder for a higher hit ratio create more of that contention per request. Past a point, the extra hits cost more in coordination than they save in avoided misses. The size of this effect varies between cache implementations and concurrency models, but the paper notes it is likely to grow as storage gets faster and core counts rise, because the gap between a cheap miss and a contended hit narrows.

This means the hit rate cannot be optimized in isolation. Throughput and response time are the outcomes; the hit ratio is one input among several, and focusing on it may move numbers the wrong way.

Why a 99% hit rate still produces poor tail latency

The clearest illustration of this problem is what a high hit rate does to tail latency. A high average hit rate says almost nothing about the worst requests, and the worst requests are what users notice. 

Consider one operation with a 99% hit rate. 99% of the time, it is served from cache memory in a fraction of a millisecond; 1% of the time, it falls through to slower storage. If a cache hit takes 0.5ms and a miss takes 50ms, the average looks excellent at roughly 1ms, but one request in a hundred is a hundred times slower than the rest. That is the p99 latency that shows up in user experience and in alerting.

The problem gets worse when one user request depends on several cache lookups. A request that performs five independent operations, each with a 99% hit rate, completes entirely from cache only when all five hit. Assuming the lookups are independent, that probability is 0.99 multiplied by itself five times, or about 95%. The per-operation hit rate looks pristine, but one request in twenty now calls slow storage at least once. 

Correlated accesses affect the figure, but not enough. Raising each operation to 99.9% helps, but across enough operations and enough traffic, the tail doesn’t disappear; it just gets thinner.

This is why hit rate and tail latency have to be read together. A system tuned for a high average hit ratio may still deliver an inconsistent experience, because the misses cluster into the requests that do the most work. For any application where predictability matters more than average speed, the tail is the metric, and the hit rate is only a loose proxy for it.

What happens when your cache goes cold

The bigger problem is when those misses arrive all at once. Suppose a database handles 300 requests per second and a cache in front of it holds a 90% hit rate. The combined system serves 3,000 requests per second, because only 10% reach the database. But if the cache empties or fails, every request becomes a miss. The database that comfortably handles 300 requests per second is suddenly asked for 3,000, and it cannot. Requests queue, time out, and retry, and the retries add more load, which slows the database further, which causes more timeouts. The system stays down even after the original trigger passes. 

This is a metastable failure: a stable state under normal load flips into a stable state of failure that sustains itself. Research cataloguing these failures across large production systems identifies caches as a trigger because a high hit rate lets normal load grow beyond what the backing store can handle on its own. 

DynamoDB's engineers documented this issue. Their metadata cache ran at roughly a 99.75% hit rate, which meant a cold cache would force the metadata store to scale from serving 0.25% of requests to 100%, a jump of several hundred times.⁵ Their fix was not a higher hit rate but a different design. Request routers called the metadata service on both hits and misses, doing constant work regardless of cache state, so the downstream load never depended on the hit ratio. Caching efficiency was traded away in exchange for removing the cliff.

The backing store should be sized to survive a 0% hit rate, or the cache should be designed so a miss storm cannot form. 

Why adding more cache memory sometimes does nothing

When a hit rate is disappointing, the instinct is to add cache memory. Sometimes that helps. Often it does nothing. Here’s why.

The first reason is measurement: If the ratio is computed from cumulative counters that were never reset, more memory may work but the lifetime number barely moves. The change is real but isn’t shown by the metric.

The second is the type of workload. Cache eviction under a least-recently-used policy has a structural ceiling on workloads dominated by items requested once and never again. No matter how much cache memory is added, those items fill space without ever producing a second hit, so the hit rate plateaus. As the working set grows toward the size of the cache, the policy begins evicting entries that are still in use, and adding a little more memory produces almost no improvement until the entire working set fits. Below that threshold, the extra memory is essentially wasted.

Hit rate is a function of workload first and capacity second. A cache holding data that is rarely re-requested will not improve with size, because the limiting factor is the workload, not the memory. 

Why eviction plateaus 

When the workload is the problem, changing the eviction policy often helps. LRU eviction is the default in most caching systems because it is simple and effective: Throw out what has gone longest untouched. It performs well on typical skewed, Zipfian access distributions, where a few popular items account for most requests and recency tracks popularity. 

Its weakness shows when a workload is different. Recency is a crude proxy for future value, so a one-hit-wonder that just arrived looks fresh to LRU and displaces an item that is accessed steadily but happened to be idle for a moment. As a workload becomes dominated by streaming or one-hit-wonder objects, LRU's hit rate flattens below what the cache memory could support.

Frequency-aware policies handle this by admitting and retaining data based on how often it is used, not just how recently. Least-frequently-used eviction keeps items proven popular. Some designs go further: the LHD policy from Beckmann, Chen, and Cidon ranks each object by its hit density, or the expected hits it produces per unit of space it occupies.⁶ Lightweight policies such as S3-FIFO and SIEVE filter out one-hit-wonders before they displace more valuable entries. S3-FIFO has been shown to outperform LRU across many evaluated workloads.⁷

No one policy works all the time. 

Cold start and cache warming start you at zero

Every cache begins empty, and an empty cache has a 0% hit rate. On a first deploy, this is expected, but the same reset happens after every restart, every configuration change that flushes the cache, and every scale-out event. Ironically, adding a cache server to handle rising traffic introduces a node that starts cold, so the extra capacity arrives with the worst possible hit rate when the system is under the most pressure, unless the cache is explicitly warmed or rebalanced. 

Distributed caches often soften this by replicating entries, rehashing keys across nodes, or prewarming a new node, so not every deployment starts from zero. 

You can tell this is happening when there’s a dip in the hit ratio after each deploy, followed by a slow climb as the cache refills from live traffic. During that window, more cache requests fall through to the origin, latency rises, and the backing store carries load it does not normally see. On a large cache or a slow-filling workload, the climb may take a long time, and if it coincides with a traffic spike, the cold node may trigger a miss storm.

Cache warming addresses this by populating the cache before it serves live traffic. Warming can be “eager,” by loading a known working set at startup, “lazy,” by accepting the cold period and letting traffic fill the cache naturally, or “scheduled,” by preloading anticipated data ahead of a known event. 

The right choice depends on how costly the cold window is. A system where a cold start risks cascading failure needs eager or scheduled warming; a low-stakes cache tolerates lazy filling. Either way, don’t treat a new node as instantly ready. 

Practical tactics that move your hit rate

Once the workload and the metric are understood, several tactics reliably improve a hit ratio. 

• Cache-Control headers and time-to-live (TTL) settings are the primary configuration settings for a CDN cache and most HTTP caching. Longer TTLs keep cached content eligible to serve more requests before it expires, which raises the hit rate. The constraint is staleness: the TTL should match how often the underlying data changes, with content-hash versioning used to allow long lifetimes on assets that change unpredictably.

• Cache key hygiene is the next setting. When query strings, cookies, or other high-variance attributes are folded into the cache key, near-identical requests are treated as distinct objects, and each first request is a forced miss. Normalizing keys so only meaningful variation creates a new entry improves a hit ratio with no additional cache memory.

• Near-caching keeps frequently accessed data inside the application process itself, avoiding a network hop to the cache server for the hottest items.

• At the CDN layer, origin shielding designates one tier to face the origin so other edge locations pull from it rather than the origin, raising the effective hit rate and cutting server load. 

These are refinements to an effective caching strategy, not substitutes for understanding the workload underneath.

Predictable latency, regardless of hit rate: a case study

For some systems, the most useful move is to stop optimizing the cache and change how misses are handled. A system where a miss is expensive must chase an ever-higher hit rate and live with the fragility that creates. But a system where a miss is cheap can stop chasing the number. 

This is the design premise behind Aerospike's patented Hybrid Memory Architecture, which keeps an index in memory while serving data from optimized flash storage with consistently low latency, reducing the performance difference between cache hits and backing-store reads. Consistent response time does not depend on maintaining a high cache hit ratio, because the slow path is more predictable and less variable than a conventional database fallback. Fraud detection provider BioCatch runs on this model to deliver predictable latency regardless of cache hit rate, where a variable or low hit rate would otherwise mean inconsistent decision times on real-time traffic.

Architecture can eliminate a problem that tuning can only manage. When the backing store is fast and predictable, the tail latency from misses shrinks, the metastable-failure cliff flattens, and cold starts stop being an issue, because a 0% hit rate is no longer a crisis. The hit rate becomes a cost-efficiency detail rather than a reliability dependency. 

Where Aerospike fits

A cache hit rate is a useful signal. It shows whether caching is doing work; it points to configuration problems, and it tracks cost as traffic served from cache avoids load on the origin and the database. 

The mistake is turning that signal into a target. The metric is inaccurate when counters are cumulative, when the OS page cache double-buffers underneath, and when invalidations are miscounted. A higher hit ratio may reduce throughput through contention on the hit path. An acceptable average hit rate still leaves a tail of slow requests, and that tail compounds across multi-operation requests. Highest of all, a high hit rate builds a dependency that turns a cold cache into a cascading outage. If the hit rate is the only number being watched, these issues don’t show up, and they’re all caused by a slow backing store. 

Aerospike is built to remove that root cause. It is a distributed database that keeps its primary index in memory and serves data from optimized flash at consistent, low latency, so the path a cache miss falls back to is fast and predictable. That resolves problems. Tail latency shrinks, because the slow path is no longer much slower than the fast one. The metastable-failure cliff flattens, because the backing store handles the load a cold cache would otherwise redirect to it. Cold starts stop being a problem, because a 0% hit rate is no longer a crisis. The hit rate becomes a cost-efficiency detail rather than a reliability dependency.

This is why teams running real-time workloads, where inconsistent response time is the failure that matters, use Aerospike rather than bolting a cache in front of a slower database and hoping the hit rate holds. The question stops being "how high can we push the hit rate" and becomes "what does this cost, and can the system survive a miss?" When the answer to the second question is yes, the first stops being a source of risk.

If your architecture currently depends on a high cache hit ratio to meet latency targets, that dependency is worth examining. Aerospike handles high-throughput, low-latency workloads.