See It Flow
Production Engineering

Cache Stampede

What happens when a popular cache key expires and thousands of requests simultaneously fall through to the database — and how engineers prevent it.

Mangesh MandlikLast updated:
01 · The Problem

What Is Cache Stampede?

A cache stampede occurs when a frequently-accessed cache entry expires and many concurrent requests simultaneously discover the empty cache, all independently falling through to the database at once.

In a healthy caching system, the vast majority of read requests are served directly from cache — Redis, Memcached, or any in-memory store. The database only handles the occasional cache miss and write operations. This keeps database load low and response times fast.

The problem arises at the moment of expiration. When a cache entry's TTL (time-to-live) reaches zero, the entry disappears. If traffic is low, the next request misses the cache, queries the database, and repopulates the entry. One miss. No harm done.

But under high concurrency, dozens or hundreds of requests can arrive in the narrow window between expiry and repopulation. Every single one discovers the empty cache independently. Every single one decides to query the database. The database receives a sudden burst of identical queries it was never designed to handle.

Cache stampede is often confused with two related but distinct problems. Cache penetration is repeated lookups for a key that doesn't exist in the cache or the database — every request misses and hits the database because there's nothing to cache. Cache avalanche is a mass expiration of many different keys at once (often after a cache restart or a bulk TTL set), overwhelming the database with queries for many distinct values. Cache stampede is narrower: concurrent requests for the same hot key during its single expiry window.

02 · The Mechanism

Why Does It Happen?

Three conditions must be true simultaneously: the key must be popular, the key must expire, and traffic must be high enough that multiple requests arrive during the miss window.

TTL Expiration

Every cached value has a TTL. When it expires, the entry disappears from the cache. This is expected behavior — TTLs prevent stale data. The problem isn't that the key expires. The problem is what happens next.

Concurrent Requests

Under high traffic, many requests arrive within milliseconds of each other. If a cache entry expires during a traffic spike, dozens or hundreds of requests will check the cache in the same instant — and they'll all get a miss.

Independent Rebuilds

Each request that misses the cache independently decides to query the database. There's no coordination between them. Request A doesn't know that request B is already rebuilding the same value. So they all query the database in parallel.

The critical insight is that this is a concurrency problem, not a cache problem. The cache (commonly Redis) is working exactly as designed. The vulnerability is the window between expiry and repopulation — and how many requests can arrive during that window.

Normal (1 miss)
1 req
→
Redis
MISS
→
DB
1 query
✓ Database handles it fine
Stampede (N misses)
500 req
→
Redis
500 MISS
→
DB
500 queries
✗ Database overwhelmed
Normal flow vs stampede: one miss is harmless, but N concurrent misses overwhelm the database. (500 is an illustrative count, not a measurement.)
03 · Animated Demonstration

Watch It Happen

Press play to watch a cache stampede unfold in a simulated system — from healthy operation through expiration, the stampede, mitigation, and recovery.

Production Engineering

Cache Stampede

⤢ Expand for synchronized narration timeline

04 · Impact

Why Is It Dangerous?

A cache stampede can take down a healthy system in seconds. The damage extends far beyond the single expired key.

When hundreds of identical queries hit the database simultaneously, they compete for CPU, memory, I/O bandwidth, and connection pool slots. The database wasn't sized for this burst — it was sized for the steady-state load that the cache was absorbing.

⏱️Latency Spike

Response times jump from milliseconds to seconds as the database queues hundreds of identical queries.

🔴Error Rate Increase

Connection pool exhaustion causes timeouts and 503 errors returned to users.

🔗Connection Pool Exhaustion

Every stampede request holds a database connection, starving all other queries — not just the stampeded key.

💥Cascading Failure

Application threads block waiting for the database, causing upstream services to timeout and fail.

How to spot it

A stampede leaves a recognisable trail. Look for these signals, which appear in this order within seconds of each other:

  1. Cache hit ratio drops sharply — a sudden dip, not a slow decline, and it recovers abruptly once the key is repopulated.
  2. Database QPS jumps for one query — the same SQL fingerprint (same normalised statement, near-identical parameters) dominates the statement statistics, while other queries are unchanged.
  3. Connection pool wait time rises — pool in-use hits its maximum and callers queue for a connection, so even queries for unrelated keys slow down.
  4. Latency, timeouts and 5xx follow — requests exceed the client timeout, errors appear, and retries add more identical queries.
Illustrative timeline (constructed for teaching, not measured data). Assumptions: one hot key serves 300 of 500 req/s, the baseline hit ratio is 98%, the rebuild query takes 3 s, the pool has 50 connections and the client timeout is 5 s.
  1. T+0 s: the key expires. Baseline DB load is 500 × 2% = 10 queries/s; now about 300 + 4 misses/s, so the hit ratio falls from 98% to about 39% (196 of 500 requests).
  2. T+0.2 s: the identical rebuild query is running 50 times, the pool is full (300 req/s ÷ 50 connections ≈ 0.17 s to fill it), and new callers start to queue.
  3. T+3 s: 300 × 3 = 900 requests have missed and only the first 50 queries finish. The cache is repopulated, but the other 850 requests are already queued and each still runs its own query (850 × 3 s ÷ 50 connections ≈ 51 s of database work).
  4. T+5 s: clients time out; most of the 850 queued requests return 5xx while their queries keep occupying the pool, which is why a stampede can outlive the key expiry that caused it.
05 · Mitigation

How Do Engineers Fix It?

There are four common strategies. Each trades a different cost for stampede protection. Production systems often combine two or more.

Request Coalescing (Single-Flight)

When multiple requests ask for the same missing key simultaneously, only one actually queries the database. The others wait and share the result.

How it works

The first request for a missing key acquires a 'slot' and starts the rebuild. All subsequent requests for the same key subscribe to the pending result instead of starting their own rebuild. When the first request completes, all waiting requests receive the same value.

Trade-off

Adds a small coordination layer. Waiting requests see slightly higher latency (they wait for the leader request), but the database sees exactly 1 query instead of N.

Distributed Locking

Acquire a lock on the cache key during rebuild. Only the lock holder queries the database. Others wait for the lock to release, then read from the now-populated cache.

How it works

When a request discovers a cache miss, it attempts to acquire a distributed lock (e.g., Redis SET with the NX and PX options). If it gets the lock, it rebuilds the value. If not, it waits briefly and retries — expecting the cache to be populated by the lock holder.

Trade-off

Requires a distributed lock mechanism. Risk of deadlock if the lock holder crashes without releasing. Must handle lock expiration carefully.

TTL Jitter

Instead of all keys expiring at exactly the same time, add a random offset to each TTL. Spreads expiration across a window, preventing synchronized stampedes.

How it works

When setting a cache entry, use TTL = base_ttl + random(0, jitter_range). For example (illustrative numbers), instead of TTL=120s for every key, use TTL=120 + random(0, 30) seconds. Keys expire at different times, preventing a burst of simultaneous misses.

Trade-off

Does not prevent a stampede on a single hot key: one key still has exactly one expiry instant, and jitter only desynchronises different keys (or per-node copies of the same key). Most effective when many keys would otherwise expire at the same instant (e.g., after a cache restart or bulk load).

Stale-While-Revalidate

Continue serving the slightly-stale cached value while refreshing it asynchronously in the background. Users never see a cache miss.

How it works

Before the TTL expires, a background process refreshes the value. The cache always has data — it may be a few seconds old, but it's never empty. Alternatively, the first request that discovers the value is 'soft-expired' triggers a background refresh while immediately returning the stale value.

Trade-off

Users may briefly see slightly stale data. Requires background refresh infrastructure. Best for data where a few seconds of staleness is acceptable (product catalogs, user profiles, aggregated counts).

06 · Takeaway

Engineering Takeaway

Key Takeaways
  • Cache stampede is a concurrency problem, not a cache problem. The cache is working as designed — keys expire, and that's expected.
  • The vulnerability is the window between expiry and repopulation. Under low concurrency, it's harmless. Under high concurrency, every request in that window falls through to the database.
  • Request coalescing is the most direct fix: N concurrent misses become 1 database query. The cost is minimal coordination overhead.
  • TTL jitter prevents synchronized expiration of many keys at once — especially important after cache restarts or bulk loads.
  • Stale-while-revalidate eliminates the miss window entirely by always serving data (possibly slightly stale) while refreshing in the background.
  • Think about what happens when a cache key expires under load — not just when it expires. The difference between 1 miss and 500 misses is the difference between a normal day and an outage.