Distributed Cache System Design: From One Server to a Resilient Cache Cluster

A cache is a small, fast, disposable copy of data that lives somewhere cheaper to reach than the place the data is really kept. A distributed cache is that copy spread over several machines so that it can hold more, serve more and survive the loss of a node. Everything in this page follows from one tension: the copy makes reads fast, and the copy is, by construction, sometimes wrong.

Meridian Commerce is a fictional storefront, and every number on this page is a teaching input rather than a measurement. Its product database is a relational database and the authority for prices, descriptions and stock. A fleet of Product API pods answers shoppers. On an ordinary day the fleet serves about 12,000 requests per second; at a flash-sale peak it serves 60,000. Most of those requests ask the same few questions again and again: what is product 4821, what does it cost, is it in stock. The database computes the same answer thousands of times a minute, and the answer has not changed since the last time.

Two words need pinning down before we go further. A hit is a request answered from the cache; a miss is a request the cache cannot answer, so the application goes to the database and (usually) stores the result for the next caller. The hit rate is hits divided by cache lookups. A cache earns its keep when the hit rate is high and a miss is not catastrophic.

Latency is the second pressure. A read that must cross to the database, plan a query, touch storage and return might take about 85 ms in our illustrative numbers; a read served from an in-memory cache one network hop away might take about 3 ms. Neither figure is a vendor benchmark. They are chosen so the arithmetic later in the page is easy to follow and so that the gap between them matters.

Three truths will recur. Authoritative writes belong to the source of truth. A cache hit is not the same as a fresh, complete or authorized answer. And distributing an in-memory store adds failure modes of its own, even while it makes reads faster. We build the design one weakness at a time so that each new box is there because the previous design broke.

Where this cache sits among the other caches

A request can be answered from several caches before it reaches the database, and each solves a different part of the problem. The browser cache and the CDN cache sit in front of the API and are driven by HTTP caching headers; the application cannot reach in and delete one entry on demand. An application-local cache lives in each pod's memory. A distributed cache is a shared service that every pod uses. This page designs the last one, and the others remain useful complements.

This page also carries an interactive Visual Design Journey with 44 scenes that animates the same progression: requests, hits, misses, invalidations, shards and failovers moving through the architecture. Read the chapters for the reasoning and use the journey to watch the interleavings that are hard to hold in your head.

Before choosing any technology, decide what the cache is allowed to hold and how wrong it is allowed to be. A cache for product descriptions and a cache for payment tokens are different systems even if they run on the same servers.

What the cache must do

Functionally, the service stores a value under a key, returns it on lookup, expires it after a time to live (TTL, the number of seconds after which an entry must no longer be served), and can delete it early when the authority says it changed. Non-functionally, Meridian wants most reads to be served in single-digit milliseconds, the database protected from the peak, a node failure to degrade rather than stop the storefront, and one tenant's behavior never to harm another's. Tenants are merchants who sell through the platform: Cedar is a large one that can be noisy, Birch is a small, well-behaved one.

Not everything deserves the same freshness

The useful question is how stale each kind of data can be before someone is harmed. Meridian sorts its data into classes, and the rest of the design takes its policy from the class.

The displayed price is non-binding: checkout always revalidates price and availability against the authoritative service. That one rule is what lets C2 tolerate a bounded delay at all. If the card shows $99 for a few seconds after the database says $109, the customer is told the real price before paying.

Scope

In scope: the read path, write and invalidation ordering, expiry and eviction, stampede and hot-key protection, sharding and routing, replication and failover, tenant isolation, regions and operations. Out of scope: building the product database, the search index, a CDN (see CDN fundamentals for edge caching of static assets), and any promise that a cache can act as a system of record. Read-heavy data tolerates a cache well; a write-heavy dataset whose entries are rarely re-read gains little, because every write forces an invalidation or a refill that no later read will use.

Capacity numbers are what turn a vague intention to add a cache into a design you can argue about. All of the following are illustrative inputs; real deployments measure them.

Traffic and hit rate

Take 12,000 requests per second on average and 60,000 at peak. Suppose 80% are reads of reusable product data, so they are cache-eligible, and the rest (cart changes, checkout, anything personalised) are not. At a 90% hit rate on the eligible reads, the average day looks like this: 12,000 × 0.80 = 9,600 eligible requests per second, of which 8,640 are hits and 960 are misses; the other 2,400 non-eligible requests per second go straight to the database. That is 960 + 2,400 = 3,360 database-facing requests per second, assuming every non-eligible request touches this database and nothing else does. Without a cache the database would see all 12,000, so the cache cuts database load to 28%.

At peak the same mix gives 48,000 eligible requests per second, 43,200 hits, 4,800 misses and 12,000 non-eligible, so 16,800 per second reach the database against 60,000 without a cache. The reduction is 72% in both cases. Note that this is not the same number as the hit rate. A 90% hit rate on eligible reads does not mean a 90% cut in backend load, because the 20% of traffic that is not eligible never benefits.

Suppose, as a teaching input and not a benchmark, that the database comfortably sustains 24,000 reads per second. Peak without a cache is 60,000, which is 2.5 times that capacity. Peak with a cache is 16,800, or 70% of capacity, under a conventional 80% operating ceiling of 19,200 per second. The cache is not an optimization here. It is what makes the peak survivable, and that is exactly why its failure later becomes dangerous.

Latency is a mixture

With a 3 ms hit and an 85 ms miss, the mean over eligible requests at a 90% hit rate is 0.9 × 3 + 0.1 × 85 = 2.7 + 8.5 = 11.2 ms. The mean looks respectable, but one request in ten pays 85 ms, so the 95th and 99th percentiles (p95 and p99: the value below which 95% or 99% of requests fall) sit at roughly the miss latency, about 85 ms. Averages hide tails. Quoting 3 ms or 11.2 ms to a user whose p99 is 85 ms misleads, and neither number is the full API latency, which also includes the rest of the request.

Memory: payload is not footprint

The dataset has 8,000,000 potentially hot objects of about 1.5 KiB each. 8,000,000 × 1.5 KiB = 12,000,000 KiB, which is 11.44 GiB (a gibibyte is 2³⁰ bytes) or about 12.3 GB (decimal gigabytes, 10⁹ bytes). Those are payload bytes only. Now apply an *assumed* overhead model: a key of about 40 bytes and about 64 bytes of hash-table and pointer metadata per entry make 1,536 + 40 + 64 = 1,640 bytes per item, or 12.22 GiB across 8 million items. A further assumed 1.25 factor for allocator fragmentation brings it to 15.27 GiB.

Spread across 4 primaries, that is 3.82 GiB per shard on average. If the unluckiest shard holds 15% more (uneven distribution), it needs 4.39 GiB, 54.9% of an 8 GiB budget. The sample cluster of 4 × 8 GiB is 32 GiB nominal, so the average utilization is 47.7%. Add one replica per primary and the memory bill doubles to 30.5 GiB across 8 nodes, because a replica holds a full copy. Finally, replication buffers, client buffers and any snapshot or fork behavior need headroom of their own. The point of the worksheet is not the answer. It is that the number you must provision is meaningfully larger than the number you started with, and every factor in it must be measured with the engine's own tooling.

The simplest design is also correct: the shopper asks a Product API pod, the pod runs a query, the answer is returned. No copy exists, so no copy can be stale.

That correctness is worth stating, because every later design spends some of it. In V0 a price change committed at 10:00:00 is visible to the next read at 10:00:00. The costs are the numbers from the last chapter. At the flash-sale peak, 60,000 requests per second against a database that handles 24,000 means requests queue, connection pools fill and latency climbs for everyone, including the reads of data that nobody was contending for. Because the traffic is dominated by repeats of the same few hot products, almost all of that work is redundant.

It is tempting to answer with a bigger database. Vertical scaling helps, but it buys capacity for the repeated work rather than removing it, and a primary relational database is among the more expensive and harder-to-scale parts of a stack (see database scaling). Read replicas help with read volume but still execute the same queries. The cheaper fix is to stop asking the question twice.

The first cache most teams build is a hash map in the application process, often called an L1 cache (level one: the nearest, smallest cache). A read checks the map, and on a miss queries the database and stores the result with a TTL.

It is attractive because it costs one function call: no network hop, no new service, no serialization. A local hit is faster than a shared-cache hit, which still pays a network round trip. The warm-up behavior is the first surprise. A freshly started pod has an empty map, so it misses on everything and every deploy or autoscale event pushes a wave of cold misses to the database. The second surprise is multiplication. With 20 pods (a hypothetical fleet size), product 4821 can be fetched from the database once per pod per TTL window rather than once in total, and memory is spent holding 20 copies of the same hot data.

The decisive weakness is divergence. When the price moves from $99 (record version v5) to $109 (v6), the pod that handled the write may update its own map, but the other 19 do not know. Shopper requests round-robin across pods, so the same shopper can see $99, then $109, then $99 again on refresh. Invalidation is the hard part: to purge the key from every process you need a broadcast channel, each pod must subscribe and apply it, and any pod that misses the message keeps serving the old value until its TTL runs out.

A local cache is not wrong, merely limited. It is an excellent *second* layer for a few extremely hot keys, with a deliberately tiny TTL, which we return to in the hot-key chapter. As the primary cache for a fleet it forces a choice between very short TTLs (little benefit) and long TTLs (visible disagreement).

Moving the cache out of the pods into a separate service that all pods share fixes both V1 weaknesses at once: there is one copy of each entry, and one place to delete it.

A pod asks the cache for tenant:cedar:product:4821 with a GET. On a hit the value comes back in a couple of milliseconds. On a miss the pod reads the database, then SETs the value with a TTL so that the next caller hits. With 20 pods sharing the cache, a cold key is fetched once rather than twenty times, and the 72% reduction in database traffic from the capacity math becomes possible. A deploy no longer empties the cache, because the cache outlives the pods.

Two products are commonly used for this role. Redis is a networked in-memory data store with rich value types, replication and a cluster mode. Memcached is a simpler in-memory key-value cache with a deliberately small command set. We will keep the page vendor-neutral where we can and point out specific behaviors where the engines differ. Memcached has no built-in clustering or replication in its typical deployment (check the documentation for your version); distribution across memcached servers is done by the client library hashing keys across servers.

The price of centralisation is a new dependency. Every cacheable read now crosses the network, so a slow or unreachable cache becomes a latency problem for the entire storefront, and a *single* cache server is a single point of failure whose loss sends 48,000 eligible requests per second at the database at peak. The cache is also a finite pool of RAM shared by all tenants and all classes of data. The remaining chapters take those problems in order: freshness first, then memory, then load concentration, then partitioning, then availability.

Cache-aside (also called lazy loading) is the pattern most systems use: the application owns the logic. It reads the cache; on a miss it reads the database, fills the cache, and returns. On a write it changes the database and then removes or refreshes the cached entry. The cache never talks to the database by itself.

There are three siblings of this pattern worth naming. Read-through puts the loading logic inside the cache layer: the application asks the cache and the cache fetches from the database on a miss. It centralises the loader (and can coalesce concurrent loads) but needs a cache product or library that supports it. Write-through writes to the cache and the database together, synchronously, so reads see fresh data but every write pays both latencies and a cache failure can fail the write. Write-behind acknowledges the write after updating the cache and persists to the database later; it is fast, but the cache briefly holds the only copy, so losing the cache loses acknowledged writes. For Meridian's product catalog, where the database is the authority, write-behind is the wrong tool. Cache-aside with invalidation is the default.

Which order for a price change?

When the price goes from $99 (v5) to $109 (v6), there are two obvious orders. *Delete the cache, then write the database* leaves a window in which a reader misses, reads the old row from the database, and fills the cache with $99 again, after which the write lands. *Write the database, then delete the cache* is better, since the window where the cache is wrong is the short gap between commit and delete. Neither is perfect, and the second still has a trap.

Why can the delete be lost? If the application process crashes between the commit and the delete, nothing remembers that a delete was owed. The robust answer is a transactional outbox: the price update and a row describing the change commit in the same database transaction, and a separate relay publishes that row to an invalidation stream after commit. A subscriber on the stream, which has authority to touch the cache, deletes or refreshes the key. An event does not edit caches by magic. It is a message that a subscriber must act on, and it can be delayed, duplicated or lost, so the TTL stays on as a backstop. (For the outbox idea see the outbox pattern.) Duplicate delete events are harmless because deleting an absent key is idempotent.

The refill race

Even with post-commit invalidation, a reader can undo it. Reader R misses on 4821 and reads the database, getting $99 (v5). Before R writes to the cache, the price update commits ($109, v6) and the subscriber deletes the key. R then completes its fill and stores $99 (v5). The cache now holds stale data *after* the invalidation ran, and no further event will correct it. It stays wrong until the TTL expires: up to about 360 s for the card price.

There are two honest remedies. One is to accept it: bound the damage with the TTL backstop and the non-binding display rule, which is a perfectly reasonable choice for descriptive content. The other is a version-aware conditional fill.

Each cached value carries a recordVersion that came from the *authority*; a timestamp taken by a pod is not good enough, because pod clocks disagree. A fill says 'store this only if the cached version is lower or the key is absent'. On delete, the subscriber can leave a short-lived tombstone recording the new version (v6), so a late v5 fill is rejected. The conditional step must be atomic on the cache node: in Redis that is done with a server-side script or a transaction-style command, and in Memcached with the cas command and its compare-and-set token. Atomicity is per command and per node, which will matter when we shard.

An entry can disappear because it *expired* (its time was up, a policy decision) or because it was *evicted* (the cache ran out of memory and had to choose a victim, a resource decision). They are separate mechanisms with separate failure modes, and confusing them leads to mysterious hit-rate drops.

Expiry: TTL as a backstop

A TTL says: do not serve this entry after N seconds. It bounds staleness even if every invalidation message vanishes, which is why Meridian keeps one on every class. It is not write synchronization; it does nothing at the moment the database changes, and a long TTL on volatile data simply keeps wrong data longer. In Redis, expiry is both *passive* (a key is checked when it is accessed and removed if expired) and *active* (the server periodically samples keys that have an expiry and deletes the expired ones), so an expired key can briefly occupy memory but is never returned. In Redis replication the primary decides when a key expires and sends an explicit delete to its replicas. Memcached treats a non-zero expiry as a guarantee that the item will not be retrievable after it elapses; an expiry of zero means 'no expiry', although the item may still be dropped to make room for others. Values above 30 days are interpreted as an absolute Unix timestamp rather than seconds from now, an easy mistake to make.

Eviction: choosing a victim

A cache has a memory limit. When it is full and a new entry arrives, an eviction policy chooses which existing entry to drop. LRU (least recently used) drops the entry untouched for the longest time, betting that recent use predicts future use. LFU (least frequently used) drops the entry with the fewest accesses, which resists being flushed by a one-off scan but needs a way to forget old popularity. Random eviction is cheap and works acceptably when access is close to uniform. TTL-based policies evict the entries closest to expiry.

Real engines usually approximate. Redis's LRU samples a small number of keys (configurable via maxmemory-samples) and evicts the best candidate among them rather than keeping an exact global ordering; its LFU uses a small probabilistic counter that decays over time. Redis offers policies prefixed allkeys- (any key may be evicted) and volatile- (only keys with an expiry), plus noeviction, which rejects writes when memory is full. A volatile-* policy on a dataset where nothing has a TTL behaves like noeviction and writes start failing. The documentation describes allkeys-lru as a good default if you have no specific reason to choose otherwise; verify the policy names for your version, since newer releases add more. Memcached allocates items into *slab classes* by size and keeps an LRU, so a shift in object sizes can leave memory stranded in the wrong class unless pages are rebalanced (it offers slab reassignment and automove, documented as subject to change).

Why both mechanisms? TTL alone is not capacity control: 8 million items with no eviction policy can exhaust memory long before TTLs fire. Eviction alone is not freshness control: a rarely evicted hot entry can stay stale forever. One consequence worth seeing is size: one 1 MiB value costs as much memory as roughly 680 entries of 1.5 KiB, and if eviction is by count rather than bytes a few giant values quietly push out thousands of small ones. That is why Meridian rejects values over 1 MiB, and caps keys at 250 bytes, a limit that happens to match memcached's own.

Negative caching

A lookup for a product that does not exist misses every time and always reaches the database. When bots probe random identifiers on purpose, the pattern is called cache penetration: the cache offers no protection because there is never anything to cache, so the attack is cheap for the attacker and expensive for you. A negative cache entry stores 'not found' for a short while, 30 s for Meridian, which absorbs repeats of the same missing id; ids that never repeat need validation or rate limiting rather than caching. Two guards matter. Cap the number of negative entries per tenant, otherwise an attacker enumerates IDs and evicts real entries with junk. And never store an authorization failure (401 or 403) as 'not found' for other callers: whether a product is visible depends on who is asking.

A cache stampede happens when many requests miss the same key at the same moment, and all of them go to the database to do identical work.

Product 4821 is Meridian's hottest item; assume it takes 18% of the 48,000 eligible reads per second at peak, which is 8,640 requests per second (the hot-key chapter uses the same figure; it only coincides with the average-day hit count in the sizing table). Its entry expires. Until the first refill completes, every request misses. With an 85 ms miss path, 8,640 × 0.085 ≈ 734 requests arrive before the first one fills the cache, and all 734 query the database for the same row. In a flash-sale variant, 5,000 concurrent requests hit one expired key at once. The database does the same work 5,000 times, slows down, so the refill takes longer, so more requests pile up: the cache that existed to protect the database has become the trigger.

Single-flight and leases

Single-flight (request coalescing) lets one request fetch while the others wait for its result. Inside one process this is a map from key to an in-flight promise. With 20 pods that gives at most 20 database fetches instead of 5,000, about 250 waiters per pod. A distributed lease goes further: the pod that wins an atomic 'set if absent' with a short expiry on a lock key does the fetch, and others wait or serve stale data. That brings the database cost close to one fetch.

The edges matter. The lease needs an expiry or a crashed leader blocks everyone forever; but if the expiry is shorter than the fetch, a second leader starts, so two leaders can briefly fetch the same key, which is a race on lease expiry. Waiters must have a deadline and a fallback (serve a stale copy for classes that allow it, otherwise fail fast), because an unbounded wait is a thread-pool outage in disguise. And locks do nothing for a different shape of the same problem, in which thousands of *different* keys expire together.

Jitter, stale-while-revalidate and early refresh

If 6,000 keys are warmed at the same instant with an identical 300 s TTL, they all expire at the same instant. That is a cache avalanche: many different keys going cold together, as opposed to the stampede of many readers on one key. (The same word is sometimes used for a whole cache layer failing, which chapter 17 covers.) Jitter randomises the TTL: uniform over 240–360 s, a spread of 120 s, turns that into roughly 6,000 / 120 = 50 expiries per second. Stale-while-revalidate serves the just-expired value to callers while one background request refreshes it, which is allowed only for classes that tolerate it (C1 up to five minutes, never permissions). *Probabilistic early refresh* lets a request volunteer to refresh slightly before expiry, with probability rising as expiry nears. Jitter does not fix a single hot key, a mass purge or simultaneous invalidation, or a cold cluster. Those need coalescing, rate-limited refilling and capacity planning.

A cache read can be wrong in only a few ways, and naming them helps you pay only for the guarantees you need.

Return to the price change. The commit lands at time 0. The outbox relay publishes in well under a second in a healthy system, the subscriber deletes the key, and the next read misses and fetches $109 (v6). In the normal case propagation takes seconds. If the event is lost, the TTL backstop caps the worst case at about 360 s. During that window a shopper can see $99 while the database says $109, and a second shopper on a different pod or region may see something else. That is eventual consistency with a bounded staleness budget, not a stronger promise.

Business invariants decide what is safe. Because checkout revalidates price and inventory, a stale card price costs a moment of confusion, not a wrong charge. A permission cached for a day after revocation would be a security incident; so authorization decisions are C4 and re-check authority (or compare against a versioned *revocation epoch*) on every use. The cache is permitted to be stale where staleness is cheap and the system is forbidden from asking it questions whose wrong answers are expensive.

Notice what versioning does not fix: it cannot make a pod that never receives the invalidation correct, and it adds storage (tombstones) and an extra comparison on every fill. Pair it with the TTL, and with alerting on invalidation lag, the time from commit to the key being gone, which is the real freshness metric.

One cache node has a ceiling. In the sample worksheet the whole dataset needs about 15 GiB of memory and the peak issues 48,000 GETs per second, and a single machine and a single network card are the limit on both. Sharding splits the keyspace into partitions, each owned by one node (a shard), so that memory and traffic divide by the number of shards.

The question is the placement function: given a key, which shard? Everyone who touches the cache, whether clients, a proxy or the cluster itself, must agree on the answer.

Modulo hashing and why it hurts

The simplest function is hash(key) mod N. With three shards, a key goes to shard hash mod 3. It is even and trivial, and it breaks when N changes: moving from N to N+1 shards changes the owner of N/(N+1) of keys, so 3→4 reassigns 75% of all keys, 4→5 reassigns 80%. Reassigned keys become instant misses on their new owners, so resizing the cluster is the same as flushing most of it, and the database takes the stampede.

Consistent hashing

Consistent hashing places both nodes and keys on a ring of hash values. A key belongs to the first node token clockwise from its hash. Adding a node takes over only the arcs just before its tokens, so only about 1/(N+1) of keys move when the ring is balanced. In the toy ring below (a hash space of 0–99, not real hash values) adding node D takes over 20 of 100 positions, all from B, and product 4821 stays on C. The split ends up uneven because a handful of tokens per node is a coarse slicing, which is why real rings give each node many virtual nodes: load evens out and a departing node's keys scatter across many survivors. The Architecture Deep Dive walks the example token by token.

Fixed hash slots

Redis Cluster uses a different method, not consistent hashing. The keyspace is divided into 16,384 hash slots; a key's slot is CRC16(key) mod 16384, and each primary node is assigned a set of slots. The slot-to-node table is explicit, and resharding means migrating whole slots between nodes. With three primaries splitting evenly, A owns slots 0–5460, B 5461–10922 and C 10923–16383 (5,461 + 5,462 + 5,461 slots). The key product:4821 hashes to slot 13107, so C owns it, while the tenant-scoped tenant:cedar:product:4821 lands in slot 8298 on B.

One feature deserves care. A {...} section in a key is the only part that is hashed, so related keys can be forced into one slot, which is what lets multi-key commands work on them. It also puts everything with that tag on one shard: a hot tenant becomes a hot slot, and a slot is the unit of migration, so it cannot be split. Use tags for small groups of related keys, not whole tenants; the Deep Dive shows the slot arithmetic.

Sharding spreads *different* keys across nodes. It does nothing for one key everyone wants, because that key still lives on exactly one shard.

At peak, 48,000 GETs per second spread evenly across 4 shards is 12,000 each, 48% of a hypothetical per-shard budget of 25,000 operations per second. But suppose product 4821 takes 18% of eligible reads: 48,000 × 0.18 = 8,640 requests per second on one key. The shard that owns it then serves 8,640 plus a quarter of the remaining 39,360, that is 8,640 + 9,840 = 18,480 per second, 73.9% of its budget, while its three siblings serve 9,840: a 1.88× skew. (The 8,640 here is the hot key's rate, coincidentally the same figure as the average-day hit count; they are different quantities.) A trending SKU taking 45% means 21,600 on one key and 28,200 on its shard, over budget while the others idle at 6,600.

This is *load skew*, and it differs from *dataset skew* (some shards simply hold more bytes). Adding shards fixes the second and not the first. The remedies move reads closer to the caller. A tiny L1 cache in each pod for just the hot key, with a TTL of 2 s, bounds shard reads for that key at 20 pods ÷ 2 s = at most 10 per second, a drop from 8,640. Alternatively, store the hot key under several suffixed names and read from a random copy, or serve it from replicas. Each trick buys load relief with staleness risk (up to 2 s for the L1) and extra invalidation work: the delete event must reach every local copy, which is the V1 problem again, now deliberately confined to a few keys.

Detection comes first. Per-key request counters with sampling, tell you which keys are hot; guessing is how you cache the wrong thing. Promotion should be automatic, expire quickly, and stay limited to a handful of keys, so the local cache never grows into V1 again. Each pod should also coalesce its own refresh of an L1 entry through single-flight, otherwise 20 pods all refetching the moment the 2 s TTL lapses simply recreate a small stampede on the owning shard.

Sharding makes a node failure partial rather than total, but the shard on the failed node is still gone, and its keys still miss. A replica is a second copy of a shard kept on another node, so that if the primary dies the replica can be promoted to take over. That switch is called failover.

Most caches replicate asynchronously: the primary applies a write, replies to the client, and ships the change to the replica afterwards without waiting. Replication is cheap on the write path, and it has a precise failure. The primary acknowledges SET product:4821 $109 v6; the replica, 120 ms behind in our hypothetical, has not yet applied it; the primary dies; the replica is promoted and holds $99 (v5). The write the client was told succeeded has vanished.

Redis's own cluster documentation says as much. It tries to retain writes made by clients connected to the majority side, but small windows exist in which acknowledged writes can be lost, and they are larger on the minority side of a partition. Its WAIT command can block until a number of replicas have acknowledged, and that reduces the probability of loss considerably, but it does not turn the system into a strongly consistent one. Stating that Redis Cluster never loses acknowledged writes would be wrong.

For a cache this is usually tolerable, and the reason is the whole philosophy of the page: the lost write is a *copy*. The authority has v6; the invalidation event or the TTL will correct the cache; and checkout revalidates. A failed-over cache can briefly serve v5, a bounded stale read, rather than corrupt anything. The same loss is a business error if the cache is the only place the data lives. That is the danger of write-behind designs and of any use of a cache as a store of record: durability must be bought where the data is authoritative, not assumed from a cache.

Operationally, failover has a detection delay (nodes must agree the primary is unreachable), a promotion step, and a client re-routing step. During it, requests to that shard fail or time out. Clients need short deadlines and a rule for what to do, which is the subject of the operations chapter. Two further failure shapes belong here. First, a deletion can be lost just as a write can: if the primary acknowledged an invalidation delete that had not reached the replica, the promoted replica still holds the old key, so a stale entry can reappear after failover; the TTL and the version check bound it, which is another reason to keep both. Second, a partition can leave the old primary reachable by some clients while a replica is promoted on the other side. Redis Cluster's documented answer is that a primary cut off from the majority for longer than the node timeout stops accepting writes, and the cluster resolves rival claims with a configuration epoch (a counter where the highest wins), so writes accepted on the losing side are discarded when the partition heals. A custom design needs the same two ideas, a time limit on a primary acting alone and an epoch or fencing number on every promotion. When the old primary returns it rejoins as a replica and resynchronises from the new one; the cache refills the keys it lost from the database as readers miss them. Atomicity also needs care: an increment or compare-and-set that times out is ambiguous (it may or may not have been applied), so non-idempotent retries need a token or have to tolerate duplication.

Peak traffic grows and the dataset outgrows four nodes, so Meridian adds a fourth primary, D. The goal is to do it while serving traffic and without sending a large share of reads to the database.

With slots the plan is explicit. Illustratively, A gives slots 4096–5460 (1,365 slots), B gives 9558–10922 (1,365) and C gives 12288–13653 (1,366) to D: 4,096 slots in all, 25% of the table. Real planners choose their own ranges. Product 4821, in slot 13107, falls inside C's donated range and moves from C to D; tenant:cedar:product:4821, in slot 8298, stays on B. Only the keys in those slots move, and everything else keeps hitting where it always did.

The tricky time is the middle of a slot's migration, when some of its keys are on the old node and some are on the new. In Redis Cluster the source node, in a *migrating* state, answers for keys it still holds. For a key it does not hold (already handed over, or simply absent) it replies -ASK <slot> <target>, and the client sends an ASKING command followed by the original command to the target, because a node accepts queries for an importing slot only after ASKING. -ASK is a one-time detour: the client does not update its slot map. When the slot has fully moved, any node that no longer owns it answers -MOVED <slot> <host:port>, which is the permanent signal: the client should update its slot map. A client that handles neither will fail during a rebalance, which is why client libraries matter as much as the server. Clients may also send a command to any node and follow the redirect.

For a hand-built ring the same safety is on you. Version the placement map so every client knows which topology it is using. During migration, read from the new owner first and fall back to the old one (a *dual read*), or copy keys ahead of the cutover. Warm the new node before routing real traffic to it, and keep the ability to roll back the map. A cold new shard that suddenly owns 25% of traffic is a mini-outage.

A shared cache is a shared room. Without discipline one merchant can read another's data, crowd it out of memory, or fill the cache with junk.

The first measure is namespacing: keys follow the schema v2:<tenant>:<type>:<id>, as in v2:cedar:product:4821, with the schema version first so that incompatible serialisers or application versions cannot read each other's values after a deploy. Naming alone is not security. The authority must establish who the caller is, the application must build the key from that verified identity rather than from a client-supplied string, and the cache itself should require authentication and access control lists (ACLs) limiting which application roles can touch which key prefixes.

The second concern is capacity fairness. If Cedar sends a burst of unique keys, eviction picks victims among *all* entries, so Birch's hot items get pushed out and Birch's hit rate falls for something it did not do. Per-tenant memory quotas, per-tenant request limits at the API layer (see rate limiting), and for large or sensitive tenants a dedicated shard tier, all limit that blast radius. Dedicated tiers cost more memory and operations but give hard isolation.

Third, what must *not* go in the cache. Secrets and unredacted payment data stay out of a shared cache entirely. Personal data, if cached at all, needs a short TTL, encryption in transit and an owner who can purge it on request. Negative entries are capped per tenant, and 'forbidden' is never stored as 'not found' for other identities. Finally, authorization answers are not served from a TTL-only copy: if a merchant revokes an employee, the next request must see the revocation, either by checking the authority or by comparing against a versioned revocation epoch.

Serving shoppers from two regions means putting a cache close to each. It also means a price change in one place must reach a cache on the other side of an ocean.

Each region, EU and US, gets its own cache cluster. Shoppers hit the nearest one and enjoy the 3 ms hit path rather than a transatlantic round trip. A write goes to the authoritative database, and invalidation events are replicated to the other region's stream. That replication has delay, can fail, and can arrive after a regional fill that read the old value. So the same product can briefly be $109 in the EU and $99 in the US, and *every statement about cross-region freshness must be a bound, never an 'instantly consistent'*.

Three design choices follow. Keep the TTL backstop in each region so a lost cross-region event ends in bounded staleness. Prefer *invalidating* over trying to push the new value across regions, since an invalidation is idempotent and a refill will fetch the right data locally. And be clear that a globally linearizable cache, one that never returns a stale value anywhere, is a different and much more expensive system: it needs synchronous coordination across regions on every write, so it costs write latency and availability during partitions. Data residency may also forbid copying certain tenants' data to the other region, in which case their keys are region-pinned and not replicated at all. See multi-region architecture for the wider context.

The cache has to be observed and survivable. The scariest scenario is also the most instructive: the whole cache cluster becomes unavailable at peak.

At peak every one of the 48,000 eligible requests per second now goes to the backends. For a one-minute outage that is 2,880,000 requests. Meridian's planned database ceiling is 19,200 per second, of which 12,000 is already non-eligible traffic, leaving 7,200 per second of admission for fallback. Against 48,000 offered, that is 6.7 times oversubscribed, and 40,800 per second, 85.0% of the eligible requests, must be shed, served stale from a class that allows it, or degraded. Without such a rule, the database falls over and the failure cascades into the checkout path.

The fallback policy therefore lives in the application, class by class. C1 content can be served stale-if-error from a local copy. C3 badges can be dropped from the page. C2 prices can show a last-known value flagged 'confirm at checkout'. C4 data fails closed. A circuit breaker around the cache stops pods from burning their time on a dead dependency, a bulkhead limits concurrent fallback queries so the database sees a bounded stream, and load shedding returns an error promptly for the excess. See circuit breakers, bulkheads and load shedding. Retries deserve special care: unbounded retries multiply the load exactly when capacity is gone (retry storms).

A *cold* cache after a restart is the same problem in slow motion: hit rate starts near zero and climbs, so the database sees the miss-heavy workload while the cache refills. Warm important keys from a list before routing traffic, and ramp the traffic. The same care applies to deployments: restart cache nodes one at a time, let a replica take over deliberately rather than by timeout, wait for resynchronisation before touching the next node, and roll API pods in small batches so their empty L1 maps do not all miss at once. Under overload, protect the database first: lower the fallback admission limit rather than letting retries decide who gets through.

What to watch

Put the pieces together and the path of one request is short to describe, even though each stage hides a mechanism.

A shopper's request reaches a Product API pod. For an eligible read the pod checks its small L1 (hot keys only, TTL about 2 s), then asks the cache shard that owns the key, found by slot or ring lookup. A hit returns in about 3 ms. A miss is collapsed by single-flight or a lease, read from the authoritative database, and filled with a version-aware conditional write and a jittered TTL. Writes go to the database; a transactional outbox emits an invalidation to a stream; subscribers in each region delete or refresh the key; the TTL ensures the worst case is bounded even if an event is lost. Replicas keep a copy of each shard; the cache cluster has quotas per tenant; and the fallback policy bounds what the database must absorb when something fails. Checkout does not trust the cache for price or stock.

The guarantees, stated honestly

Decisions to carry away