Cassandra Fundamentals

Data is organized into tables with rows and columns like a relational database, but each row can have a different set of columns — closer to a distributed, sorted map of maps than a fixed relational schema.

Every node runs the same code and can serve any request — there's no primary/replica split to manage or fail over.

Every read and write specifies a consistency level (ONE, QUORUM, ALL, ...) — the trade-off between latency and consistency is a per-query decision, not a global one.

Adding nodes adds both storage and throughput roughly proportionally, because responsibility for the keyspace is spread evenly across all nodes.

The part of the primary key that determines which node(s) own a row. Cassandra hashes the partition key to compute a token, and the token determines placement on the ring.

Additional key columns that determine the sort order of rows within a single partition — they don't affect placement, only ordering on disk.

A per-keyspace setting: RF=3 means every row is stored on 3 different nodes. Common production values are RF=3 (single datacenter) or RF=3-per-datacenter for multi-DC deployments.

Whichever node receives the client's request. It doesn't need to own the data itself — it just needs to know (via the ring topology) which nodes do, and forwards the request to them.

Only a single replica needs to acknowledge. Fastest and most available, but offers the weakest durability — if that one replica is lost before it syncs to others, the write is gone.

A majority of replicas (⌊RF/2⌋ + 1) must acknowledge. A common production choice (not a server-wide default; cqlsh starts at ONE and driver defaults vary) — combined with the R + W > RF overlap condition (covered below), it gives a strong read-your-writes guarantee for the common case. It is not automatically the same as linearizable consistency: clock skew, hinted handoff, and concurrent writes can still produce surprising orderings unless the workload needs the stronger, slower guarantees of lightweight transactions (LWT / SERIAL).

Every single replica must acknowledge. The strongest consistency, but the weakest availability — a single down replica blocks every request, which defeats much of Cassandra's purpose.

Once per second, every node exchanges state (heartbeats, load, schema version, which nodes are up/down) with 1-3 random peers. Within a few rounds this information has propagated to the entire cluster — no broadcast, no central registry required.

If a replica is temporarily unreachable during a write, the coordinator stores a 'hint' — the missed write — and replays it once the replica comes back, avoiding the need to block the write or run a full repair for short outages.

The replica appends the mutation to the commit log (a sequential, append-only file used only for crash recovery) and applies it to the table's in-memory, sorted memtable. Only then does it acknowledge the coordinator. No read-before-write and no random disk I/O is involved, which is why writes are cheap. With the default periodic commit-log sync (commitlog_sync_period, 10 s by default), a replica may acknowledge a write before it has fsynced its commit log, so a sudden crash can lose that replica's unsynced tail. Replication and the chosen consistency level determine whether another replica still holds the write; they reduce the risk rather than guarantee survival (batch mode fsyncs before acknowledging, at a latency cost).

When memtable memory crosses its threshold, or the commit log reaches its size cap and has to free segments, the memtable is flushed to disk as an SSTable — an immutable, sorted file with its own partition index and Bloom filter. Commit-log segments are discarded once every memtable they cover has been flushed. After a crash, the surviving log is replayed into a fresh memtable.

Because SSTables are never rewritten in place, one partition is usually spread over the memtable and several SSTables. A read consults the memtable plus each SSTable that might hold the partition (Bloom filters rule most out) and merges the results by timestamp — last write wins per cell. The more SSTables a read has to touch, the slower it gets.

An update just writes a newer-timestamped cell; a delete writes a tombstone marker. Nothing is physically removed at write time, so old versions and tombstones sit in older SSTables until compaction rewrites them.

A tombstone must outlive the window in which some replica might still hold the deleted data, otherwise a replica that missed the delete could resurrect the row. gc_grace_seconds (default 864000 s = 10 days) is that window: compaction will not purge a tombstone before it elapses, and repair must complete on every replica within it. Tombstones are also scanned on reads — by default Cassandra logs a warning at 1,000 tombstones in a single query and aborts the query at 100,000 (tombstone_warn_threshold / tombstone_failure_threshold; defaults as of Cassandra 4.1/5.0).

During a normal read at CL=QUORUM or higher, the coordinator compares timestamps across the replicas it happened to query. If one is stale, the newer value is written back in the background. This is an opportunistic, read-path mechanism — it only touches the replicas involved in that particular read, and rarely-read data can stay stale indefinitely without it.

A background process (nodetool repair) that compares full replica datasets using Merkle trees and synchronizes any differences across all replicas, not just the ones a recent read happened to touch. This — run on a schedule, typically within gc_grace_seconds — is the mechanism that actually guarantees comprehensive reconciliation; read repair alone is a nice-to-have, not a substitute for it.

During a network partition, Cassandra's default configuration keeps accepting reads and writes on both sides (Availability) rather than refusing requests to guarantee Consistency — the classic AP lean, as opposed to a single-master relational database, which typically chooses CP. But this is a tendency, not a fixed law: Cassandra is more accurately described as partition-tolerant and highly available by default, with tunable consistency — pushing reads and writes to QUORUM/ALL, or using lightweight transactions (LWT), moves specific queries much closer to CP behavior.