Trade-offs & Decisions

Realtime chat is bidirectional, but not every product with updates needs the full operational cost of WebSocket.

You can combine connection handling and message logic at small scale, but the two workloads scale differently.

A combined service is simpler early: one deployment, fewer hops, fewer moving parts. It is completely reasonable for modest concurrency.

A separate gateway tier becomes attractive when persistent connection count, TLS termination, draining behavior, and connection churn need independent tuning from message validation, authorization, and storage throughput.

Push-first feels lower latency, but it makes ACK meaning ambiguous and weakens reliability.

Durable-first says Alice gets a positive ACK only after the write contract succeeds. Delivery to Bob may still happen immediately afterward, but the sender-facing promise is crisp.

Push-first can look fast in demos, but a crash between transient push and durable write turns a UI success into silent loss. Reliable messaging products almost always choose durable-first for accepted messages.

Retries happen before the client knows any server-assigned identifier, which is why many messaging systems use both IDs.

Client-generated request IDs make retries safe because the first and second attempt still name the same logical send. Server-generated message IDs remain useful as canonical durable identities for storage, sync, and downstream references.

The more you want strict contiguous order, the more serialization or ownership correctness you usually need.

Strict sequence numbers make gaps explicit, history reads simple, and conversation order deterministic. Time-sortable IDs reduce coordination and may be sufficient where approximate order is acceptable.

The right choice depends on the product promise. A consumer chat app often benefits from strict per-conversation ordering; a looser social inbox may accept approximate timestamp order.

Conversation logs minimize duplication; inboxes can simplify user-centric fetches. Hybrids are common because messaging needs both perspectives.

Conversation logs store once and naturally preserve order. Per-user or per-device inboxes accelerate unread and delivery bookkeeping but amplify group writes. A hybrid log-plus-sync-index model often balances cost and ergonomics.

Read deeper: Conversation Log vs Inbox.

Scanning every conversation can work for tiny products, but reconnect at scale usually wants a durable cursor or change index.

A sync cursor is efficient when one device may have hundreds of conversations and only a few changed. Per-conversation polling stays useful for local history reads and targeted gap repair.

Group architecture depends on size, activity, online ratio, receipts, and storage budget; no single threshold is universally right.

Write fan-out helps readers and notifications but amplifies writes. Read fan-out keeps writes cheap but shifts cost to reads and caching. Hybrids let the product use one strategy for small interactive groups and another for huge audiences.

Read deeper: Group Message Fan-Out.

Worked example (illustrative assumptions)

Illustrative assumptions, not measurements: write fan-out costs one delivery record per member per message; a fan-out worker pool sustains 50,000 such writes per second; at any moment 50% of a 200-member friends group is online, but only 5% of a 1M-member channel is actively reading.

The crossover lies between these two audience sizes, and where exactly depends on how much per-message write budget you are willing to spend, how many members are typically online, and what receipts the product promises. That is why the page recommends a tunable policy rather than one hard-coded member count.

Inline fan-out can be simpler for small groups; asynchronous workers protect sender latency as group cost grows.

Sync fan-out gives immediate completion semantics for a small recipient set. Async fan-out decouples sender ACK from thousands of downstream deliveries, but then operational lag in the worker path becomes user-visible.

Presence can be exact-ish and expensive, or approximate and cheap; messaging products usually choose approximate within a bounded staleness window.

Aggressive heartbeats and broad subscriptions make status fresher but increase battery, fan-out, and infrastructure cost. TTL-based leases plus selective subscriptions make presence cheaper and still adequate for user expectations.

Per-message, per-device receipts are precise but can explode in cardinality. Aggregation trades detail for scale.

A product can track delivery per device internally while exposing a simpler per-user aggregate. Read receipts may use high-watermarks rather than one event per message. Very large groups may suppress detail further.

There is no universal rule that messaging storage must be Cassandra. Workload size and required semantics decide.

At smaller scale, relational storage can be entirely reasonable and may simplify transactional logic. At very high append rates, wide-column or distributed KV designs often fit better because partition-local ordered appends are the dominant pattern.

Partitioning by conversation alone is simple, but long-lived or hot conversations often force a bucketing strategy.

Conversation-only partitions minimize read complexity until growth or hotspot behavior becomes unacceptable. Conversation plus time or sequence buckets bound physical size and repair cost but add pagination logic.

A durable log can be the message path, or it can be the side path. The trade-off is between explicit log semantics and lower end-to-end dependency count on every send.

Putting Kafka on the critical path can simplify ordered append and replay semantics, but it also means Kafka availability directly gates accepted sends. Keeping it on the async side path preserves a shorter synchronous chain but requires a separate durable handoff for downstream async work.

Active-active writes reduce local latency everywhere, but conversation ordering and partition behavior become much harder.

Single-region or home-region write authority keeps ordering simple and split-brain risk lower. Active-active can improve local write latency for geographically distributed participants, but only if you also define strong ownership transfer or conflict semantics.

E2EE improves privacy and changes what the platform can inspect, search, moderate, or back up.

Server-readable content enables richer server-side search, indexing, moderation, and analytics on content itself. E2EE constrains those capabilities and pushes more intelligence and key state to clients, especially for multi-device and group history behavior.