Architecture Deep Dive

The gateway owns connection lifecycle, not message correctness. It authenticates a device session, enforces socket-level backpressure, and gives the rest of the system a stable way to talk to a currently connected client.

On connect, the gateway verifies the client token, establishes a device or session identity, allocates a server-side connection_id, and registers liveness metadata. When Alice reconnects after losing her ACK, she does not resume an existing TCP connection; she creates a new session that later recovers state through sync.

Heartbeats refresh a lease. Missing heartbeats eventually mark the connection stale, but the lease window is deliberately wider than one missed ping so transient mobile-network jitter does not cause flapping. Gateways also support graceful draining: stop admitting new sockets, tell clients to reconnect with jitter, and let healthy gateways absorb them gradually.

A user is not one connection. The routing model is user_id -> {device/session -> gateway, connection_id, epoch} so delivery can target several active devices and safely ignore stale entries.

The registry is ephemeral routing state refreshed by connect, heartbeat, disconnect, and session replacement. When Bob opens his laptop and phone, the messaging service may get two live routes. If the phone is offline, the registry simply has no live phone route; that does not change durable message state.

Use a gateway epoch or session version so the newest connection wins when an old socket lingers. If a route points to a dead socket, the target gateway rejects it, the sender-side service re-looks up routing, and if no live session remains, the system falls back to offline sync and optional push hints.

A retry is expected behavior, not an exception. Alice creates client_message_id = cmid-123 before the first send attempt so every later retry still refers to the same logical message.

The server can then map (sender_id, sender_device_id, client_message_id) to one canonical server message_id, one acceptance record, and one sequence assignment. If the first attempt persists successfully but the ACK is lost, the second attempt should reconstruct the original result instead of creating a second bubble in Bob's chat.

The key requirement is atomicity: do not do a racy read-before-write existence check. Use a conditional insert, uniqueness constraint, transactional record, or equivalent mechanism so two concurrent retries cannot both "see nothing" and both insert.

The message lifecycle is not one mutable row that everything updates in lockstep. It is a sequence of events that advance user-visible state monotonically.

Alice's local client starts at CLIENT_PENDING. After the server durably accepts M, Alice may show a single checkmark. That does not imply Bob has received anything yet. DELIVERY_PENDING covers the period where Bob might be online, offline, partially connected, or reachable only on one device.

This event-oriented state model lets the system survive partial failure. A recipient delivery event can be duplicated, delayed, or batched without violating correctness as long as updates remain idempotent and monotonic.

Users need ordering inside a conversation, not across the whole product. The hard part is deciding which component has write authority for that conversation when senders race or owners fail.

Several implementations are reasonable: an ordered log partition keyed by conversation, a conversation owner or sequencer with a lease and fencing token, or a storage-side conditional sequence allocator. What they share is serialization of accepted writes for one conversation. Consistent hashing can route traffic toward one owner, but it does not by itself prove that only one owner is active during rebalance or failover.

Approximate time-sortable IDs reduce coordination but do not guarantee strict contiguous order under concurrency. If the product promise is strict per-conversation sequencing, some ordering authority must assign it before messages are treated as accepted.

Bob's laptop can legitimately receive sequence 420 before 419 because the transport and fan-out layers retry independently. The client must detect the gap without assuming the backend is broken.

The client keeps the highest contiguous sequence it has safely applied. If it receives 420 while waiting for 419, it can buffer 420 within a bounded window, request the missing range, and continue once 419 arrives. If the gap is large or the repair times out, it escalates to a conversation sync or a device sync cursor replay.

Do not hard-code a universal wait like “sleep 500 ms.” The right bound depends on network conditions, device memory, and product tolerance for temporarily withholding a newer message until the missing one is repaired.

Message storage needs fast append, ordered history reads, and bounded physical partitions. A logical conversation should not become one forever-growing physical shard.

A practical schema is (conversation_id, bucket) as the physical partition key and sequence or time within the bucket as the clustering key. Buckets can be time-based, sequence-range based, or size-based. This preserves ordered scans while preventing one long-lived or high-volume conversation from creating an unbounded partition.

Reads page naturally: start at the newest bucket and sequence, then walk backward as the user scrolls. For Alice and Bob's direct conversation, the bucket abstraction is invisible to the product, but it protects compaction, repair, and hotspot behavior underneath.

The source of truth question matters: do you store one canonical conversation log, one per-user inbox, or both?

For this module, the clean baseline is hybrid: one durable conversation log plus a sync-oriented change stream or cursor index. That keeps Bob's laptop and phone convergent without turning every message into a fully duplicated inbox write.

Offline recovery should start from a durable cursor, not from scanning every conversation the user ever joined.

When Bob's phone reconnects hours later, it presents its last durable device cursor. The server replays missed message, membership, and receipt events after that cursor, deduplicates naturally by stable IDs, and advances the cursor only after the device safely applies the batch.

Conversation range queries are still useful when Bob opens one specific thread or repairs a localized gap, but reconnecting a device with hundreds of conversations should not require N independent scans just to discover which threads changed.

“Delivered” is a product semantic, not a transport fact. You have to define what event actually turns Alice's UI from one checkmark to two.

A strong delivery event is device-level acknowledgement after the recipient app has received, validated, and safely enqueued or persisted the message locally. Merely handing bytes to Bob's gateway is weaker; the gateway still might disconnect before Bob's app processes them.

This module uses the explicit product rule: user-visible delivered means at least one active recipient device acknowledged receipt. Internally, the system may still track per-device delivery state for later sync and convergence.

Read state is usually compressible. Bob does not need to emit one receipt event for every historical message when he opens the chat on his laptop.

A conversation-level high-watermark like read_up_to_sequence = 418 is often enough. It says Bob has read every message up to 418 on behalf of the chosen product semantic. This is far cheaper than sending hundreds of individual read events and is naturally monotonic.

Large groups may suppress, aggregate, or omit individual read receipts entirely because the state and fan-out explode. The storage model should reflect that by avoiding one mutable per-message status column that pretends all conversations have the same receipt shape.

Group messaging is not just fan-out. The system must also decide who counts as a member at the instant a message is accepted.

Keep a durable membership source of truth plus a membership version or snapshot concept. On send, authorize the sender against current membership and bind the accepted message to the membership snapshot or version used for fan-out. That gives a deterministic answer when a remove and a send race each other.

If Carol is removed at version 91 while Alice's send is accepted against version 90, the product can intentionally decide whether Carol should still get that message. The important part is that the decision is explicit and replayable, not left to timing luck across multiple workers.

Sender ACK should depend on durable acceptance, not on waiting for a huge group fan-out to finish.

Small or moderately active groups may use mostly synchronous routing because the member set is small enough to resolve quickly. Larger or hotter groups often switch to async fan-out workers so Alice still gets a prompt ACK while background workers handle recipient-by-recipient delivery, retries, and offline fallback.

The threshold is workload-dependent. Group size, online fraction, activity level, receipt semantics, and gateway pressure all matter, which is why hard-coded “256 means write fan-out forever” rules age badly.

A broadcast channel is not just “a really big group.” It behaves more like a write-once feed with audience fan-out and caching concerns.

Broadcast or channel products typically store once, maintain an audience membership index, and let recipients fetch or be notified rather than pretending every subscriber needs per-message synchronous delivery state. Few publishers write; a huge audience reads.

That changes several expectations: per-recipient read receipts are usually omitted, ordering is easier because there are few senders, and read-time caching matters more than per-subscriber realtime certainty.

Presence is ephemeral lease state derived from active sessions. It should be approximate, privacy-aware, and selectively fanned out.

A practical rule is: user is online if any allowed session currently holds a live lease; otherwise show offline or last seen according to privacy settings. Heartbeats refresh TTLs, but the service should fan updates only to explicitly interested viewers such as users who are actively looking at that conversation.

That means presence failure should not block message delivery. Alice can still send M even if the product temporarily cannot say whether Bob is “online now.”

Typing is the clearest example of lossy ephemeral messaging: useful when fresh, worthless when late, and never worth durable storage.

Typing start and stop events should be rate-limited, coalesced, and TTL-based. If Bob closes his laptop or the packet is lost, the indicator simply expires. There is no replay backlog of “Bob was typing 20 minutes ago.”

This is exactly the kind of traffic the platform can drop during overload long before it considers degrading accepted user messages.

Push is a wake-up path for clients that lack a usable realtime route. It is not where message correctness lives.

The safe order is: durably accept the message, fail to find a usable live route for Bob's phone, enqueue a push hint, then let the phone reconnect and sync authoritative state. A delayed or duplicated push may change wake-up latency, but not whether M exists.

When end-to-end encryption is enabled, the push payload must also respect metadata and plaintext leakage constraints. Usually that means a minimal hint instead of the full message body.

Media should share conversation semantics but not the byte path. Upload and download are separate from the realtime messaging lane.

The client uploads bytes directly or resumably to object storage through an authorized media service, then sends a tiny message containing a media reference and metadata. Recipients fetch through an authorized object or CDN path. That keeps the text and control-message pipeline small and predictable.

With E2EE, media may be encrypted client-side before upload, and the message reference carries the information recipients need to fetch and decrypt it. The storage layer still sees bytes and metadata, but not cleartext content.

End-to-end encryption changes what the server can inspect, store, and derive. It does not remove the need for routing, device management, or metadata handling.

At architecture level, the server stores and routes ciphertext plus metadata, while key material and cryptographic session state live primarily on clients. Multi-device support gets harder because each linked device often has its own cryptographic identity or session state, and adding or removing a device changes who can decrypt future content.

Group messaging gets harder too: membership changes interact with key distribution, message history, and who is allowed to decrypt. Server-side content search, indexing, moderation, and backup options become constrained, which is why this section matters even without turning into a protocol tutorial.

Streaming infrastructure is useful when it cleanly separates synchronous acceptance from asynchronous side work. It becomes dangerous when the boundary is fuzzy.

A messaging event stream can power async group fan-out, push notification triggering, receipt propagation, analytics, and abuse workflows. But a design should state clearly whether those features degrade when the stream is unavailable. If large-group fan-out depends on Kafka, large-group delivery is degraded during Kafka outage.

For events that must survive publication failure, use a durable handoff such as a transactional outbox or store-backed retry record. In-memory producer buffers are for smoothing short hiccups, not for durable outage recovery.

A messaging system must be willing to disconnect a slow client. Otherwise one stalled socket quietly turns into an infrastructure incident.

Gateways need bounded per-connection queues, write deadlines, and pressure signals back toward admission control. If Bob's laptop stops reading, the gateway should eventually drop that socket and rely on offline sync after reconnect rather than retaining arbitrary pending bytes in memory.

Backpressure also applies upstream. During overload, the system should protect durable accepted messages first, then defer or drop ephemeral work such as typing, some presence updates, and maybe batched analytics.

Edits, deletes, and reactions are useful extensions because they reveal how the system handles derived events that reference an existing stable message ID.

Treat each mutation as its own ordered event against a target message_id. Edits may carry a new version, delete-for-self and delete-for-everyone need distinct product semantics, and reactions can often be aggregated because they are high-frequency overlays rather than new core content.

There are also hard limits: if Bob saw the original plaintext on one device already, delete-for-everyone cannot mean “nobody ever saw it.” E2EE and retention policies further constrain what the server can enforce retroactively.

The clean baseline is conversation home-region write authority with cross-region delivery. Start there before discussing active-active writes.

In that model, each conversation has one write authority region or owner at a time. Users may connect to nearby gateways globally, but accepted writes for that conversation serialize through its current authority, preserving the ordering contract. Other regions can cache, deliver, and replicate, but they do not independently assign competing sequence numbers for the same conversation.

Active-active multi-region writes are possible only if you also define conflict or global-order semantics, lease ownership, or stronger consensus around who is allowed to assign order. During partition, two regions believing they own the same conversation is the classic split-brain corruption case.

What to remember across all 23 sections