Stock Exchange Matching Engine Architecture — Deep Dive

The matching engine for one instrument is a single-writer event loop: one thread, one book, one command at a time. This sounds like a limitation, but it is the single design decision that makes every correctness guarantee in this page possible.

This page assumes familiarity with the problem framing, functional requirements, and evolution from a single process to a production architecture covered on Complete Design — this page goes deeper into the internals of the components that design introduces, rather than re-explaining why each one exists.

Why a matching engine is single-writer per book

Matching ACME orders requires reading and mutating shared state — the bid ladder, the ask ladder, the FIFO queue at each price level — on every single command. If two threads could apply commands to the same ACME book concurrently, the engine would need locks around every read and mutation, and even with perfect locking, the order in which two colliding commands were applied would depend on OS scheduling, not on the exchange's stated priority rules. That violates the most basic invariant a matching engine owes its members: that the sequence used for matching is deterministic and reproducible, not a race.

The practical fix used across the industry is to assign exactly one thread (or one single-threaded process) to own the ACME book, and exactly one (possibly different) thread to own NOVA's book. Within one book, every mutation happens on that one thread, in the order the sequencer handed it to the engine. Across books, ACME and NOVA can be matched fully independently and in parallel — there is no cross-instrument lock, because there is no cross-instrument shared state to protect.

The five-stage pipeline: intake, validate, sequence, apply, emit

A command's path through the system has five distinct stages, and conflating any two of them is a common source of subtle bugs. Intake: the gateway receives a new/cancel/replace message over a session, assigns it a durable command ID, and forwards it toward validation — this happens outside the matching engine entirely. Validation: pre-trade risk and basic syntax/session-state checks run before the command is allowed to consume a sequence number; a rejected order never touches the book and never consumes a sequence slot in the per-book stream. Sequencing: an accepted command is appended to the per-partition ordered command log, receiving a monotonically increasing sequence number — this is the moment the command's position in history becomes fixed. Apply: the single-writer engine thread for that instrument reads the next sequenced command and mutates the in-memory book — inserting a resting order, crossing against the opposite side, or removing a cancelled order. Emit: the engine produces execution reports and trade events as a direct consequence of the apply step, which are then journaled and fanned out asynchronously.

Why threads are still useful even in a single-writer design

Single-writer-per-book does not mean single-threaded-per-process. A production matching engine host typically runs many threads: one (or a small, pinned set) per owned instrument's book-mutation loop, plus separate threads for session I/O, pre-trade risk evaluation, journal writes, and market-data serialization. The rule that must hold is narrower than "use one thread": no two threads may ever concurrently read-modify-write the same instrument's book. Networking threads hand sequenced commands to the book-owning thread through a lock-free or mutex-protected queue; journaling and market-data threads consume the *output* of the apply step (execution reports, trade events, book deltas) without ever reaching back into the live book structure. This separation is what lets the system use normal multi-core hardware productively while keeping the one shared mutable structure — the book — single-writer.

Determinism and replay as a design constraint, not an afterthought

Because the apply step is a pure function of (current book state, next sequenced command), replaying the same sequenced command stream from the same starting state must always produce the same book and the same trade IDs — this is the property Component 09 (Journal, Commit and Replay) depends on entirely. That constraint shapes engine implementation choices that look unrelated to matching logic at first glance: no reliance on hash-map iteration order (which can vary across runs or language versions) when choosing between equally-priced orders, no reliance on floating-point arithmetic (which can round differently across CPUs/compilers) for price or quantity comparisons, and no reliance on system clock reads inside the apply step itself. Every one of these would make "replay produces the identical result" a probabilistic claim instead of a guaranteed one.

An order book for one instrument is really three cooperating structures: a sorted map of price levels, a FIFO queue of resting orders inside each level, and a hash map from order ID straight to a queue node — so that cancellation never has to scan the book.

The ACME book, concretely

Recall the canonical ACME book from Flow Exchange. Tick size is $0.01 and lot size is 1 share, so every price is stored as an integer number of ticks ($101.00 is the integer 10100) and every quantity is stored as an integer number of shares — never as a float, because floating-point rounding is not an acceptable source of error in a system that owes participants exact fills.

Sorted price levels: ordered map or tick-indexed array

The book needs "what is the best ask right now" and "what is the next price level above this one" answered constantly during matching, so price levels are kept in a structure sorted by price: a balanced binary search tree or ordered map (red-black tree, skip list) keyed by integer tick price, or — when the instrument's tradable price range is bounded and known in advance — a flat array indexed directly by tick offset from a reference price. The ask side is ordered ascending (best ask is the lowest price); the bid side is ordered descending (best bid is the highest price). Each node in this structure holds a FIFO queue of the orders resting at that exact price.

FIFO queues within a level, and the cancel-lookup hash map

Within one price level, orders must execute in strict arrival order — this is the "time" half of price-time priority. A doubly linked list per level gives O(1) append-at-tail (new resting order), O(1) pop-from-head (next order to fill), and O(1) removal of an arbitrary node once you already have a pointer to it (a cancellation). That last property is why a second structure is required: a hash map from exchange order ID directly to its linked-list node. Without it, cancelling order S2 would require scanning every price level's queue looking for a matching ID — unacceptable when the book can hold tens of thousands of resting orders.

function cancelOrder(bookState, orderId):
    node = bookState.orderIndex.get(orderId)      // O(1) hash lookup
    if node is null or node.status != RESTING:
        return CancelResult.UNKNOWN_OR_ALREADY_TERMINAL

    level = node.priceLevel
    level.queue.remove(node)                       // O(1): node already holds prev/next pointers
    bookState.orderIndex.delete(orderId)

    if level.queue.isEmpty():
        bookState.levels.remove(level.price)        // O(log P) removal from the sorted price-level map

    node.status = CANCELLED
    return CancelResult.CANCELLED

Memory layout and allocation overhead

At sustained order rates, naively allocating a new heap object for every incoming order and freeing it on every fill or cancel creates allocator and garbage-collection pressure that shows up directly as matching-engine jitter — the opposite of the tail-latency goal covered in Component 14. Production engines commonly use object pools: a pre-allocated array of order-node structures that are recycled (returned to a free list) instead of deallocated when an order is cancelled or fully filled. This also improves cache locality — nodes that are frequently accessed together (the live orders at the current best price levels) can be kept densely packed rather than scattered across a heap by allocation order. These are implementation-level optimizations layered on top of the sorted-levels-plus-FIFO-plus-hash-map design, not a replacement for it.

Matching an incoming order means repeatedly comparing it to the best opposing price, filling the minimum of the two remaining quantities against the head of that level's FIFO queue, and repeating until the incoming order is exhausted, the book no longer crosses, or a time-in-force rule stops the loop early.

Execution-price policy

This design uses resting-order-price execution: every trade executes at the price of the order that was already on the book, never at the incoming order's limit price. An incoming limit buy's price is a ceiling on how much it is willing to pay, not a promise of what it will pay — if the best ask is cheaper than the incoming buy's limit, the trade executes at that cheaper resting price. This single policy choice is what Test B demonstrates below.

Matching pseudocode

function matchIncoming(book, incoming):            // incoming: side BUY/SELL, price (ticks), remaining qty
    oppositeSide = book.asks if incoming.side == BUY else book.bids

    while incoming.remaining > 0 and not oppositeSide.isEmpty():
        bestLevel = oppositeSide.bestLevel()

        crosses = (incoming.side == BUY  and incoming.price >= bestLevel.price) or \
                  (incoming.side == SELL and incoming.price <= bestLevel.price)
        if not crosses:
            break

        headOrder = bestLevel.queue.peekHead()      // oldest order at this price level
        fillQty = min(incoming.remaining, headOrder.remaining)

        emitTrade(buyOrderId = incoming.id if incoming.side==BUY else headOrder.id,
                  sellOrderId = headOrder.id if incoming.side==BUY else incoming.id,
                  price = bestLevel.price,           // resting order's price, never incoming's limit
                  quantity = fillQty)

        incoming.remaining -= fillQty
        headOrder.remaining -= fillQty

        assert incoming.remaining >= 0 and headOrder.remaining >= 0   // invariant: no negative remaining qty

        if headOrder.remaining == 0:
            bestLevel.queue.popHead()
            book.orderIndex.delete(headOrder.id)
            if bestLevel.queue.isEmpty():
                oppositeSide.removeLevel(bestLevel.price)

    if incoming.remaining > 0 and incoming.timeInForce == DAY:
        restOnBook(book, incoming)                   // insert as a new price level or append to existing level's tail

    return incoming

Test A, worked fill by fill

Incoming: limit BUY 60 @ $101.00. Starting book is the canonical ACME book above.

Fill 1: best ask is $101.00 (S1, S2 tied on price; S1 has lower sequence number 10, so it is at the head of that level's FIFO queue). Incoming price $101.00 >= level price $101.00, so it crosses. fillQty = min(60, 50) = 50. Trade: BUY (incoming) 50 shares vs SELL S1 50 shares @ $101.00. S1's remaining becomes 0, so S1 is popped and removed from the order index; incoming's remaining becomes 10.

Fill 2: the $101.00 level now has only S2 at its head (sequence 11). It still crosses ($101.00 >= $101.00). fillQty = min(10, 30) = 10. Trade: BUY (incoming) 10 shares vs SELL S2 10 shares @ $101.00. S2's remaining becomes 20 (stays resting, now at the head of the $101.00 level); incoming's remaining becomes 0.

Loop exits because incoming.remaining == 0. Final state: two trades printed at $101.00 for 50 and 10 shares; S2 rests with 20 remaining; the incoming buy is fully filled and never rests. This is price-time priority in action — S1 filled completely before S2 got any fill, purely because of arrival sequence (10 before 11), not price (they were tied).

Test B, reset first: sweeping three levels

Reset to the canonical book. Incoming: limit BUY 100 @ $101.50.

Fill 1: best ask $101.00 (S1 head, seq 10). Crosses ($101.50 >= $101.00). fillQty = min(100, 50) = 50 @ $101.00. S1 exhausted and removed; incoming remaining = 50.

Fill 2: $101.00 level now holds only S2. Crosses. fillQty = min(50, 30) = 30 @ $101.00. S2 exhausted; the $101.00 level is now empty and removed from the ask side entirely; incoming remaining = 20.

Fill 3: best ask is now $101.50 (S3, seq 12). Crosses ($101.50 >= $101.50). fillQty = min(20, 70) = 20 @ $101.50. S3's remaining becomes 50; incoming remaining = 0.

Loop exits. Three trades printed: 50 @ $101.00, 30 @ $101.00, 20 @ $101.50. S3 rests with 50 remaining. The incoming order's $101.50 limit was a ceiling, not a guaranteed price — two of its three fills executed at the strictly better price of $101.00.

Test C, reset: no execution, new best bid

Reset to the canonical book. Incoming: limit BUY 20 @ $100.75. Best ask is $101.00. incoming.price (100.75) >= bestLevel.price (101.00) is false, so the loop never executes its body — zero crosses. Since incoming.remaining is still 20 and time-in-force is DAY, it rests: a new price level at $100.75 is inserted into the bid side (sorted between B1 at $100.50 below it and the ask side above it), becoming the new best bid. The book is never crossed after this: best bid $100.75 < best ask $101.00.

A market order, an IOC order, and a FOK order all reuse the same cross-or-rest loop from Component 03 — what differs is what happens to any quantity that is not immediately fillable, and in FOK's case, a feasibility check that must run before a single share is touched.

Market orders: sweeping depth without a price guarantee

A market order has no limit price — it crosses against the best available opposing price regardless of level, continuing until it is fully filled or the opposite side of the book is exhausted. This is the matching loop from Component 03 with the crosses check replaced by an unconditional true as long as the opposite side has any resting quantity. A market order can therefore sweep through several price levels, each at a progressively worse price (worse for the aggressor), and the average execution price is not known in advance. If the opposite side empties out before the market order is filled, the unfilled remainder is handled according to the venue's policy for that order type — Flow Exchange cancels any unfilled remainder of a market order rather than resting it at an undefined price, since a market order by definition carries no limit to rest at.

IOC: fill what you can, cancel the residual

Immediate-Or-Cancel runs the identical cross-or-rest loop with one change at the end: if incoming.remaining > 0 after the loop exits, the residual is never rested — it is immediately cancelled, regardless of time-in-force defaults that would otherwise apply. Test E demonstrates this from the canonical book: IOC BUY 100 @ $101.00. The loop fills 50 @ $101.00 against S1 and 30 @ $101.00 against S2 — both price levels at $101.00 are now exhausted and removed, since together they only had 80 shares — leaving incoming.remaining = 20. Because this is IOC, that remaining 20 is cancelled immediately rather than resting as a new bid. Net effect: 80 shares filled across two trades, 20 shares cancelled, nothing added to the book.

FOK: feasibility pre-check with zero partial side effects

Fill-Or-Kill is the one order type that cannot reuse the matching loop directly, because the loop mutates the book incrementally as it fills — and FOK's defining property is that either the entire order fills immediately, or none of it does; there is no partially-filled-then-reversed state. The only safe way to guarantee that is to compute, before mutating anything, whether the full requested quantity is actually fillable within the order's limit price, and only then either run the real matching loop (which is now guaranteed to complete fully) or reject the order outright.

function canFillCompletely(book, incoming):        // read-only: must not mutate the book
    oppositeSide = book.asks if incoming.side == BUY else book.bids
    needed = incoming.remaining
    availableAtLevel = oppositeSide.levelsIterator()  // in price priority order

    for level in availableAtLevel:
        crosses = (incoming.side == BUY  and incoming.price >= level.price) or \
                  (incoming.side == SELL and incoming.price <= level.price)
        if not crosses:
            break
        needed -= level.totalRestingQuantity()
        if needed <= 0:
            return true
    return false

function handleFOK(book, incoming):
    if canFillCompletely(book, incoming):
        matchIncoming(book, incoming)                // same loop as Component 03; guaranteed to fully fill
        assert incoming.remaining == 0
    else:
        rejectOrder(incoming, reason = FOK_INFEASIBLE)  // book untouched; no partial fill, no rest

Test F from the canonical book: FOK BUY 100 @ $101.00. The feasibility pass sums resting quantity crossable at or below $101.00: S1 (50) + S2 (30) = 80 total available, which is less than the requested 100. canFillCompletely returns false, and the order is rejected with zero fills — the book is completely untouched, exactly as Section 4's Test F specifies.

FOK feasibility under concurrent requests

Because the matching engine is single-writer per book (Component 01), "concurrent" FOK requests for the same instrument are not actually concurrent from the book's point of view — they are sequenced one after another by the sequencer, and each one's feasibility check and (if feasible) execution happen atomically within that single order's turn on the single-writer thread. No other command can be interleaved between the feasibility check and the matching call for the same instrument, because there is only one thread touching that book. This is what makes the feasibility check sound: it is impossible for the available liquidity to change between the check and the fill, because nothing else can run in between.

Stop and stop-limit orders

A stop order is not a resting order in the visible book — it is a trigger condition (typically "last traded price crosses a stop price") held in a separate, engine-managed watch list. The matching engine evaluates the watch list after every trade print: if a trade's price satisfies a pending stop's trigger condition, that stop order is converted into a live order (a market order, or a limit order at its stop-limit price) and submitted into the normal matching loop at that moment — receiving a fresh sequence number at trigger time, not at original submission time. This has a direct priority consequence: a stop order triggered a moment after a regular limit order arrives at the same effective price will queue behind that regular order, because its time priority is set by when it was triggered and converted, not by when it was originally placed. Trigger-source policy (last trade vs best bid/offer) and whether stops are supported at all are venue-specific choices, not a universal exchange rule.

Day expiry and auctions as a separate matching mode

A DAY order that is never filled or cancelled is expired by the engine at the end of the trading session — removed from the book without being matched, with an execution report reflecting EXPIRED status. Auctions (opening, closing, volatility) are explicitly not an extension of the continuous cross-or-rest loop above: an auction accumulates orders without matching them during a call phase, then computes a single uncrossing price that maximizes executable volume and executes eligible orders at that one price simultaneously. That is a distinct matching mode with its own algorithm, outside the scope of the continuous-matching pseudocode in this page, and real venues differ meaningfully in how they run it.

Every order has a state machine, and the cancel/replace rules only make sense in terms of that state machine: an order can be cancelled only while it is still RESTING, and a replace is modeled as a new version of the same logical order, not as an in-place edit that erases history.

Order state machine

FILLED, CANCELLED, REJECTED, and EXPIRED are terminal — no further command can transition an order out of these states. This matters directly for cancel races: once an order reaches FILLED, no subsequently processed cancel command for that order can be accepted; it can only be told "this order is already filled."

Cancel-before-fill vs fill-before-cancel

Consider Test D: a cancel request for S1 races against an incoming buy order that would match S1. Both commands are, eventually, sequenced into the same per-instrument stream — and whichever one the sequencer places first is, unconditionally, the one that determines the outcome. If the cancel is sequenced before the incoming buy, S1 is removed from the book; when the buy order is then applied, it skips past where S1 used to be and matches whatever is now at the head of the $101.00 level (S2) instead. If the incoming buy is sequenced before the cancel, S1 is (partially or fully) filled first; when the cancel is then applied, the engine finds S1 already in a terminal or partially-terminal state and can only cancel whatever quantity, if any, is still legitimately RESTING — it can never retroactively undo the trade that already executed.

Idempotent cancel and unknown-order handling

A cancel request for an order ID the book does not currently recognize as RESTING can mean one of three things: the order was already fully filled, it was already cancelled by an earlier request, or the order ID simply never existed on this book. The engine distinguishes these by checking the order's last known terminal state (kept available via the order index / journal, even after removal from the live book) rather than treating "not found in the live book" as a single generic error. A retried cancel for an order that was already successfully cancelled returns the same successful outcome again — cancel is idempotent by construction, not by accident.

Replace: price/size changes and queue-priority consequences

A replace (cancel/replace or amend) request changes an order's price and/or remaining quantity. The policy question with real consequences is: does the replaced order keep its original time priority, or does it go to the back of the queue at its (possibly new) price level? The common and defensible rule — used in this design — is: any price change, or any quantity increase, loses time priority (the order is effectively treated as cancelled and re-entered at the tail of its new price level's queue); a quantity decrease alone keeps the order's original time priority (it is still, legitimately, the same resting interest, just smaller). This prevents a participant from using a trivial price tweak to "jump the queue" ahead of orders that have been resting longer at that price.

Versioning, correlation IDs, and why "undo the order" is not a model

Every accepted command (new, cancel, replace) increments a version counter on the logical order and carries the member's original client order ID alongside the exchange-assigned order ID, so that execution reports can always be correlated back to the request that caused them even across several replaces. Thinking of cancel/replace as "editing the order in place and forgetting the old version ever existed" breaks this correlation and, worse, implies that a fill that already happened under an earlier version could somehow be revisited by a later replace — it cannot. Once a quantity has traded, it is gone from the order's remaining quantity permanently; replace can only ever act on whatever quantity is still legitimately resting at the moment it is applied.

The gateway's receive order is just an input signal — the sequencer is the single authority that assigns each accepted command its true, final position in the per-instrument stream that the matching engine will ever see.

Gateway receive order vs accepted sequence

Multiple gateway sessions serve many members concurrently, and network jitter means the order in which TCP packets physically arrive at a gateway host has no necessary relationship to when members actually clicked submit. If the exchange matched strictly in gateway-arrival order, a member on a congested network path would be unfairly and inconsistently disadvantaged relative to one on a faster path, in a way that has nothing to do with trading intent. A per-partition sequencer instead takes validated, risk-approved commands from potentially many gateway sessions and assigns each one a single monotonically increasing sequence number for that instrument's stream — this sequence number, not arrival time, is the only input the matching engine ever consults for ordering.

Per-partition sequencing, not one global sequencer

Sequencing is scoped per instrument partition (Component 11), not globally across the whole exchange. ACME orders and NOVA orders are sequenced into two entirely independent streams with two independent sequence-number spaces; there is no claim, and no need for a claim, that an ACME command and a NOVA command have any defined relative order with respect to each other. Price-time priority is a per-book guarantee — promising a single total order across every instrument on the exchange would require a bottleneck shared by every instrument, which is exactly the kind of unnecessary global coordination this design avoids.

Retries and reordering

A member's session may retry a command after a timeout, believing it was lost, when in fact the original was already accepted and sequenced. Each command carries a durable client-assigned command ID; the gateway/risk layer checks this ID against recently processed commands for that session before forwarding anything to the sequencer, so a retried duplicate is recognized and answered with the original outcome rather than being sequenced a second time and matched twice. This is a narrower, more honest guarantee than "exactly-once delivery over the network" — it is duplicate-suppression via durable command IDs, not a claim that the network itself cannot duplicate a packet.

Ordering under gateway failover

If a gateway instance fails and a session reconnects through a different gateway instance, that new gateway must resume at the correct per-session outbound sequence number — not silently restart from zero, which would make it impossible to tell a genuinely new command from a replay of one already accepted before the failover. Gateways coordinate session sequence state (often via the same durable command-ID/session-sequence tracking used for retry suppression) so that a failover is, from the sequencer's point of view, indistinguishable from the member simply reconnecting on the same gateway.

Timestamps, clock uncertainty, and audit records

The sequencer's assigned sequence number is the authority for matching order — but wall-clock timestamps are still recorded at intake, at sequencing, and at apply time, because they matter for audit, latency measurement, and regulatory reporting even though they do not determine matching outcome. Because clocks on different hosts can never be perfectly synchronized, these timestamps carry inherent uncertainty on the order of the clock synchronization protocol's guaranteed bound (commonly single-digit milliseconds or better with modern time sync, though the exact bound is an infrastructure assumption, not a universal constant). Treating a timestamp comparison between two different hosts as a tie-breaker for matching priority would silently reintroduce the exact race condition the sequencer exists to eliminate.

Order entry needs a reliable, ordered, recoverable session — which is why FIX and FIX-like binary protocols run over TCP — while market-data fan-out to thousands of subscribers favors UDP multicast, where occasional loss is acceptable because the feed is explicitly not the source of execution truth.

FIX-style order entry

FIX (Financial Information eXchange) is a widely used, tag-value message protocol for order entry, cancels, replaces, and execution reports; it is one illustrative example of a venue-level order-entry protocol, not a claim that every exchange uses FIX for its hot path. A FIX session carries its own session-level sequence number, independent of the matching engine's per-instrument sequence number from Component 06 — the FIX sequence number tracks message delivery integrity on that one TCP connection (detecting drops or gaps between gateway and member), while the matching sequence number tracks the command's position in the per-instrument matching stream. Resend requests let a member ask the gateway to retransmit a range of session-sequenced messages it is missing, and heartbeats at a fixed interval let either side detect a dead connection faster than TCP's own keepalive timers typically would.

Binary order-entry protocols as a venue-specific alternative

Many venues additionally offer (or exclusively offer) a proprietary binary order-entry protocol alongside or instead of FIX, trading FIX's human-readable, widely-supported tag-value encoding for a fixed-layout binary message that is cheaper to parse and serialize — relevant when shaving microseconds off gateway processing time matters. Binary protocols still need the same conceptual pieces as FIX: message framing (how a reader knows where one message ends and the next begins), a schema/version tag (so the gateway can reject or translate messages from an outdated client library rather than misinterpreting bytes), a checksum (to detect corruption), sequence numbers, and heartbeats. There is no single standardized binary protocol across the industry the way there is a single FIX specification — this is explicitly venue-specific territory.

Why TCP for order entry and UDP multicast for suitable market-data fan-out

Order entry needs guaranteed, in-order, exactly-once-processed-per-command delivery on one connection between one member and the exchange — a small number of connections, each carrying commands whose loss or reordering is unacceptable, which is exactly what TCP (plus FIX/binary session semantics on top) is built for. Market-data dissemination has the opposite shape: one feed, potentially thousands of subscribers, where occasional packet loss is tolerable because Component 12's snapshot-plus-delta protocol already assumes gaps will happen and designs explicit resynchronization for them. UDP multicast lets the network fan out one physical send to every subscriber on the multicast group, instead of the publisher opening and feeding a separate TCP connection per subscriber. This is a genuine trade-off between two different reliability shapes, not a claim that UDP is simply "faster TCP" or that every venue's market-data transport is UDP multicast.

Message framing, checksums, and session resets

Every message, regardless of protocol, needs: framing (a length prefix or delimiter so a stream of bytes can be split back into discrete messages), a schema/version identifier (so a gateway can refuse to apply a message it does not understand rather than silently misparsing it), and a checksum (to detect in-flight corruption before acting on a message). On the order-entry side, a detected sequence gap on a session triggers a resend request rather than silently skipping ahead; a session that becomes too desynchronized to recover incrementally is reset entirely, with both sides agreeing to restart sequence numbering from an explicit, logged point — never by silently assuming the gap did not matter.

Gateway rate limiting and connection management

Gateways enforce per-session rate limits (maximum messages per second, maximum open orders) both to protect downstream risk and sequencing capacity from being overwhelmed by a single misbehaving or malicious session, and to keep one member's burst from degrading fairness for everyone else sharing that gateway instance. A session that exceeds its limit receives explicit rejections (with a clear reason code) rather than being silently queued indefinitely — an unbounded queue at the gateway would just move the overload problem one hop downstream instead of solving it, and would make tail latency for every other session on that gateway unpredictable.

Pre-trade risk checks run before a command is allowed to consume a sequence number, which means a rejected order never touches the book and never costs the matching engine any work — the risk gate is a filter in front of sequencing, not a step inside matching.

What the exchange's risk gate checks

At minimum: is this member/session authenticated and authorized to trade this instrument at all (membership and symbol permissions); does the order's price fall within an acceptable band around a reference price (a price collar, rejecting obvious fat-finger errors like an extra zero on the quantity or price); does the order's size or notional value exceed a configured per-member limit; and is the instrument currently in a trading state (open, halted, auction-only) that allows this order type to be accepted right now. Position, credit, and margin checks are also common, but exactly which of these an exchange enforces, and with what thresholds, is venue policy — not a universal rule every exchange implements identically.

Exchange risk controls vs a broker's own checks

It is worth being explicit about a distinction that is easy to blur: a broker typically runs its own pre-trade risk checks on behalf of its end customer (does this customer have sufficient buying power, does this order fit their account's risk profile) before the order ever reaches the exchange gateway. The exchange's risk engine is a separate, independent layer that protects market integrity and the exchange's own operational safety — it does not know or care about a retail customer's account balance; it cares about whether this member's order, at the exchange level, could destabilize the book or exceed the member's exchange-level limits. An exchange cannot rely on a broker having done its job correctly, so the exchange-level gate is mandatory regardless of what any broker claims to have already checked.

Risk state freshness and reservations

A member's remaining order-size or notional limit is itself mutable state that changes as the member's orders are accepted, filled, or cancelled — which raises the same kind of ordering question Component 06 addresses for matching. If risk checks read a limit, approve an order, and only update the limit afterward, two near-simultaneous orders from the same member could each be approved against a limit that, combined, they actually exceed. The practical approach is to treat a limit check as a reservation: approving an order provisionally decrements the available limit immediately as part of the same risk-check step, and that decrement is released (or finalized as consumed) based on the order's eventual terminal outcome — not re-read as if nothing had changed in between.

Interaction between risk checks and the sequencer

Because risk checks happen before sequencing (Component 01's pipeline), they do not need to be serialized through the same single-writer-per-instrument structure the matching engine uses — a member's orders across different instruments can be risk-checked in parallel. What does need care is ensuring that risk state shared across those parallel checks (like one member's aggregate exposure limit spanning several instruments) is itself updated atomically per check, using the same reservation-based approach described above, so that parallelism in the risk layer does not reintroduce the same kind of race the matching engine was built to avoid.

Fail-safe policy and latency implications

If the risk service backing a gateway becomes unavailable or unresponsive, the safe default is to reject (fail closed) rather than to accept orders without a risk check (fail open) — accepting unchecked orders because the risk service happens to be down would let exactly the kind of fat-finger or limit-breach scenario the gate exists to catch through unexamined, at the worst possible moment. This fail-closed choice has a direct latency cost: every order's acceptance path includes waiting for a risk-check response, and that response time becomes part of the exchange's synchronous order-acceptance latency budget (Component 14), not something that can be skipped to save time without changing the safety guarantee.

The journal is the exchange's durability contract made concrete: an order is only truly acknowledged once its effect has been written to durable storage, and after any crash, replaying that journal from the last snapshot must reconstruct an identical book and identical trade IDs — not merely an equivalent one.

What gets journaled, and the acknowledgment contract

Every sequenced command that the matching engine applies, along with the execution events it produces (trades, state transitions), is appended to an immutable, append-only event journal before the corresponding acknowledgment or execution report is released to the member. This ordering — write to durable storage, *then* acknowledge — is the entire point of the journal: an acknowledgment that went out before the write was durable would be a promise the exchange could not actually keep if it crashed a moment later. "Durable" here specifically means the write has reached storage that is expected to survive the failure modes the exchange is designed to tolerate (e.g., process crash, single-disk failure, depending on the replication/commit policy chosen in Component 10) — not merely that it has been handed to an OS buffer.

Sync, group commit, and replication: durability vs latency

A fully synchronous write (fsync every single event to disk before acknowledging) gives the strongest per-event durability but adds a disk-latency cost to every single command's acknowledgment path. Group commit batches several sequenced events into one fsync call, trading a small, bounded amount of added latency (waiting briefly to batch) for substantially higher sustained throughput, since the fixed per-fsync overhead is now amortized across many events instead of paid once per event. Layering synchronous replication to a standby on top of local durability (Component 10) changes the question from "is this safe on disk" to "is this safe even if this entire host is lost" — at the cost of acknowledgment latency now including a network round trip to the standby. None of these is universally correct; the choice trades acknowledgment latency against how much loss (measured as recovery point objective, RPO) is acceptable if the worst assumed failure occurs.

Snapshot at sequence N, replay N+1 onward

Replaying the entire journal from the beginning of time after every restart would make recovery time grow without bound as the exchange operates longer. Instead, the engine periodically writes a snapshot of its full in-memory book state, tagged with the exact sequence number N it reflects. Recovery then means: load the most recent valid snapshot (state as of sequence N), then replay journal events starting at sequence N+1 forward, re-applying each one through the identical apply logic Component 01 and Component 03 use during live matching. Because that apply logic is a pure function of (current state, next command), replaying N+1 onward against the snapshot at N must produce exactly the same book — and critically, the same trade IDs — as if those events had never been interrupted.

Checksums, schema versions, and corruption detection

Every journal record and every snapshot carries a checksum, verified on read during recovery — a corrupted record is detected and treated as a recovery-blocking event (falling back to an earlier known-good snapshot and replaying forward from there) rather than silently applied with garbage data. Journal and snapshot formats also carry an explicit schema version, so that a later software version can still correctly parse and replay journal segments written by an earlier version — schema evolution for the journal must stay backward-compatible for exactly the same reason event schema evolution matters for market data and post-trade events.

A concrete crash-between-commit-and-ack example

Suppose an incoming command is matched, the resulting trade event is successfully appended to the journal and fsynced — durably committed — but the process crashes before the execution report reaches the member's session. From the exchange's point of view, this trade is fully committed: it will appear, correctly, in the replayed book after recovery, with the same trade ID it would have had without the crash. From the member's point of view, the outcome is genuinely unknown until they reconnect. The correct resolution is not to re-execute anything — the trade already happened and is already durable — but to redeliver the already-generated execution report from the journal once the session reconnects, using the trade's durable ID so the member can recognize it as the same event rather than a new one. This is why execution reports and trade IDs must themselves be derived deterministically and durably at commit time, not regenerated fresh on each redelivery attempt.

Exactly one host may be the active matching authority for a given instrument partition at any moment — replication exists to make recovery from that host's failure possible, never to let a second host match concurrently against the same book.

Active-passive partition ownership

Each instrument partition (Component 11) has exactly one active (primary) engine instance applying sequenced commands to the live book, and one or more standby (passive) replicas that continuously receive the same journal stream the primary is writing and apply it to their own copy of the book, staying caught up without ever accepting live commands themselves. The standby's book is therefore always a close, lagging mirror of the primary's — useful for fast takeover, but it is not a second active matching engine and must never be treated as one.

Epochs, fencing tokens, and why naive active-active is unsafe

Every time leadership for a partition changes — whether through planned failover or because the primary was declared dead — the new primary begins operating under a strictly higher epoch (term) number than any epoch that has ever been active for that partition. This epoch is attached to every journal write and every outbound message the new primary produces. Downstream consumers (the journal storage itself, replicas, gateways) are configured to reject any write or message tagged with an epoch lower than the highest epoch they have already observed — this is the fencing mechanism. It exists specifically to handle the dangerous case where a former primary that was declared dead (because it stopped responding to health checks, for example) was not actually dead — perhaps it was just partitioned from the network — and reconnects later still believing it is the active primary. Its epoch is now stale, so every write it attempts is rejected by fencing, even though the process itself is still alive and still trying to act as primary.

Standby catch-up and the takeover protocol

Promotion is not instantaneous and must not be treated as such. Before a standby is allowed to begin accepting new commands as the new primary, it must confirm it has replayed the journal through at least the last sequence number the previous primary is known to have durably committed — promoting a standby that is behind would silently lose committed trades from the new primary's view of history. This is the same quorum/durability question Component 09 raises for commit policy: a replication scheme with stronger durability guarantees (synchronous replication to the standby before acknowledgment) bounds how far behind the standby can ever be, directly shrinking the catch-up window and the risk of promoting a replica that is missing recent commits.

Split-brain prevention and operator controls

Beyond epoch fencing at the write layer, split-brain prevention typically also requires a quorum or external arbitration mechanism to decide, in the first place, whether the old primary is truly unreachable or whether the standby's view ("I can't reach the primary") is itself the result of a network partition that the primary doesn't share. Promoting a standby without that confirmation risks creating two hosts that each believe they are the legitimate primary, even before either one writes anything — which is why production failover procedures combine automated health detection with either a quorum system or an explicit operator-in-the-loop decision for anything but the clearest, most unambiguous failure signal, and always retain a manual override to force a specific host into standby/read-only mode regardless of what automated detection believes.

Scaling a matching system to thousands of instruments means running many independent single-writer engines side by side, one per instrument (or small group of instruments) — not making one engine faster at serving all of them, and not relaxing the single-writer rule for any one of them.

Routing by instrument to a stable owner

Every instrument is assigned to exactly one owning engine instance at a time, tracked in routing metadata that gateways and internal services consult to know where to forward a given instrument's commands. This metadata carries its own version number: when ownership changes (instrument migration, failover), the version increments, and components are expected to refresh their routing view rather than trust a cached mapping indefinitely. A stale route — forwarding a command to an engine instance that no longer owns that instrument — must be detected and rejected rather than silently accepted, for exactly the same reason stale-epoch writes must be fenced in Component 10: accepting it would let two different places believe they're both legitimately handling the same instrument's commands.

Why one hot instrument stays a bottleneck no matter how many engines exist

Adding more engine instances helps throughput only to the extent that load is spread across different instruments' independent books. A single instrument experiencing an extreme order-rate spike — a volatility event concentrated in ACME, for instance — is still, by design, served by exactly one single-writer engine thread, because splitting one instrument's book across multiple concurrent writers is precisely the concurrency hazard Component 01 exists to avoid. This means per-symbol throughput has a real ceiling set by that one thread's processing rate, and no amount of horizontal scaling across other instruments raises that particular ceiling. Multi-core, multi-engine scaling helps the exchange as a whole serve many instruments in parallel; it does not, and architecturally cannot, make any single instrument's book faster to mutate.

Backpressure, throttling, and admission policy under hot-symbol load

When commands arrive for one instrument faster than its single-writer engine can apply them, a bounded queue in front of that engine absorbs the burst rather than either dropping commands silently or letting the queue grow without limit. A queue that is allowed to grow unboundedly converts a throughput problem into a latency-and-memory problem: commands sit waiting long enough that their eventual acknowledgment is effectively stale by the time it arrives, and queue memory itself becomes a new failure risk. Bounded queues paired with admission control (the gateway explicitly rejecting or throttling new commands for an overloaded instrument, rather than accepting them into an already-saturated queue) keep the degradation visible and recoverable instead of silently compounding.

Safe instrument migration without losing ordering

Moving an instrument's ownership from one engine instance to another (for rebalancing load, or during planned maintenance) must not create a window where two engines could both apply commands for that instrument. The safe sequence mirrors leader fencing: the old owner stops accepting new commands for that instrument and finishes applying anything already in its queue, confirms (durably) the final sequence number it processed, the routing metadata is updated to the new owner with an incremented version, and only then does the new owner begin accepting commands for that instrument, picking up the sequence number stream exactly where the old owner left off. Gateways that still hold the stale routing version must detect the mismatch (via the version number) and refresh before forwarding further commands, exactly as in Component 10's fencing discussion.

Market data is published as a snapshot plus a stream of sequenced incremental deltas, specifically so that a subscriber who detects a gap in that sequence can resynchronize from the next snapshot rather than silently trusting a feed that may have lost a message — because the feed itself is never the authoritative source of what actually executed.

What gets published: top of book, depth, and trade prints

Three related but distinct kinds of market data come out of the matching engine's apply step. Top of book: the current best bid and best ask (price and aggregate quantity), updated whenever either changes. Depth (level 2): multiple price levels on each side, typically up to some configured number of levels, showing aggregate resting quantity per level (not necessarily individual order identities). Trade prints: a record of each executed trade — price, quantity, and the book sequence number at which it occurred — published as its own event distinct from the book-depth deltas, since a trade print and a depth update are different facts about the same underlying match (a trade print says what executed; the depth delta says how the resting book's visible levels changed as a result).

Snapshot plus delta protocol

A subscriber joining the feed mid-session cannot make sense of a stream of deltas without first establishing a baseline: it requests (or is served, depending on the feed's design) the current full snapshot of book state, tagged with the market-data sequence number it reflects, and then applies every subsequent delta whose sequence number is exactly one greater than the last sequence number it has applied. This is a distinct sequence-number space from both the matching engine's internal per-instrument command sequence (Component 06) and the journal's commit sequence (Component 09) — a market-data sequence number tracks position in the *published feed*, which is downstream of, and asynchronous relative to, the matching engine's own internal ordering.

Gap detection and resynchronization

If a subscriber receives a delta whose sequence number is not exactly one greater than the last one it successfully applied, it has detected a gap — some delta between the two was lost in transit (common over UDP multicast, per Component 07) or arrived out of order. The subscriber's only correct response is to stop applying deltas, request (or wait for) a fresh full snapshot at a new sequence number, and resume applying deltas from that new baseline forward. Guessing at the missing content — interpolating, or simply skipping ahead and hoping the book state converges anyway — is explicitly not a valid recovery strategy, because an incrementally-applied delta stream has no way to self-correct a missed update; it would compound into a silently wrong book.

Fan-out tiers, entitlement, and publisher isolation

Not every subscriber receives the identical feed at the identical latency tier — direct, co-located low-latency feeds (often used by a smaller set of latency-sensitive subscribers under specific commercial/entitlement terms) are a different distribution tier from a public, higher-latency consolidated data API serving a broader audience; conflating the two, or assuming every exchange offers identical entitlement tiers, misrepresents how market-data distribution actually works across venues. Regardless of tier, the publisher process(es) responsible for market-data fan-out are architecturally isolated from the matching engine's own synchronous critical path — a slow, backlogged, or even temporarily failed market-data publisher must never be allowed to apply backpressure into the matching engine itself, since that would let a market-data problem degrade order-acceptance latency, which is a strictly higher-priority concern.

Trade-event order vs dissemination order, and feed latency vs execution truth

The order in which trades are disseminated to market-data subscribers is derived from, and must be consistent with, the order in which they actually occurred inside the matching engine (tracked by the engine's own committed sequence) — but dissemination itself is asynchronous and happens after the fact, so there is an inherent, variable delay between "this trade executed and was journaled" and "a given subscriber's feed client has applied the corresponding delta." This gap is exactly why market data must never be treated as the authoritative record of what executed: a market-data feed can be delayed, can be temporarily lossy, and reflects only what has been disseminated so far, while the journal (Component 09) reflects the exchange's actual, already-committed execution history at the moment it happened.

A trade executing on the matching engine and that trade being cleared, netted, and settled are three separate events in three separate systems, connected by asynchronous, durable handoff — not one synchronous database write that does all of it at once.

Trade capture and immutable execution records

The moment the matching engine emits a trade event (Component 03/09), that record — buy order ID, sell order ID, price, quantity, trade ID, committing sequence number — is captured into an immutable trade-capture store, independent of whatever happens downstream afterward. This record is the exchange's own durable, authoritative account of what executed; it exists and is complete the instant the trade is journaled, regardless of whether clearing, netting, or settlement have even begun yet, let alone finished.

Clearinghouse/CCP, netting, and depository/CSD as distinct downstream entities

A clearinghouse / central counterparty (CCP) interposes itself between the two sides of a trade, becoming the buyer to every seller and the seller to every buyer, and typically nets many trades across a session into a smaller number of net obligations per member before settlement — this is a financial risk-management and operational function, entirely distinct from, and happening after, the exchange's own matching. A depository / central securities depository (CSD) is the entity that actually holds and transfers the underlying securities' book-entry ownership at settlement. None of clearing, netting, or depository transfer is a step the matching engine performs or waits on — the exchange's job (matching and reporting an execution) is complete well before any of this downstream processing finishes.

Asynchronous processing, reconciliation, and acknowledgments

Trade records flow from trade capture to clearing/post-trade systems asynchronously — typically as a durable event stream or batched export, not a synchronous call made as part of the matching path. The downstream clearing system processes these at its own pace and eventually acknowledges receipt/processing back to the exchange's post-trade pipeline; a reconciliation process compares the exchange's trade-capture record against the clearing system's received records on an ongoing basis to detect any discrepancy (a trade the exchange recorded that clearing never received, for instance) before it becomes a real operational problem.

Downstream failures do not reverse valid exchange executions

If the clearing/post-trade system is unavailable or a batch export fails, the exchange's already-committed trade remains exactly as valid as it was the moment it executed — a downstream outage is a delivery and reconciliation problem to be resolved (retry the export, replay the durable event stream once clearing recovers) once the clearing side is healthy again, never a reason to retroactively void or alter a trade that the exchange has already journaled and reported to its participants. The matching engine's execution is final the moment it is committed; everything downstream either succeeds in processing it (perhaps late) or is repaired to eventually process it — it is never a basis for reversing it.

Corrections require explicit, controlled processes

Genuine trade errors do occasionally need correction (a trade bust) — but this is explicitly not the same operation as a regular order cancel, and it is not something clearing or any downstream system can trigger unilaterally against the exchange's record. A trade bust/correction is a distinct, governed, explicitly-authorized process (typically requiring defined authority and an auditable justification) that produces its own new, clearly-linked correction event — it does not silently rewrite or delete the original trade record, preserving a complete, honest audit trail of both the original execution and its later correction.

Latency engineering in a matching system is mostly about controlling the tail — p99 and p99.9, not the average — because a handful of unpredictably slow acknowledgments is far more damaging to fairness and trust than a slightly higher but consistent typical latency.

Three different latencies, not one number

"Latency" in this system means at least three genuinely different measurements, and conflating them produces misleading claims. Network-to-network (or wire-to-wire) latency is the time from a member's packet leaving their network to the exchange's response packet arriving back — dominated by physical distance and network hops, largely outside the matching engine's control. Gateway-to-acknowledgment latency is the time from the gateway receiving a command to the member receiving their acknowledgment or execution report, which includes validation, risk check, sequencing, matching apply, and journal commit (per the acknowledgment contract in Component 09) — this is the latency figure that is actually meaningful to compare across architectural choices. Engine processing latency is narrower still: just the time the single-writer engine thread spends on the apply step itself for one command, excluding networking, risk, and journal I/O entirely. Any illustrative latency number in this page refers to one of these three specifically, and conflating them (quoting an engine-only number as if it were the full gateway-to-acknowledgment latency a member actually experiences) is a common and misleading mistake.

Illustrative latency budget

Kernel/network stack, CPU affinity, NUMA, and lock avoidance

Common latency-engineering techniques at the host level include: pinning the engine thread to a dedicated CPU core (CPU affinity) so the OS scheduler never preempts it mid-apply or migrates it between cores, which would invalidate CPU cache state; busy-polling the network interface instead of relying on interrupt-driven I/O, trading higher constant CPU/power usage for lower and more predictable latency on each incoming packet; being deliberate about NUMA (non-uniform memory access) topology so the engine thread's memory allocations stay on the same NUMA node as the core it runs on, avoiding the extra latency of cross-node memory access; and avoiding locks entirely on the single-writer engine's hot path (which Component 01's single-writer design already enables, since there is no concurrent writer to lock against) in favor of lock-free handoff queues only at the boundary where other threads hand sequenced commands to the engine thread.

Cost and operational consequences

None of these techniques are free. Dedicating cores to busy-polling and pinned engine threads reduces the number of cores available for other work on the same host, and increases baseline power/cooling cost regardless of actual traffic. Kernel bypass networking and specialized NICs add deployment and operational complexity — fewer engineers on a team are typically equipped to debug a kernel-bypass network path than a standard socket stack. This is precisely the trade-off the Trade-offs & Decisions companion page frames explicitly as "custom low-latency stack vs. conventional managed infrastructure": worthwhile when the workload and competitive latency requirements justify it, and a real ongoing cost when they do not.

Benchmarking fairly

A fair latency benchmark fixes and discloses its assumptions: hardware, network topology (same rack vs. cross-datacenter), order-book state at the moment of measurement (an empty book's first fill is not representative of a fill that sweeps several price levels), and which of the three latency definitions above is being reported. Benchmarks that omit these details, or that quote a best-case single measurement as if it were a representative p99, are not comparable to anything and should be treated with the same skepticism this page asks readers to apply to any unverified microsecond claim.

Every command the exchange accepts must be traceable back to an authenticated identity, every meaningful state transition must leave a tamper-evident audit record, and the operations team must be able to see book-integrity, lag, and latency signals continuously — because a matching engine that cannot prove what happened, and when, cannot be trusted with other people's capital.

Identity, certificates, and authorization

Each member session authenticates using credentials appropriate to the chosen protocol and venue policy — commonly mutual TLS certificates or session keys established at connection time — and every subsequent command on that session is attributable to the authenticated member identity, not merely to a source IP or an unauthenticated claim embedded in the message itself. Authorization is layered on top of authentication: a correctly authenticated member may still be rejected for a specific instrument or order type they are not permitted to trade, enforced at the pre-trade risk gate (Component 08) using membership/symbol-permission data that is itself a durable, auditable configuration, not an ad hoc check.

Tamper-evident audit logs and event correlation

The same immutable journal that provides crash-recovery durability (Component 09) doubles as the backbone of the audit trail: every accepted command, every risk decision (including rejections and the specific rule that triggered one), every match, and every cancel/replace outcome is recorded with its sequence number, timestamp, and the authenticated identity that caused it, in a log structure designed to make undetected retroactive modification difficult (e.g., chained checksums across journal segments, write-once storage policies). Correlation IDs tie every execution report and trade event back through version history (Component 05) to the original client order ID, so a compliance review or a member dispute can reconstruct the complete causal chain for any order from submission to its final terminal state.

Market abuse and self-trade controls as policy-specific extensions

Self-trade prevention (prevent a member's own buy and sell orders from matching against each other) and broader market-abuse surveillance (detecting patterns like layering or spoofing) are real, important concerns, but they are explicitly policy-specific extensions layered on top of the baseline matching and risk architecture described in this page, not something every matching engine implements identically or that this design treats as a universal default. Where self-trade prevention is enabled, it is typically enforced as an additional pre-trade or pre-match check (reject, cancel the resting order, or cancel the newer order, depending on configured policy) rather than a change to the core price-time priority algorithm itself.

Observability: book invariants, journal lag, failover readiness

Continuous operational monitoring for a matching system centers on a specific set of signals that map directly to the correctness and durability guarantees described throughout this page: book invariant checks (bid side still strictly descending, ask side still strictly ascending, no crossed book outside an explicitly allowed auction/halt state, no negative remaining quantity anywhere) run periodically as a live sanity check rather than trusted to be true by construction alone; journal/replication lag (how far behind a standby's replayed sequence is from the primary's committed sequence) directly measures failover readiness — a standby with large lag means a longer, riskier catch-up window if failover is needed right now; queue depth at the gateway and at the engine's command intake measures how close the system is to the backpressure conditions described in Component 11; and latency histograms (not just averages) across the stages in Component 14's budget make tail-latency degradation visible before it becomes a member-facing incident.

Time synchronization and audit traceability

Although matching order is determined by sequence number rather than timestamp (Component 06), accurate and monitored time synchronization across hosts still matters for audit traceability and regulatory reporting — reconstructing "approximately when did this happen" for a compliance inquiry, or correlating an exchange-side event with a member's own logs, depends on clocks that are kept within a known, monitored bound of each other. Clock drift beyond that bound is itself an observability signal worth alerting on, independent of its (explicitly absent) effect on matching order.