See It Flow

Core Concepts

Outbox Pattern

Persist the event in the same transaction as the business data. Publish reliably from there.

Mangesh MandlikLast updated:

What Is the Transactional Outbox Pattern?

Making the decision to publish as durable as the data change that caused it

The transactional outbox pattern records a business change and the event that announces it in one local database transaction. Instead of updating an order row and then calling Kafka or RabbitMQ, the service updates the order and inserts a row into an outbox_events table, and a separate relay publishes that row to the broker afterwards. If the transaction commits, both the state change and the intent to publish exist; if it rolls back, neither does.

It exists because a service cannot atomically commit to its own database and to an unrelated broker using two ordinary writes. The outbox does not make the broker publish exactly once, does not make consumer side effects exactly once, and does not make downstream systems consistent instantly. What it guarantees is narrower and very useful: a committed change is never silently left without its event.

The mental model: the database is the only place where you can make a durable decision atomically, so write the decision to publish there too. The outbox table is a to-do list kept in the same ledger as the data. Publishing becomes a retryable chore that anyone can pick up after a crash, rather than a fragile step that has to happen inside the request. The price is that the chore may be performed twice, and that the event arrives some milliseconds or seconds after the commit, not during it.

Visual Story

Dual-write failure → inconsistency → outbox transaction → relay publish → reliable delivery

Outbox Pattern — Step-by-Step Visual Walkthrough

Outbox Pattern — Step-by-Step Visual Walkthrough
Dual Write Problem
0 / 0
💔Up nextInconsistent State

⤢ Expand for synchronized narration timeline

The Dual-Write Problem

Two systems, two commit points, and a crash window between them

Take a checkout service that marks an order paid and announces it:

Dual write (illustrative pseudocode)
UPDATE orders SET status = 'PAID' WHERE order_id = 'ord_42';   -- database commit
publish(OrderPaid { order_id: 'ord_42' });                      -- broker call

Each line is a separate commit point on a separate system. If the process dies, the network drops or the broker is unavailable between them, the two systems disagree, and nothing records that they do. Suppose customer order #4211 is committed as paid and the Kafka publish times out. The order exists. Inventory never reserves stock, shipping never prints a label, the confirmation email never goes out, and analytics never sees the sale. Nothing errored loudly enough to page anyone, and the order sits in a state no one is looking at until a customer complains.

Dual write versus transactional outboxLeft, a dual write: the service commits to the database, then crashes before the publish, so the broker and the consumers never hear about the change. Right, an outbox: the service writes the order row and the event row in one database transaction, a relay reads the committed event row and publishes it to the broker, and consumers receive it at least once. Crashes after the commit delay the event but do not lose it.Dual write: two independent writesServiceDatabaseorder PAID committedBrokerreceives nothingConsumersnever hear of itOutbox: one write, then relayServiceDatabaseorder + outbox row, one txnRelaypolling or CDCBrokerConsumersat least once1 commit2 crash before publishnothing sent1 one transaction2 read committed row3 publish4 deliver
Dual write versus transactional outbox. Left, a dual write: the service commits to the database, then crashes before the publish, so the broker and the consumers never hear about the change. Right, an outbox: the service writes the order row and the event row in one database transaction, a relay reads the committed event row and publishes it to the broker, and consumers receive it at least once. Crashes after the commit delay the event but do not lose it.

Swapping the order does not help. Publishing first and committing second means a crash before the commit leaves a consumer reacting to a payment that never happened, which is usually worse than a missing event. Retrying both steps from the start can insert duplicates or publish an event for state that a concurrent request has since changed. Wrapping the broker call inside the database transaction does not make it atomic either: if the publish succeeds and the commit then fails, the message is already out and cannot be recalled.

Why not a distributed transaction?

Two-phase commit through XA can coordinate a database and a broker that both support it. In practice many brokers, including Kafka, do not offer an XA interface, the coordinator becomes a point of blocking and recovery complexity, and every participant must stay available for the transaction to finish. Teams that adopt event-driven designs generally accept eventual consistency in exchange for avoiding that coupling. The outbox is the standard way to buy back reliability without it.

Other ways to avoid the dual write

  • Event sourcing makes the append-only event log the source of truth, so there is only one write. It changes how the whole service stores state; see Event Sourcing.
  • Change data capture on the business tables publishes row changes without application code, but what leaves the service is a row-level change coupled to your table schema, not a deliberate domain event. An outbox table lets the service decide what it announces and in what shape.
  • Publishing from the database itself, for example by listening to the log, is the CDC relay described below. It still starts from an outbox row.

Writing State and Event in One Transaction

The outbox row must live in the same transactional database as the data it describes

The mechanism is ordinary SQL. Inside one transaction the service applies the state change and inserts the event row. Both become visible at COMMIT, or neither does.

One local transaction covers the state change and the event rowThe service sends two statements inside one database transaction: an UPDATE of the order row and an INSERT of an outbox_events row. COMMIT makes both visible together; ROLLBACK discards both. Only after the commit does a relay read the outbox row (step 3), publish it to the broker (step 4) and then mark it published in a separate, later statement (step 5). The broker is never part of the transaction.One local database transactionServiceorders rowPENDING → PAID, version + 1outbox_events rowOrderPaid, event_id, versionCOMMIT: both rows become visibleROLLBACK or crash: neither does1 UPDATE2 INSERTRelayafter commitBroker3 read row4 publish5 mark sentSteps 3 to 5 run outside thebusiness transaction, later.
One local transaction covers the state change and the event row. The service sends two statements inside one database transaction: an UPDATE of the order row and an INSERT of an outbox_events row. COMMIT makes both visible together; ROLLBACK discards both. Only after the commit does a relay read the outbox row (step 3), publish it to the broker (step 4) and then mark it published in a separate, later statement (step 5). The broker is never part of the transaction.

Steps 1 and 2 are the business transaction. Steps 3 to 5 run later, outside it, and are the relay's job. The broker is never enlisted in the transaction, which is the whole point: the transaction can only make promises about the database it runs in.

A minimal schema for PostgreSQL, assuming an existing orders table with a version column:

PostgreSQL: outbox table (runnable)
CREATE TABLE outbox_events (
  id                 bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  event_id           text        NOT NULL UNIQUE,
  aggregate_type     text        NOT NULL,
  aggregate_id       text        NOT NULL,
  aggregate_version  bigint      NOT NULL,
  event_type         text        NOT NULL,
  schema_version     int         NOT NULL DEFAULT 1,
  payload            jsonb       NOT NULL,
  created_at         timestamptz NOT NULL DEFAULT now(),
  published_at       timestamptz,
  UNIQUE (aggregate_type, aggregate_id, aggregate_version)
);

-- Keeps the relay's claim query cheap however large the table grows.
CREATE INDEX outbox_unpublished_idx ON outbox_events (id) WHERE published_at IS NULL;

The business write and the insert then run as one unit. Writing them as a single statement with a data-modifying CTE means the event row is only produced if the conditional update actually changed a row:

PostgreSQL: state change and event in one transaction (runnable)
BEGIN;

WITH paid AS (
  UPDATE orders
  SET status = 'PAID', version = version + 1
  WHERE order_id = 'ord_42' AND status = 'PENDING'
  RETURNING order_id, version
)
INSERT INTO outbox_events
  (event_id, aggregate_type, aggregate_id, aggregate_version, event_type, payload)
SELECT 'evt_' || order_id || '_v' || version,
       'order', order_id, version, 'OrderPaid',
       jsonb_build_object('order_id', order_id, 'status', 'PAID')
FROM paid;

COMMIT;

If the order was already paid, the UPDATE matches no row, the INSERT produces none, and the command tag is INSERT 0 0. The application should treat that as 'nothing to announce', not as success.

Several details here carry real weight:

  • Same database, same transaction. If the outbox lives in a different database from the orders, the problem is back, because you again have two commits. The pattern requires the outbox table to sit next to the data it describes.
  • The event id is derived, not random per attempt. evt_ord_42_v2 is built from the aggregate and its version, so a retried request that reaches the same state cannot mint a second identity for the same fact. Consumers deduplicate on this value, and the unique constraints make a double insert fail loudly.
  • Payload shape is a contract. Carry a schema_version, decide whether the event is thin (identifiers only, consumers call back) or fat (the data they need), and keep personal or sensitive fields out unless every consumer is allowed to hold them. The outbox row can sit in a table for days and is copied into a broker with its own retention.
  • A status flag is not an event. The published_at column is relay bookkeeping. Business logic should never read it.

The extra write is cheap relative to the work around it: one more insert into a table in the same database, in the same transaction, sharing its commit and its fsync. The cost shows up elsewhere, in the relay you now operate and in the latency between commit and delivery.

The Polling Relay

A worker that claims unpublished rows, publishes them, and records progress

The simplest relay is a loop: read unpublished rows, publish each to the broker, wait for the acknowledgment, and only then record that the row was published. Running several relay instances for availability or throughput needs an atomic claim so two instances do not take the same rows. In PostgreSQL that is FOR UPDATE SKIP LOCKED, which takes row locks on the rows it returns and silently skips rows another transaction already holds, instead of waiting for them.

PostgreSQL: claim, publish, mark in one relay transaction (SQL runnable, publish step illustrative)
BEGIN;

SELECT id, event_id, aggregate_type, aggregate_id, event_type, payload
FROM outbox_events
WHERE published_at IS NULL
ORDER BY id
LIMIT 100
FOR UPDATE SKIP LOCKED;

-- Application, not SQL: publish every row and wait until the broker
-- acknowledges each one. If any publish fails, ROLLBACK and retry later.

UPDATE outbox_events
SET published_at = now()
WHERE id = ANY($1::bigint[]);       -- ids returned by the SELECT

COMMIT;

The order of operations is the safety property. The row is marked published after the broker has acknowledged it. If the relay crashes anywhere before the COMMIT, the transaction rolls back, the row locks are released when the connection drops, and another claim picks the rows up again. The failure mode is a duplicate, never a loss. The next section examines exactly where.

The cost of holding a transaction open

The version above keeps a database transaction, a connection and row locks open while it talks to the broker. With a healthy broker and batches of 100, that is a few milliseconds to tens of milliseconds. With a slow or partitioned broker it becomes seconds, pins a pooled connection per relay, and holds back vacuum cleanup while the transaction is open. The alternative is a short claim with a lease: a first statement marks rows as claimed until a deadline, the connection is released, the relay publishes, and a second statement records completion.

PostgreSQL: lease-based claim (assumes extra columns claimed_by text, claimed_until timestamptz)
UPDATE outbox_events
SET claimed_by = $1, claimed_until = now() + interval '30 seconds'
WHERE id IN (
  SELECT id
  FROM outbox_events
  WHERE published_at IS NULL
    AND (claimed_until IS NULL OR claimed_until < now())
  ORDER BY id
  LIMIT 100
  FOR UPDATE SKIP LOCKED
)
RETURNING id, event_id, aggregate_id, event_type, payload;
-- RETURNING does not guarantee order: sort by id in the application.

If a relay stalls past its lease, another relay re-claims the rows and the broker may see the same event twice. That is acceptable here because every publish carries the same event_id; the lease only has to bound wasted work.

Leases in a job queue raise harder questions, such as a slow worker that finishes after its lease expired and then writes a stale result. Those matter far more when the work has side effects that cannot be deduplicated, and Job Processing covers lease expiry and fencing tokens in depth. A relay is a simpler case because publishing the same event id twice is already tolerated by design.

What "acknowledged" means depends on the broker

The relay may only mark a row after a success signal it can trust, and what counts differs:

  • Kafka. Wait for the producer's send future to complete with acks=all so the leader and in-sync replicas have the record. Setting enable.idempotence=true removes duplicates caused by the producer's own internal retries, and preserves per-partition order across them. That protection is scoped to one producer session: a relay that restarts is a new producer and the broker cannot tell its resend from a new message. Use the aggregate id as the record key so one aggregate maps to one partition.
  • RabbitMQ. Enable publisher confirms and wait for the confirm. Also set the mandatory flag or publish to an exchange with an alternate exchange: without it, a message the exchange cannot route to any queue is still confirmed and then dropped, so the relay would mark a row published that no queue ever received.
  • Amazon SQS. A successful SendMessage response means the message was stored. A FIFO queue can deduplicate on MessageDeduplicationId, and using the event id there is natural, but that deduplication only applies within a five-minute window, so it narrows duplicates rather than eliminating them.

Practical tuning

  • Latency is the poll interval. A relay polling every second adds up to a second on top of broker time; the old rule of thumb of one to five seconds is a choice about load, not a property of the pattern. PostgreSQL LISTEN/NOTIFY can wake the relay immediately after a commit, but treat it as a hint: notifications can be missed if the listener is disconnected, so keep the periodic poll as the source of truth.
  • Index the claim predicate. The partial index above keeps the query proportional to the backlog, not to the table's history.
  • A poison row blocks its batch. A payload the broker rejects every time, for example one over the size limit, will fail the whole batch forever if the relay simply rolls back and retries. Count attempts per row, back off between tries (see Retry & Backoff), and park rows that exhaust them so one bad event does not stall everything behind it. What to do with the parked row is a dead-letter question.
  • Respect a slow broker. Cap in-flight publishes and slow the claim rate when acknowledgments lag, as described in Backpressure. The outbox table is a durable buffer, so slowing down costs latency, not data.

The CDC Relay

Reading the transaction log instead of polling the table

A change data capture relay reads committed changes from the database's transaction log and turns inserts into outbox rows into broker messages. For PostgreSQL, Debezium does this with logical decoding through a replication slot, and its outbox event router transformation maps outbox columns onto the message: the aggregate id becomes the record key, an aggregate type selects the topic, and the payload becomes the value. Column names default to a documented convention and are configurable, so follow the documentation of the Debezium version you run.

Two ways to get a committed outbox row to the brokerTop lane, polling: a relay process repeatedly runs SELECT with FOR UPDATE SKIP LOCKED against the outbox table, publishes the rows to the broker and marks them published. Bottom lane, change data capture: a connector such as Debezium reads the PostgreSQL write-ahead log through a logical replication slot, an outbox event router turns each inserted outbox row into a message, and the message is written to Kafka. Both lanes end with consumers receiving at least once.Polling relay (application-managed)outbox_eventstableRelay workersBrokerConsumersChange data capture (log-based)PostgreSQL WALcommit orderDebeziumconnector + routerKafkaConsumersSELECT … SKIP LOCKEDpublish + await ackdeliverlogical decodingroute by aggregatedeliver
Two ways to get a committed outbox row to the broker. Top lane, polling: a relay process repeatedly runs SELECT with FOR UPDATE SKIP LOCKED against the outbox table, publishes the rows to the broker and marks them published. Bottom lane, change data capture: a connector such as Debezium reads the PostgreSQL write-ahead log through a logical replication slot, an outbox event router turns each inserted outbox row into a message, and the message is written to Kafka. Both lanes end with consumers receiving at least once.

The top lane is the polling relay: an application component reads the table. In the bottom lane nothing queries the table. The connector follows the write-ahead log, so it adds no polling load, picks up changes as soon as they are committed, and reads transactions in commit order, which sidesteps the identity-ordering hazard described below. Because the insert is visible in the log, the row can even be deleted in the same transaction that wrote it, so the table stays empty and cleanup disappears. Debezium's outbox documentation describes that approach; confirm it suits your recovery and audit needs before relying on it, since an empty table also removes your ability to inspect or re-drive past events from SQL.

What CDC adds to operate

  • Replication slot retention. PostgreSQL keeps WAL segments until the slot's consumer has confirmed them. A stopped or lagging connector makes WAL accumulate on the primary and can fill its disk, which is an availability incident for the source database itself. Alert on slot lag and set a safety limit such as max_slot_wal_keep_size (PostgreSQL 13 and later), accepting that exceeding it invalidates the slot and forces a re-snapshot and a recovery plan.
  • At-least-once, not exactly-once. Debezium commits source offsets periodically. After a crash the connector resumes from the last committed offset and can re-emit events it had already written to Kafka. Duplicates are expected, and consumers still need to deduplicate on the event id.
  • Snapshots and schema changes. When the connector first starts, its snapshot mode decides whether rows already in the table are emitted. Renaming or retyping outbox columns changes what the router sees. Both need a deliberate rollout.
  • More moving parts. Kafka Connect, connector configuration, offset storage and its own lag metric join your on-call surface. The guarantee must be evaluated for the exact source, sink and failure mode you run. No CDC tool makes a downstream HTTP call exactly once.

Choosing between them

Start with polling when throughput is moderate, you want everything inside the application's own deployment and tooling, and a second of added latency is fine. It needs nothing beyond SQL and a worker. Move to CDC when polling load or latency becomes a real constraint, when you already run Kafka Connect, or when you need strict commit-order publication without engineering it yourself. The earlier idea that a platform already running CDC makes an application outbox unnecessary is only half right: CDC on the business tables gives you row changes. The outbox table is what turns them into deliberate domain events, and CDC is one way to relay it.

Why Duplicates Still Happen

The relay prefers publishing twice to publishing never

Consider the window between the broker's acknowledgment and the UPDATE that records it. The relay publishes evt_ord_42_v2, the broker stores it and acknowledges, and the relay process is killed before it can mark the row. As far as the database is concerned nothing happened. After restart the row is still unpublished, so the relay publishes it again.

Relay crashes after the broker acknowledged, then replaysThe service commits the order and the outbox row. The relay claims the unpublished row, publishes it and receives the broker acknowledgment, then crashes before the UPDATE that marks the row published. The row is still unpublished, so after restart the relay claims it again and publishes the same event_id a second time, receives another acknowledgment, and finally marks the row published. The broker delivers two copies; the consumer's inbox table recognises the second event_id and skips it.ServicePostgreSQLRelayConsumerBroker1. COMMIT order + outbox row2. claim unpublished row3. evt_ord_42_v24. publish evt_ord_42_v25. ack6. crash before UPDATE7. restart: claim again8. publish evt_ord_42_v2 (2nd)9. ack10. UPDATE published_at11. deliver (copy 1 and 2)12. inbox: 2nd copy skipped
Relay crashes after the broker acknowledged, then replays. The service commits the order and the outbox row. The relay claims the unpublished row, publishes it and receives the broker acknowledgment, then crashes before the UPDATE that marks the row published. The row is still unpublished, so after restart the relay claims it again and publishes the same event_id a second time, receives another acknowledgment, and finally marks the row published. The broker delivers two copies; the consumer's inbox table recognises the second event_id and skips it.

Step 6 is the crash. The acknowledgment in step 5 is an ack from the broker to the relay and is not recorded anywhere durable, so the relay cannot know it already happened. This is not a defect to engineer away. When you are told only that you hold unfinished work, resending is the only safe reaction. The alternative is to risk the event vanishing, and the pattern exists to rule that out.

The order of marking is therefore not a detail. Marking before publishing converts the duplicate into a loss. Use the explorer to see both orders under every crash point:

after step 3
Where a relay crash leaves the eventOrder of operations: Mark after acknowledgment. Steps: 1 claim row, 2 send to broker, 3 receive ack, 4 mark published, 5 COMMIT. Crash point: after step 3. Copies at the broker before recovery: 1. The published mark is not committed, so the row is still unpublished and the relay retries. Copies after recovery: 2. One duplicate: the broker holds two copies of the same event_id.1 claim row2 send to broker3 receive ack4 mark published5 COMMITcrash after step 3published mark becomes durable
Where a relay crash leaves the event. Order of operations: Mark after acknowledgment. Steps: 1 claim row, 2 send to broker, 3 receive ack, 4 mark published, 5 COMMIT. Crash point: after step 3. Copies at the broker before recovery: 1. The published mark is not committed, so the row is still unpublished and the relay retries. Copies after recovery: 2. One duplicate: the broker holds two copies of the same event_id.
Copies at broker before recovery
1
Row after restart
published_at NULL
relay will retry
Copies after recovery
2

One duplicate: the broker holds two copies of the same event_id.

With the safe order, the relay only ever risks one extra copy, and a crash after the send but before the ack may or may not have reached the broker, so the relay honestly cannot say whether the retry duplicates. With the unsafe order, a crash after the commit but before the send leaves the row marked published for an event no broker ever saw, and nothing in the system will notice. Some connector and broker combinations offer stronger, scoped transactional guarantees, for example Kafka transactions between Kafka topics, but none of them extend to your database row or to a consumer's external side effects.

The consequence is a contract with every consumer: events are delivered at least once, and the same event id can arrive more than once, including after a long delay. How a consumer honours that contract is covered in the inbox section below; the general theory of delivery guarantees, redelivery and acknowledgment is the subject of Queues & Workers, and the handling of repeated requests in Idempotency.

Event Ordering and Aggregate Versions

What order the database committed in is not necessarily the order rows were numbered in

Many workflows need OrderCreated to be seen before OrderPaid. An outbox relay can preserve that order, but only if you decide what "order" means and design for it. Two things look reliable and are not.

Timestamps and identity values are not commit order

In PostgreSQL, now() returns the transaction's start time, so created_at can be far earlier than the moment the row became visible. Identity and serial values are allocated when the row is inserted, not when the transaction commits. The figure shows two transactions on different aggregates.

Identity values are allocated at insert, not at commitTransaction A inserts its outbox row first and receives id 10 but stays open until tick 60. Transaction B, working on a different aggregate, inserts later, receives id 11 and commits at tick 40. A relay poll at tick 50 can only see committed rows, so it sees id 11 and not id 10. The poll at tick 70 finds id 10. Events are therefore published as 11, then 10, even though id 10 was assigned first. A relay that remembered 'highest id seen = 11' would skip id 10 forever. Ticks are arbitrary and illustrative.020406080ticks (illustrative)Txn A: insert row id 10, still opencommits at 60Txn B: insert row id 11commits at 40Relay poll at tick 50sees id 11 onlyRelay poll at tick 70sees id 10
Identity values are allocated at insert, not at commit. Transaction A inserts its outbox row first and receives id 10 but stays open until tick 60. Transaction B, working on a different aggregate, inserts later, receives id 11 and commits at tick 40. A relay poll at tick 50 can only see committed rows, so it sees id 11 and not id 10. The poll at tick 70 finds id 10. Events are therefore published as 11, then 10, even though id 10 was assigned first. A relay that remembered 'highest id seen = 11' would skip id 10 forever. Ticks are arbitrary and illustrative.

Transaction A gets id 10 and stays open; transaction B gets id 11 and commits first. A poll at tick 50 sees only id 11. A relay that keeps a high-water mark of the highest id published would then skip id 10 forever when it finally commits. A relay that selects WHERE published_at IS NULL does not lose it, as in the query above, but it publishes 11 before 10. Across unrelated aggregates that is harmless. Within one aggregate it cannot happen by itself: two transactions that both update the same order row serialize on its row lock, so the second one cannot even insert its outbox row until the first commits. That is a reason to always touch the aggregate row, for example by bumping its version, in the same transaction that writes its event.

Per-aggregate order needs three cooperating pieces

  • A version in the event. The unique constraint on (aggregate_type, aggregate_id, aggregate_version) makes gaps and duplicates detectable, and consumers whose projections require strict sequencing can check that each event is exactly one version ahead of the last they applied, buffering or re-fetching when it is not.
  • An ordering-aware relay. Several relay instances each claiming arbitrary batches with SKIP LOCKED can publish version 3 of an order before another instance has published version 2. Either run a single relay, or shard so that one instance owns every event of a given aggregate (for example by hashing the aggregate id), or claim only the oldest unpublished row per aggregate. The CDC relay gets this from the log's commit order, provided the publish step keeps it.
  • A partition key. In Kafka, order holds only within a partition, so use the aggregate id as the record key. For RabbitMQ and SQS, ordering depends on queue type and consumer concurrency; an SQS FIFO queue orders within a message group, so the aggregate id fits as the group id. How consumer groups, partitions and ordering interact is a Pub/Sub topic.

Ordering is a design choice with a throughput price. One single-threaded publisher gives a strong global order and becomes a bottleneck. Most systems only need order within an aggregate, which allows parallelism across aggregates, so ask which order the business actually requires before paying for more. A retry that republishes version 2 after version 3 has gone out can also reorder events at the consumer, so stale-version checks on the consumer side are cheap insurance.

Consumers: the Inbox Pattern and External Side Effects

The outbox protects the producer's intent. The consumer still has to survive duplicates

Where each guarantee starts and stopsFrom left to right: the database transaction makes state and event row atomic; the relay publishes the row to the broker at least once; the broker stores the message until a consumer acknowledges it; the consumer's inbox table makes its own local database effect happen once per event_id; and any call to an external API outside that database needs its own idempotency key. Each box guarantees only what its sub-label says.Database txnstate + event atomicRelayat-least-once publishBrokerdurable per its configConsumer inboxone local effectExternal callneeds idempotency key
Where each guarantee starts and stops. From left to right: the database transaction makes state and event row atomic; the relay publishes the row to the broker at least once; the broker stores the message until a consumer acknowledges it; the consumer's inbox table makes its own local database effect happen once per event_id; and any call to an external API outside that database needs its own idempotency key. Each box guarantees only what its sub-label says.

Read the figure left to right. The transaction makes state and event row atomic. The relay turns that into an at-least-once publish. The broker holds the message according to its own durability settings. Each further box makes its own, smaller promise, and none of them extends the transaction's atomicity beyond the database. A consumer that receives the same event twice and applies it twice has not been helped by the outbox at all.

The standard remedy is the inbox pattern: the consumer records the ids of events it has processed in its own database, in the same transaction as its business update. A duplicate then collides with the marker and changes nothing.

PostgreSQL: inbox marker and business effect in one consumer transaction (runnable)
CREATE TABLE inbox_events (
  consumer      text        NOT NULL,
  event_id      text        NOT NULL,
  processed_at  timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (consumer, event_id)
);

BEGIN;

INSERT INTO inbox_events (consumer, event_id)
VALUES ('inventory', 'evt_ord_42_v2')
ON CONFLICT DO NOTHING;
-- Application: if the command tag is INSERT 0 0 this event was already
-- processed. COMMIT (or ROLLBACK), acknowledge the message, and stop here.

UPDATE stock
SET reserved = reserved + 1
WHERE sku = 'sku_9' AND available - reserved >= 1;
-- Application: if no row was updated, handle out-of-stock as a business outcome.

COMMIT;

Primary-key uniqueness serializes concurrent copies: a second transaction inserting the same marker waits for the first to commit, then does nothing.

Acknowledge the broker message only after this transaction commits. If the consumer crashes after the commit but before the acknowledgment, the broker redelivers and the marker turns the second delivery into a no-op. The inbox table needs retention longer than the broker's longest possible redelivery or replay window, otherwise a very late duplicate can slip past once its marker is purged.

The inbox gives an effectively-once local effect. It does not extend to other systems. If the consumer calls a payment provider, sends an email or writes to another database, that call is outside the inbox transaction. Pass a stable idempotency key derived from the event id so the provider can deduplicate, as covered in Idempotency, or add reconciliation for effects that cannot be deduplicated. When a business operation spans several services and a failure in one step requires undoing another, such as releasing stock when a refund fails, you are in saga territory. That is coordination the outbox does not provide alone; see Saga Pattern.

Schema, Cleanup, Retention and Monitoring

The table is a queue. Operate it like one

Cleanup without losing events

Published rows have no further use except audit and replay, so an outbox without cleanup grows until the polling query, vacuum and backups all suffer. Keep rows for a retention window that exceeds your longest expected relay outage plus the time you would want to investigate or re-drive an event, then delete in small batches so you neither lock long ranges nor bloat the write-ahead log with one giant delete.

PostgreSQL: batched purge of old published rows (runnable)
DELETE FROM outbox_events
WHERE id IN (
  SELECT id
  FROM outbox_events
  WHERE published_at < now() - interval '7 days'
  ORDER BY id
  LIMIT 5000
);
-- Repeat until it deletes fewer than 5000 rows.

The published_at predicate is the safety property: a row with published_at NULL can never match, so unpublished events are never purged.

For very high write volumes, partitioning by time lets you drop a whole partition instead of deleting row by row. The risk is that a partition is indexed by created_at, not by publication status. A relay outage long enough to outlive the retention window would have rotated away partitions that still hold unpublished rows. Before dropping a partition, verify that it contains no row with published_at IS NULL. For CDC with the delete-on-insert approach, there is nothing to purge.

The signals that matter

The primary health signal is the age of the oldest unpublished event. An API that returns 200 while the relay is stuck is not a healthy event-driven system, because the business state is right and everything downstream is quietly going stale.

PostgreSQL: backlog and oldest unpublished age (runnable)
SELECT count(*)                              AS unpublished_rows,
       coalesce(now() - min(created_at), interval '0') AS oldest_unpublished_age
FROM outbox_events
WHERE published_at IS NULL;
  • Oldest unpublished age and backlog size: the SLO signal for the relay.
  • Publish attempts and failures per row, and broker acknowledgment errors: how close a row is to being parked.
  • Outbox table size and dead tuples: whether cleanup and vacuum are keeping up.
  • CDC connector lag and replication slot retained WAL: the log-based equivalent, with disk risk attached.
  • Consumer lag and duplicate-event rate (inbox conflicts): whether downstream keeps up and how often replay happens.
  • Business commits versus events delivered: alert on sustained divergence, not on single slow events.

To size the problem, use the model below. It is deliberately simple: it assumes the relay publishes at its healthy capacity as soon as it is back and ignores broker and consumer limits, so real recovery is slower. The point is the shape: backlog grows with the outage, and drains only at the margin by which relay capacity exceeds the arrival rate.

100 / s
300 / s
10 min
Unpublished backlog while the relay is down, then drainingWith 100 new outbox rows per second and a relay down for 10 min, the unpublished backlog grows linearly to 60,000 rows. When the relay returns it publishes 300 rows per second while 100 new rows keep arriving, so the backlog shrinks by 200 per second. It takes 5.0 min to drain. Business data stays correct throughout; downstream views are stale. Illustrative model: it ignores broker and consumer limits.60,0000relay downrelay back, drainingunpublished rows over time (illustrative)
Unpublished backlog while the relay is down, then draining. With 100 new outbox rows per second and a relay down for 10 min, the unpublished backlog grows linearly to 60,000 rows. When the relay returns it publishes 300 rows per second while 100 new rows keep arriving, so the backlog shrinks by 200 per second. It takes 5.0 min to drain. Business data stays correct throughout; downstream views are stale. Illustrative model: it ignores broker and consumer limits.
Backlog at restart
60,000 rows
Oldest unpublished age
10 min
at restart
Net drain rate
200 / s
capacity − arrivals
Time to drain
5.0 min

Production Failure Analysis

Each failure has its own symptom, metric and recovery

Take a relay outage at a store that creates about three outbox rows per second. The numbers are illustrative, and chosen to be easy to follow:

  1. 10:00: the relay is deployed with a bad broker credential and starts failing every publish. Customers still check out normally and orders commit, because the request path never touches the broker.
  2. 10:01: oldest unpublished age crosses a 60-second alert threshold. This is the first signal. The API dashboards are green.
  3. 12:00: after two hours the backlog is about 21,600 rows (3 per second × 7,200 seconds). Inventory, shipping and email have seen no new orders in two hours.
  4. 12:05: the credential is fixed and the relay drains the backlog in order, without losing a single event, because every row survived in the database.
  5. 12:10: consumers receive a burst of two hours' worth of events. Those with a limited downstream capacity need their own protection (see Backpressure), and delayed events must not be treated as fresh by handlers that assume recency.

That story shows both sides of the trade: nothing was lost, and nothing was prompt. The other failures follow the same pattern:

  • The database commit fails. Neither the state change nor the outbox row exists. The caller sees the error, nothing was announced, and there is nothing to reconcile.
  • The relay crashes after the broker acknowledged. The row is republished. Symptom: a rise in the duplicate rate at consumers. Recovery: none needed if consumers deduplicate; if they do not, you find out as double reservations or double emails.
  • A consumer rejects the schema. The relay sees a successful publish, but the consumer fails repeatedly. The event stalls at the consumer or lands in a dead-letter queue, and the fix is to repair the contract and replay, not to touch the outbox. See Dead-Letter Queues.
  • The outbox table grows without cleanup. Claim queries slow down, storage fills, and vacuum falls behind. The partial index limits the damage to polling but not to storage.
  • A consumer assumes exactly-once. It works for months because duplicates are rare, then a relay restart or a connector rollback produces them and a customer is charged twice. The defect is in the consumer, and the outbox only exposed it.
  • Events are published out of order. OrderPaid reaches a consumer before OrderCreated because parallel relays split an aggregate. The remedy is versions, per-aggregate relay ownership and consumers that detect gaps, as above.
  • A CDC slot stalls. The source database retains WAL it cannot discard, and disk usage grows on the primary. This one can degrade the database that serves your users, so it deserves a louder alert than an ordinary backlog.

A Worked Example: Order Paid

One event, from checkout to inventory, with every duplicate and delay accounted for

A payment service confirms a card charge for ord_42. In one transaction it moves the order to PAID at version 2 and inserts evt_ord_42_v2 as in the SQL above. The HTTP response returns as soon as that commits.

  1. A Debezium connector or a polling relay picks up the new row and publishes it to the order topic with ord_42 as the key, so every event for this order is in one partition.
  2. Inventory consumes the event, inserts ('inventory', 'evt_ord_42_v2') into its inbox and reserves stock in the same transaction, then commits its offset.
  3. Shipping, email and analytics are separate consumer groups on the same topic and each applies its own inbox. They progress independently; the slow one only delays itself. This fan-out is the topic of Pub/Sub.
  4. If the relay is killed after the broker acknowledged but before it marks the row, the relay publishes again and the broker holds two copies. Inventory's second delivery hits the inbox primary key, reserves nothing, and acknowledges.

The same shape appears wherever a committed change must reach other systems: a completed ride triggering billing, rewards and driver payout; a funds transfer feeding the ledger, fraud scoring and the customer notification; an account change feeding search, billing and audit. In every one of them the consumers are separate services with separate failure modes, which is why the event has to be as durable as the transaction that produced it.

When to Use the Outbox Pattern, and When Not To

Choose it because lost events are unacceptable, not because every microservice has one

Use it when a local database transaction must reliably cause an asynchronous event: order and payment state changes, billing transitions, account lifecycle events, ledger and compliance feeds, and any notification whose absence would leave another system permanently wrong. The test is simple: if the event is lost, does someone have to notice and repair it by hand?

It is the wrong tool in a few cases:

  • The event is fire-and-forget telemetry or analytics that can be sampled or dropped. A direct publish with a retry is simpler and the occasional loss is acceptable.
  • The service has no relational write of its own, only a message in and a message out. A broker-native mechanism such as Kafka transactions between topics may already cover consume-process-produce.
  • The source of truth is already an event log, so there is no separate database write to keep in step with it.
  • Nothing downstream consumes the change. Simple CRUD with no event has no dual write to fix.

What you take on is a table, a relay (a worker or a connector), cleanup, new metrics, a contract of duplicates with every consumer, and eventual-consistency latency between commit and delivery. For a system where lost events cost real money or compliance exposure, that is a small price. For one where they do not, it is overhead.

Key Takeaways

What each hop guarantees, and what stays your responsibility

  • The dual write is the bug: write the state change and the outbox row in one local transaction, so the decision to publish is as durable as the data change. A distributed transaction across database and broker is the heavier alternative.
  • The relay must mark a row published only after the broker acknowledges. Marking first turns a duplicate into a silent loss, and the safe order leaves one extra copy at worst.
  • Delivery is at-least-once end to end, so every consumer needs an inbox or equivalent dedupe in the same transaction as its effect. Duplicates can arrive long after the original.
  • Commit order is not identity or timestamp order. Per-aggregate ordering needs versions, single-owner relaying per aggregate, and consumers that detect gaps.
  • Operate the table like a queue: alert on the age of the oldest unpublished row, purge published rows in batches, and watch connector lag and retained WAL.
  • A stalled CDC replication slot can fill the primary's disk, so it warrants a louder alert than an ordinary relay backlog.

Where to Go Next

The outbox is one link in the delivery chain

The relay hands events to a broker, so Queues & Workers explains what at-least-once delivery really means once the message is there, and Pub/Sub shows how one event fans out to several independent consumers. The consumer half of this lesson depends on Idempotency, and events that cannot be processed belong in a dead-letter queue. For multi-service workflows with compensation, read Saga Pattern.