Dead-Letter Queues

A dead-letter queue (DLQ) is a separate destination for messages that a consumer or the broker cannot process under the configured delivery policy: too many failed attempts, a message judged permanently bad, an expired TTL. It keeps the main processing path moving while preserving the failed messages for investigation, correction, and possible replay. It is a quarantine, not a repair: moving a message does not fix the defect that caused the failure, and replaying it carelessly can reproduce the failure or repeat a business side effect.

A typical case is an order event containing a field the current consumer cannot parse. No number of retries will change the bytes. With no bounded policy, one of two things happens depending on the broker: the message is redelivered forever, burning worker capacity on a certain failure, or, on an ordered stream, it holds up every later message behind it. Dead-lettering bounds the damage to one message, and gives engineers something concrete to look at: the payload, the reason it failed, and how many times it was tried.

The mental model: a bounded policy with an exit

Think of every message as moving through a small state machine. It is queued, delivered, and then settled one of three ways: acknowledged, retried later, or parked. The retry path is for failures that are expected to clear by themselves; the Retry & Backoff lesson covers how those attempts are spaced. The DLQ is the exit for everything else. The exit only works if it is bounded (a message cannot cycle forever), informative (it carries enough context to diagnose), and monitored (someone is told it has been used). The unbounded alternative is not safer: it either drops messages silently or lets one bad input degrade everything.

This lesson sits at the end of the asynchronous-systems path. A message reaches the DLQ after the delivery mechanics of Queues & Workers have run their course, often having been published through the Transactional Outbox or fanned out by Pub/Sub. The visual story below shows the basic loop before the sections after it take each part apart.

The decision to dead-letter should follow a classification, not just a counter. Four kinds of failure look alike from the outside (the handler threw) but call for different responses.

• Transient. A 503, a connection reset, a lock timeout, a rate limit. The same message will probably succeed later. It belongs on a delayed retry path, not in the DLQ, unless the retry budget runs out.
• Permanent. An unsupported event version, a required field that is missing or malformed, a payload that is truncated or in the wrong encoding, a reference to an entity that cannot exist. Another attempt a second later changes nothing, so retries only delay the inevitable.
• Ambiguous. A timeout after sending a charge request. The handler failed, but the side effect may have happened. Retrying blindly risks a duplicate; dead-lettering blindly hides a possibly completed operation. The first step is to find out what happened, using the original idempotency key.
• A consumer bug. A bad release that fails thousands of valid messages. These are not poison messages at all. The right response is to stop or roll back the consumer; filling a DLQ with them is the wrong outcome.

Some failures people file under “poison” are really outcomes the business logic should handle without failing. A duplicate delivery that trips a unique constraint on the event ID is not an error: the work was already done, so the consumer should acknowledge it (see Idempotency). An order that was cancelled before its event was processed is often a valid no-op or a compensation, a domain decision rather than a fault. Only when the domain rule says the message can never be applied should it go to the DLQ.

Classify on stable codes, not on exception text

Matching on exception messages is fragile; a library upgrade rewords a string and the classification silently flips. Have the code that understands the failure raise a typed error with a stable code, and keep the mapping from code to class in one place. Unknown errors default to retryable but bounded: a budget on attempts and total elapsed time still sends them to the DLQ if they never clear. The budget and the jitter on each delay are the subject of the retry lesson and are not repeated here.

type FailureClass = 'transient' | 'permanent' | 'ambiguous';

const TRANSIENT = new Set(['DEPENDENCY_UNAVAILABLE', 'RATE_LIMITED', 'LOCK_TIMEOUT']);
const PERMANENT = new Set(['SCHEMA_UNSUPPORTED', 'FIELD_INVALID', 'ENTITY_NEVER_EXISTS']);

function classify(err: unknown): FailureClass {
  if (err instanceof AppError) {
    if (PERMANENT.has(err.code)) return 'permanent';
    if (err.code === 'OUTCOME_UNKNOWN') return 'ambiguous';   // e.g. timeout after a charge request
    if (TRANSIENT.has(err.code)) return 'transient';
  }
  return 'transient';   // unknown: retry within the budget, then dead-letter
}

Ordered streams change the question

On an ordered stream the choice is not only “retry or park”. If later events for the same entity depend on this one, moving it out of the way lets those events run ahead of it. Sometimes that is acceptable and sometimes the correct behaviour is to stop that partition or message group until the event is repaired. The section on replay and ordering below returns to this, because it is a business trade-off more than a broker setting.

A main queue carries work that is ready to be processed. A retry queue, or a delayed schedule, carries work that is expected to succeed later and usually has a next_attempt_at time attached. A DLQ carries messages that exceeded a policy or need remediation by a person or a dedicated process. They differ in what the system promises about the message: the main queue promises prompt delivery, the retry path promises another attempt, and the DLQ promises only that the message is kept until someone decides what to do with it. Nothing consumes the DLQ automatically, which is precisely why it must be watched.

How the retry path is built varies a lot between brokers, so it should not be assumed to be uniform. SQS has no separate retry queue; a failed message simply becomes visible again when its visibility timeout ends, and a consumer can lengthen that delay for a specific message with ChangeMessageVisibility. SQS also offers a per-queue delay of up to 15 minutes for newly sent messages. RabbitMQ has no native delayed delivery for arbitrary delays on its core queue types, so teams chain queues with a message TTL whose dead-letter exchange points back at the work queue, or use a delayed-message plugin. With the TTL approach, use one queue per delay tier rather than per-message TTLs of mixed lengths, because expiry is evaluated at the head of a classic queue and a short-lived message can sit behind a long-lived one. Kafka has neither: applications publish to separate retry topics or hold a message in memory and pause the partition.

The figure shows the same states regardless of broker. A message is in flight after delivery and leaves that state in one of four ways: acknowledgement, a deliberate retry later, an implicit return to the queue (a visibility timeout lapsing or a negative acknowledgement with requeue), or dead-lettering. The path from the queue directly to the DLQ is less well known: in RabbitMQ a queued message can be dead-lettered without ever being processed, because its TTL expired or the queue hit its length limit. The only way back out of the DLQ is the dashed arrow, and that one should always be a deliberate act.

One consumer, one decision flow

Within a consumer, those destinations are chosen by a short sequence of checks, drawn below. The order is the point. Schema validation comes first because it has no side effects and catches the cheapest permanent failures. The idempotency check comes before the operation so that a redelivered message that was already applied is acknowledged rather than executed again. Only then does the operation run, and only its result is classified.

Read the three exits from the classification. A transient failure consults the retry budget: while attempts and elapsed time remain, the message is scheduled for later redelivery; once spent, it goes to the DLQ along with its full attempt history. An ambiguous result does not retry and does not park; it triggers a reconcile step that asks the downstream system what actually happened. A permanent error goes straight to the DLQ, as does an envelope that fails validation before anything runs. The path out of the DLQ (inspect, fix, controlled replay) is a different actor with a different tempo from the consumer, and the later sections treat it separately.

The “already applied” check is only meaningful if it is atomic with the effect it protects. When the effect is a write to the consumer’s own database, record the event ID and make the change in one local transaction. When the effect is a call to an external service, a local inbox row cannot make that call atomic, so the external call needs its own idempotency key, and the ambiguous branch exists precisely for the case where the call may or may not have landed.

async function handle(msg: Envelope): Promise<void> {
  const check = validateEnvelope(msg);                // no side effects
  if (!check.ok) return deadLetter(msg, 'SCHEMA_INVALID', check.detail);

  try {
    await db.transaction(async (tx) => {
      const first = await tx.insertInbox(msg.eventId); // false if this ID was already recorded
      if (!first) return;                              // duplicate: nothing to do, still acknowledge
      await applyBusinessChange(tx, msg);              // same transaction as the inbox row
    });
    return ack(msg);
  } catch (err) {
    switch (classify(err)) {
      case 'transient':
        return msg.attempt < MAX_ATTEMPTS && withinDeadline(msg)
          ? scheduleRetry(msg, backoffWithJitter(msg.attempt))
          : deadLetter(msg, 'RETRY_BUDGET_SPENT', summarize(err));
      case 'ambiguous':
        return reconcile(msg);                         // look up the outcome by the original key
      case 'permanent':
        return deadLetter(msg, errorCode(err), summarize(err));
    }
  }
}

“Dead-letter queue” names a pattern, not one feature. Three widely used systems implement it in three different ways, and a policy that works on one cannot be assumed on another.

SQS: a receive count and a redrive policy

SQS tracks how many times each message has been received. When a message would be received more times than the maxReceiveCount in the source queue’s redrive policy, SQS moves it to the DLQ instead of delivering it. With a limit of 3 the handler therefore runs at most three times, and the move occurs on the attempt after the third, as the figure shows. The DLQ of a standard queue must itself be a standard queue, and the DLQ of a FIFO queue must be a FIFO queue.

Two consequences follow. First, the redrive policy sees only “not deleted”. A handler that knows a message is permanently bad must either fail it repeatedly until the count runs out, which wastes attempts, or send it to an application-level DLQ itself and then delete the original. Second, the broker-side move does not record why the message failed. Keep failure reasons in logs and metrics keyed by message ID, or write them into the application-level record when you can.

# DLQ keeps messages for 14 days (1,209,600 seconds)
aws sqs create-queue --queue-name orders-dlq \
  --attributes MessageRetentionPeriod=1209600

# orders-attributes.json
# {
#   "VisibilityTimeout": "60",
#   "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:123456789012:orders-dlq\",\"maxReceiveCount\":\"5\"}"
# }
aws sqs create-queue --queue-name orders --attributes file://orders-attributes.json

# Later: send parked messages back to their source queue, throttled to 5 per second
aws sqs start-message-move-task \
  --source-arn arn:aws:sqs:us-east-1:123456789012:orders-dlq \
  --max-number-of-messages-per-second 5

RabbitMQ: dead-letter exchanges and delivery limits

RabbitMQ dead-letters to an exchange, not a queue, so a DLQ is a queue bound to that exchange. The reasons are recorded in the x-death header: rejected (a consumer rejected or nacked with requeue set to false), expired (message TTL), maxlen (queue length limit), and, for quorum queues, delivery_limit (too many redeliveries). That last mechanism matters because a consumer that crashes mid-message, or nacks with requeue, otherwise redelivers forever. Releases differ in the default limit for quorum queues, so check the one you run.

The reliability of the hand-off itself also varies. Dead-lettering is, by default, an at-most-once transfer between queues: if the dead-letter target is unavailable, the message can be lost in transit. Quorum queues offer an at-least-once dead-letter strategy, with prerequisites such as a reject-publish overflow policy, trading that risk for the possibility of duplicates and some backpressure.

# The orders.dlx exchange and a queue bound to it must exist.
rabbitmqctl set_policy orders-dlx "^orders\.work
quot; \
  '{"dead-letter-exchange":"orders.dlx","dead-letter-routing-key":"orders.work.dead","delivery-limit":5}' \
  --apply-to quorum_queues

Kafka: the application owns the DLQ

A Kafka broker does not track whether a record succeeded; consumers track progress with committed offsets in a consumer group, and each partition is read in order. So there is no broker-native DLQ. Frameworks and Kafka Connect sink connectors provide one by publishing failed records to another topic, conventionally with the original topic, partition, offset, and the error carried in headers. Treat a deserialization failure specially: if the deserializer throws inside the client, the consumer can get stuck on the same offset forever, so wrap it in an error-handling deserializer that hands the raw bytes to your failure path instead.

The order of operations is what makes a Kafka DLQ safe or lossy, and the same rule applies to every broker: the transfer to the DLQ must be durable before the original is acknowledged. Acknowledging first and publishing second means a crash between the two loses the message. Publishing first and acknowledging second means a crash can leave the message in both places, which is the right way round, because the DLQ consumer can deduplicate by event ID.

async function deadLetter(record, reason, detail) {
  await dlqProducer.send({
    topic: 'orders.dlq',
    key: record.key,                       // keep the ordering key
    value: record.value,                   // original bytes, not re-serialised
    headers: {
      'dlq.original-topic': record.topic,
      'dlq.original-partition': String(record.partition),
      'dlq.original-offset': String(record.offset),
      'dlq.reason': reason,
      'dlq.detail': detail,
    },
  });                                      // producer configured with acks=all
  await commitOffset(record);              // only after the DLQ write is acknowledged
}

A DLQ policy has to answer a question that sounds simple: how long should the system keep trying before it gives up? The usual rule of thumb is three to five attempts. It is a reasonable default, but it hides the quantity that matters, which is a duration. On SQS, the window during which a message is retried is roughly (maxReceiveCount − 1) × visibility timeout from the first receive to the last one. If a dependency is down for longer than that window, messages that arrived at the start of the outage run out of attempts before it recovers and are dead-lettered even though nothing is wrong with them.

Try a limit of 3 and a 60-second timeout against a five-minute outage. The retry window is two minutes, so roughly three-fifths of the messages that first arrive during the outage will reach the DLQ, while the dependency recovers and the consumers sit idle. Those are the messages that should have waited, and they will need a replay that never needed to happen. Raising the limit to 8 stretches the window to seven minutes and rescues them all, at the cost of more wasted runs for the genuinely poisoned ones. The figure is a simplified model: it ignores handler duration and assumes messages arrive evenly across the outage and every attempt during it fails. The point is the shape of the trade-off, not the exact percentages.

Longer windows and shorter windows are both wrong

A very short budget (a limit of 1 or 2) turns any brief blip into a DLQ flood. A very long one keeps re-running a certainly doomed message against a dependency that may already be struggling. Size the window from the failures you actually expect: how long your dependencies take to recover, and how much latency the consumer’s caller can tolerate. Use a duration-based cap in addition to an attempt cap so the policy expresses intent, not an accident of how the timeouts multiply.

Two interactions make this go wrong in practice. If the visibility timeout is shorter than the time the handler needs, the message becomes visible while still being processed, another worker receives it, and the receive count climbs with no failure at all, so slow-but-healthy messages get dead-lettered. And when a dependency is known to be down, a circuit breaker should pause the consumer or hold messages rather than let every message burn its attempts against a dependency that cannot answer. Receive counts spent on an open circuit are receive counts stolen from the message’s real chances. Consumers should also respect backpressure signals, so that retries do not return in a synchronized wave.

Recovery is only as good as the context captured at the moment of failure. Keep the original payload exactly as received (or a secure reference to it) and, alongside it:

• the stable message or event ID, which must stay the same through replay so the idempotency key is unchanged;
• the original queue or topic, and the ordering key or partition, so ordering can be reasoned about later;
• the schema version and the content type;
• the correlation or trace ID, to find the logs of the original request;
• first-failure and most-recent-failure times, and the attempt count;
• the consumer name and version, to tie failures to a deployment;
• an error classification and a short, safe diagnostic summary.

Be strict about what the diagnostics contain. DLQs are often readable by more people than the production database, so plaintext credentials, tokens, and stack traces that embed personal data do not belong in them. Store a code and a redacted summary, and keep the detailed trace in your logging system, addressed by trace ID. On SQS the move preserves the body and message attributes but not the application’s reason; on RabbitMQ the x-death headers record the reason and count; on Kafka you add the headers yourself.

When the broker does not give you a queryable inventory, or you want operators to filter and annotate failures, an application-level table is a practical complement. Note that the body is stored as raw bytes: the failures that matter most include malformed JSON, which a jsonb column would reject.

CREATE TABLE dead_letters (
  id               bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  event_id         uuid        NOT NULL,
  source           text        NOT NULL,            -- original queue or topic
  ordering_key     text,
  schema_version   integer,
  payload          bytea       NOT NULL,            -- raw bytes; may not be valid JSON
  error_code       text        NOT NULL,
  error_summary    text        NOT NULL,            -- redacted, safe to show
  attempts         integer     NOT NULL,
  consumer_version text        NOT NULL,
  first_failed_at  timestamptz NOT NULL,
  last_failed_at   timestamptz NOT NULL,
  status           text        NOT NULL DEFAULT 'parked'
                   CHECK (status IN ('parked', 'replaying', 'replayed', 'discarded')),
  replayed_by      text,
  replayed_at      timestamptz,
  replay_reason    text,
  UNIQUE (source, event_id)                        -- a repeated dead-letter of the same message is a no-op
);

CREATE INDEX dead_letters_parked_idx
  ON dead_letters (error_code, first_failed_at)
  WHERE status = 'parked';

A poison message is usually a specific input that fails every time while its neighbours succeed. Typical causes are a malformed body, a missing required field, a schema version the consumer does not know, a payload truncated or mis-encoded in transit, or a value the business rules can never accept. A widespread failure looks different: failures begin suddenly, span many unrelated messages, and line up in time with a deploy, a schema rollout, or an unavailable dependency.

If 90% of messages start landing in the DLQ right after a release, retrying them automatically will not help, and neither will dead-lettering faster. Stop or roll back the consumer, keep the broker’s retention long enough to survive the investigation, and replay once the fix is verified. If no such pattern appears, ask whether the handler made an external call that timed out. That branch is its own category because the dangerous mistake is not a lost message but a duplicated charge; the reconcile step comes before any replay. Otherwise treat it as a single poison record: isolate it and let the rest flow, subject to the ordering rules below.

What to watch

Track DLQ arrival rate broken down by error code, event type, and consumer deployment version, and track the age of the oldest message. Alert on both a sudden spike and on slow accumulation. The age matters more than people expect: a DLQ with no new arrivals can still hold customer-impacting events that are months old and about to expire, so “the rate is zero” is not the same as “nothing needs doing”. SQS exposes queue depth and the age of the oldest message as metrics; on other brokers, derive them from consumer lag on the DLQ topic or from the inventory table’s oldest first_failed_at.

For many queues, an alert on any arrival is a defensible starting point, because a message appearing in the DLQ means a failure the policy could not absorb. For high-volume pipelines where a trickle of poison messages is normal, alert on rate relative to throughput and on age, and review the trickle on a schedule rather than paging for each message. Either way, the DLQ must have an owner, or the alert has nobody to reach.

Replay reintroduces a message to processing, but time has passed. The schema may be newer, the referenced order may have been cancelled, the customer may have been refunded, or another recovery path may already have completed the side effect. The broker’s redrive button moves bytes; it knows nothing about any of this. A safe replay therefore follows a procedure:

1. Confirm the cause is fixed. Deploy the corrected consumer, or repair the data, and test against a sampled message in a non-production path first.
2. Identify the affected subset. Filter by error code, event type, and time window, so a message that failed for an unrelated reason is not swept up.
3. Preserve the original IDs. The event ID and idempotency key must be unchanged so the consumer’s duplicate protection still applies.
4. Check current business state. Decide whether replay is still meaningful, or whether the event should be discarded as superseded.
5. Rate-limit the replay. A burst of replayed messages is a load spike on a system that may have just recovered.
6. Start small and watch. Replay a small batch, check that the error does not recur, then continue; stop automatically if the same error reappears.
7. Record who replayed what and why. Keep the distinction between original processing and manual replay in the audit trail.

Where practical, offer a dry run that shows the event ID, ordering key, reason, attempt count, target destination, and expected impact before anything is sent. A replay tool that only has a “replay all” button will eventually be used badly at 3 a.m.

Replay is another at-least-once delivery

Replay is a publish, and publishes can be repeated or interrupted. A tool that marks a message as replayed and then crashes before publishing loses it; one that publishes and then crashes before marking it replays it twice. The second failure is the one the system is designed for, so choose it: publish first, then record, and rely on the consumer’s idempotency for the duplicate. The claim step below uses FOR UPDATE SKIP LOCKED so that two operators or two replay workers do not claim the same rows; it does not make the publish itself atomic.

WITH batch AS (
  SELECT id
  FROM dead_letters
  WHERE status = 'parked'
    AND source = 'orders'
    AND error_code = 'SCHEMA_UNSUPPORTED'
  ORDER BY first_failed_at
  LIMIT 50
  FOR UPDATE SKIP LOCKED
)
UPDATE dead_letters d
SET status = 'replaying',
    replayed_by = 'alice',
    replayed_at = now(),
    replay_reason = 'INC-1423: consumer 2.8.1 accepts schema v4'
FROM batch
WHERE d.id = batch.id
RETURNING d.event_id, d.payload;

-- After the publish succeeds (outside this transaction):
-- UPDATE dead_letters SET status = 'replayed' WHERE id = ANY($1);

Reconcile before replaying external effects

For financial or otherwise externally visible operations, the rule is to reconcile first. If a payment provider call timed out, ask the provider for the status of that operation using the original idempotency key. Creating a new operation ID turns a recovery attempt into a potential second charge. If the provider confirms the charge, record the result and discard the message; if the provider has no record, replay it with the same key. For an analytics projection the situation is much easier: updates that can be recomputed or upserted by event ID tolerate replay well, and a plain redrive may be enough.

Consider an order whose events are Created(v1), Paid(v2), and Shipped(v3). If Paid(v2) cannot be parsed and is dead-lettered while Shipped(v3) carries on, the order store reaches version 3 with the payment step missing. When someone replays Paid(v2) later, naive handling can overwrite newer state or violate the state machine: an order that has shipped is moved back to “paid”.

There is no single correct response; the choice is a business decision. These are the realistic options:

• Make the consumer version-aware. Reject updates that are older than the stored version, and treat a gap in the sequence as a reason to hold the later event, not apply it.
• Hold back later events for that entity. Park Shipped(v3) too, so the pair is replayed in order once the cause is fixed. This keeps the invariant at the price of some extra parked messages.
• Pause instead of dead-lettering. If strict per-aggregate order is a hard requirement, stop the partition (Kafka) or let the message block its message group (SQS FIFO) until it is repaired. Nothing outruns it, but everything behind it waits, and that is a throughput and latency cost.
• Rebuild from an authoritative source. When the aggregate can be recomputed from a log of events, replay can be replaced with a rebuild, which does not depend on the order in which stragglers return.

The behaviour on FIFO-style queues deserves care: in SQS FIFO, a message that keeps failing blocks later messages in its message group, because the queue preserves order within the group. A redrive policy that moves it to the DLQ unblocks that group, which is the right outcome for throughput and the wrong one if later messages assumed the earlier one succeeded. In Kafka, order is per partition, so dead-lettering one record and committing past it lets later records for the same key be processed first.

The same applies to replaying in bulk: replay in commit order per ordering key and, as above, check the version before applying. Batch gradually, monitor whether the error recurs, and stop if it does.

A malformed invoice event

An invoice consumer starts failing on InvoiceIssued version 4. The producer changed amount_minor_units from an integer to a string without a compatible contract. The sequence runs like this (times are illustrative):

1. 10:02 – The producer deploys the change. The first InvoiceIssued v4 arrives; deserialization throws.
2. 10:02 – The consumer classifies the error as permanent (SCHEMA_UNSUPPORTED) and writes the event to the DLQ with the schema version and error code, then acknowledges the original. Invoices v1 to v3 keep flowing.
3. 10:04 – The DLQ arrival-rate alert fires, broken down by error code and producer version, which points at the producer deployment and not at the consumer.
4. 10:20 – Operators compare the producer’s schema with the consumer’s. They decide to fix the consumer, not roll back the producer, which already has live traffic.
5. 11:05 – The fixed consumer is deployed. A sampled DLQ message is run through it in a staging path and succeeds.
6. 11:15 – A replay of 50 messages is claimed, preserving the original event IDs. The consumer’s inbox guard ensures a message that was partly processed does not change invoice state twice.
7. 11:25 – Operators check DLQ recurrence, consumer lag, and downstream invoice totals, then replay the rest at a limited rate.

The same failure with no DLQ, on a queue where the message becomes visible again on its own, would have kept redelivering the bad invoice, spending worker capacity on a certain failure, and left nobody with a record of the payload or the reason. On an ordered stream it would have stopped the partition.

A payment provider timeout

A different message in the same DLQ is a ChargeCustomer command whose call to the payment provider timed out after three attempts. It was classified as ambiguous and parked with its original idempotency key. It must not be replayed on the basis of the invoice fix. The procedure here is domain-specific: query the provider for the operation tied to that key; if the charge exists, record it and discard the command; if not, replay it with the same key so a late success cannot produce a second charge. Both failures landed in one queue; only one of them could safely use a bulk replay.

Other situations where the pattern earns its keep

• Webhook delivery. A sender cannot reach a customer’s endpoint that is down. It retries with backoff and, rather than dropping the event, parks it. When the customer fixes the endpoint, the parked events can be redelivered in order, with a note on how long they were stale.
• Schema evolution on a stream. A producer upgrade emits an event version an older consumer group cannot read. Failing records are published to a DLQ topic with their original topic, partition, and offset; the group keeps consuming everything else, and a compatibility fix is deployed before replay.
• Background jobs. A job that fails its maximum number of attempts is parked for inspection instead of vanishing. The lease and heartbeat mechanics that decide when a job has failed belong to Job Processing.

DLQs tend to be set up once and then ignored, yet they can hold failed payloads for longer than the main queue and so retain personal or sensitive data unexpectedly. Decide on purpose: how long messages are kept, who may read them, who may replay them, and who may delete them. Encrypt at rest and in transit, redact diagnostics, and document the replay authority.

Retention has a broker-specific trap. On SQS standard queues, a message’s expiry is based on its original enqueue time, and moving it to a DLQ does not reset the clock. If the source queue keeps messages for four days and the DLQ for four days, a message that was retried for three days arrives with a single day left. Set the DLQ’s retention period well above the source queue’s, and alert on the age of the oldest message with time to spare. Kafka retention applies to the DLQ topic like any other: size it for how long investigation realistically takes, and do not use a compacted topic for a DLQ, because compaction keeps only the latest record per key and would discard earlier failures for the same key.

If a message could expire before anyone looks, decide whether a separate durable record is needed: an archive in object storage, or the inventory table with its own retention rule. “Keep everything forever” is not a recovery policy; it is a data-retention liability, particularly for payloads that contain personal data subject to deletion requests.

• No bounded policy. With no DLQ and no limit, a permanent failure is either redelivered indefinitely or dropped. The first wastes capacity and, on ordered streams, blocks later messages; the second is silent data loss. A bounded policy with a destination is the baseline for almost any production queue; the exception is the ordered stream where parking would break an invariant, in which case pausing is a deliberate alternative and not an oversight.
• Unmonitored or unowned DLQ. Messages accumulate for weeks, and the missing notifications, refunds, or fulfilment steps surface in an audit or a customer complaint. A DLQ with no alert and no owner gives false confidence and is worse than an honest failure.
• DLQ as trash bin. Tens of thousands of messages sit for months. Each one is a real user action that never completed. The DLQ is a recovery queue; the measure of health is how quickly messages are diagnosed and resolved, not how fast they disappear from the main queue.
• Threshold too low or too high. A limit of one dead-letters every blip. A limit so large it keeps re-running a certain failure delays the diagnosis. See the retry budget section for the window that governs it.
• Redrive loops. Moving messages straight back to the source queue before the cause is fixed re-fails them, consumes their budgets again, and lands them in the DLQ in a cycle. Replay only after verification, in small batches.
• Acknowledging before the DLQ write is durable. A crash in between loses the message while the DLQ looks healthy.
• Replaying out of order. A Paid (v2) event replayed after Shipped (v3) moves the order backwards; the ordering section describes the countermeasures.

A DLQ also does not turn at-least-once delivery into a guarantee that nothing is lost. A message is lost if the transfer to the DLQ fails, if it expires while parked, or if an operator discards it by mistake. What the pattern guarantees is narrower and still valuable: with a durable hand-off, a monitored queue, and sufficient retention, a failed message is inspectable until a person or a process makes a decision about it. In combination with the transactional outbox on the producing side, which makes publication at-least-once, and an idempotent consumer, a failure ends up retried, parked, or reconciled, not lost; delivery is still at-least-once, so duplicates are expected and tolerated.

Use a DLQ when failed messages must remain inspectable, when retries are bounded, and when operations need a controlled path from diagnosis to repair and replay. That describes most production queues, and most of the platforms that matter: SQS and RabbitMQ give you the mechanism natively, Kafka consumers and connectors need a deliberate design, and job queues need an equivalent for jobs that exhaust their attempts. In financial and compliance-sensitive pipelines each failed message represents a real transaction and must be preserved.

Do not use it as a substitute for the things it depends on:

• correct retry and backoff, so transient failures do not reach it;
• idempotency, so replay is safe;
• schema validation and contract testing, so avoidable poison messages are not produced;
• observability and an owner, so arrivals are noticed;
• business reconciliation, for outcomes the broker cannot know about.

There are cases where it does not apply or needs care. A synchronous API with no queue returns its error to the caller, and the caller owns the retry; there is no poison-message problem to isolate. A throwaway development queue may not justify the overhead, though production queues should have one from the start. And an event that is essential to strict ordered processing should not be dead-lettered automatically unless the resulting gap is handled, as the ordering section explained.

A healthy DLQ strategy is measured by time to diagnosis and safe recovery, not by how quickly failed messages leave the main queue.

• Classify before dead-lettering: transient failures belong to retry and backoff, permanent ones to the DLQ, and external-call timeouts to reconciliation. Not every failure deserves the same destination.
• Size the retry budget as a duration, not an attempt count. On SQS the window is roughly (maxReceiveCount − 1) × visibility timeout, and an outage longer than that dead-letters healthy messages.
• Keep context with every parked message (error code, attempts, first-failed time, consumer version), because a DLQ of opaque bytes is a graveyard rather than a work queue.
• A burst of arrivals right after a release is a widespread failure: stop or roll back the consumer and replay after the fix, instead of retrying or dead-lettering faster.
• Replay is another at-least-once delivery: reconcile external effects first, go in small throttled batches, use idempotent handlers, and respect per-key ordering.
• Alert on the age of the oldest message as well as the arrival rate, give the DLQ an owner, and keep its retention longer than the source queue's.

The two lessons most tightly connected to this one are Retry & Backoff, which decides when a message should be retried before it is ever dead-lettered, and Idempotency, which makes the replay path safe. For how messages are delivered, acknowledged, and redelivered in the first place, see Queues & Workers, and for fan-out to several consumers each with its own failures, Pub/Sub. The Transactional Outbox covers the producing side, and Job Processing covers leases and fencing for work that fails midway. To keep failures from piling up in the first place, the Circuit Breaker and Load Shedding lessons are the next read.