Guides on RabbitMQ dead-letter queues, replaying messages and alerting on queues, written by the people building Warren. Everything here works without Warren; where it fits, we say so at the end.
A producer that starts first, an auto_delete queue on a deploy, a reject without a dead-letter exchange, reject-publish without confirms, a topic star that matches one word and a headers binding that wants every header. Measured on RabbitMQ 4.3.6, with the output and the fix.
Wrong password, unknown user, guest from another host, or a vhost the user may not open: what RabbitMQ answers in each case and what the broker log says. Measured on RabbitMQ 4.3.6 with pika.
What RabbitMQ logs when a client process is killed, a load balancer drops an idle connection, or a busy client misses its heartbeats, and how to tell them apart. Measured on RabbitMQ 4.3.6 with pika.
What RabbitMQ 4.3 does to a consumer that holds a message too long: a silent basic.cancel for most clients, the channel error for the rest, and nothing at all on classic queues. Measured on RabbitMQ 4.3.6 with pika.
RabbitMQ 4 refuses messages over max_message_size (16 MiB by default) by closing the channel. Without publisher confirms the publish call returns as if nothing happened, and the next message can be lost too. Measured on RabbitMQ 4.3.6 with pika.
auto_ack, a routing-key typo that is still confirmed, a missing prefetch, x-expires on a dead-letter queue, a priority nobody reads and transient messages in a durable queue. Each one measured on RabbitMQ 4.3.6, with the output and the one-line fix.
During a disk or memory alarm RabbitMQ blocks every publishing connection. The publish call still returns, the message is not delivered, and without a timeout the client waits forever. Measured on RabbitMQ 4.3.6 with pika.
Why RabbitMQ refuses a queue declaration once you add x-dead-letter-exchange to a queue that already exists, and how to add the dead-letter exchange with a policy instead, without deleting the queue or its messages. Measured on RabbitMQ 4.3.6.
Why RabbitMQ refuses to redeclare a quorum queue without x-queue-type, and how to move a classic queue to a quorum queue when the type cannot be changed. Measured on RabbitMQ 4.3.6.
Why RabbitMQ only reports NO_ROUTE when you publish with mandatory, what happens to an unroutable message without it, and how to handle the return in your client. Measured on RabbitMQ 4.3.6.
Publishing to a missing exchange closes the channel, and without publisher confirms the publish call itself does not fail. Plus the quiet variant: a dead-letter exchange that does not exist drops every dead letter. Measured on RabbitMQ 4.3.6.
When RabbitMQ answers NOT_FOUND - no queue, and the more dangerous case where it does not: a publish to a queue name that does not exist is confirmed and the message is dropped. Measured on RabbitMQ 4.3.6.
What an exclusive queue allows other connections to do and what it refuses, why its messages vanish with the connection that declared it, and the related ACCESS_REFUSED for exclusive consumers. Measured on RabbitMQ 4.3.6.
The three ways to get 'unknown delivery tag' from RabbitMQ: acking twice, acking an auto-acked message, acking on another channel. Measured on RabbitMQ 4.3.6, including the side effect: every other unacked message on the channel is redelivered.
The fix covers one failure, the dead-letter queue holds five. How to put back only the matching messages with basic.get, why nacking each skipped message loops forever, why closing the connection can cost a quorum queue its messages, and how to split dead letters so the question does not come up.
Before 143 dead-lettered messages go back to their consumer, send one and watch whether it dies again. How to replay a single message by hand, how long to wait given your retry policy, how to tell its return from a new failure, and when a passed canary still proves nothing.
A Python consumer raises. With pika the message loops back forever, with Celery it is usually gone. How to send failures to a dead-letter queue instead, retry with a delay, and put the messages back once the bug is fixed.
An async handler throws, and the RabbitMQ message is either stuck unacknowledged, lost because NestJS acked it on arrival, or redelivered forever. How to send failures to a dead-letter queue with amqplib and NestJS, and how to put them back.
A Go consumer on RabbitMQ panics, returns an error or loses its connection. Which of those loses the message, which loops it forever, and how to send failures to a dead-letter queue with amqp091-go, count retries in x-death, and replay them.
Six tools that help with RabbitMQ dead letters, compared on what you need at 3 a.m.: can it browse a dead-letter queue, replay to the original route, does it work for a team with roles and an audit log, does it alert, and what does it cost. Checked on the vendors' own pages.
Get messages, Move messages, Publish message and Purge, measured on RabbitMQ 4.3.6: which ack mode removes a message, which one costs a delivery on a quorum queue, where moved messages end up, and what to do before you click.
A replay that the broker confirmed is not a replay that worked. How to tag replayed messages so the ones that fail again can be found, what RabbitMQ 4 keeps and what it resets when a replayed message dies a second time, and how to read the result.
On RabbitMQ 4, a message dead-lettered from a quorum queue keeps its delivery count. With all defaults, a poison message arrives in a quorum DLQ with 21 deliveries against a limit of 20, and one crash, or even one peek, of whatever reads the DLQ loses it for good. Measured, explained, and three one-line fixes.
What Spring AMQP does when a @RabbitListener throws, how to send failures to a dead-letter queue instead of an endless redelivery loop, retries with backoff, RepublishMessageRecoverer versus x-death, and how to put the messages back without losing any.
Where MassTransit puts a message when a consumer throws or nobody consumes its type, what the MT-Fault headers tell you, how retry and delayed redelivery interact with the error queue, and how to send the messages back to the endpoint without losing any.
The tools people reach for when messages have to leave a dead-letter queue, compared on the questions that matter at 3 a.m.: which messages, where to, what happens to x-death, is it loss-free, did it work, can it be undone, and who can tell afterwards what happened.
A side-by-side look at the RabbitMQ management UI and Warren for dead-letter work: browsing messages, moving them, replaying to the original route, editing, undoing a purge, history, alerting and audit. Where the management UI is enough, and where it is not.
Four ways to move messages out of a RabbitMQ dead-letter queue and back to their consumers: the management UI, a shovel, your own script, and what to watch out for with x-death, retry counters and publisher confirms.
What every field in RabbitMQ's x-death header means, how the array grows when a message cycles through retry queues, what x-first-death and x-last-death add, and how to read it from the management UI, the CLI or code.
How to detect a RabbitMQ queue whose consumers are gone before its messages expire or pile up: the management API fields to watch, Prometheus alert rules with rabbitmq_prometheus, a cron-and-curl fallback, and which queues to exclude.