Warren
Guide

Your quorum dead-letter queue can drop the dead letters: x-delivery-count travels along

2026-10-06 · 7 min read · Viktor Baumann

A dead-letter queue is supposed to be the place where failed messages are safe until someone looks at them. On RabbitMQ 4 with quorum queues and default settings, that is not quite true: a message that died because it was delivered too often can arrive in the DLQ with its delivery budget already used up, and the first consumer or tool that crashes while holding it, or even just peeks at it, makes RabbitMQ drop it for good. Here is what happens, measured on RabbitMQ 4.3.6, and three ways to make the DLQ safe again.

The short version as a video (1:42, captions, no sound needed):

Watch it on YouTube →

Delivery limits in 30 seconds

Quorum queues count how often a message was delivered and not settled. Past the queue's delivery limit (x-delivery-limit, or delivery-limit in a policy) RabbitMQ stops redelivering it: the message is dead-lettered with reason delivery_limit if the queue has a dead-letter exchange, and dropped otherwise. Since RabbitMQ 4.0 every quorum queue has a limit of 20 unless you set one, which is good: a poison message no longer loops forever.

What counts changed in 4.0 as well:

What happens to a delivered message3.134.x
basic.nack with requeue=truecountsdoes not count
basic.reject with requeue=truecountscounts, and the message goes to the back
management UI "Get messages" in its default mode, "Nack message requeue true"countsdoes not count
the channel or connection closes while it is unacknowledged (consumer crash, pod killed, network drop)countscounts
basic.reject with requeue=false (dead-lettered)not measuredcounts
put back (any of the above, or a peek with requeue) while its count is already above the queue's limitnot measureddropped, or dead-lettered on

The count is visible on the message as the header x-delivery-count (RabbitMQ 4 adds x-acquired-count, every acquisition including requeues).

What we measured

A work queue and a dead-letter queue, both quorum queues, no limits set, so both have RabbitMQ 4's default of 20:

ch.exchange_declare('lim.dlx', 'fanout', durable=True)
ch.queue_declare('lim.dlq', durable=True, arguments={'x-queue-type': 'quorum'})
ch.queue_bind('lim.dlq', 'lim.dlx')
ch.queue_declare('lim.work', durable=True,
                 arguments={'x-queue-type': 'quorum', 'x-dead-letter-exchange': 'lim.dlx'})
ch.basic_publish('', 'lim.work', b'order-4711')

Then a consumer that crashes on this message every time: fetch it without acknowledging and close the connection, until the work queue gives up.

while message_count('lim.work') > 0:
    conn = pika.BlockingConnection(params)
    conn.channel().basic_get('lim.work', auto_ack=False)
    conn.close()          # the consumer "crashed" while holding the message

The results:

abandoned deliveries on lim.work until it gave up: 21
in lim.dlq: reason delivery_limit, x-delivery-count 21
left in lim.dlq after ONE abandoned delivery there: 0

The message reached the DLQ, as it should, with reason delivery_limit. But it brought its count of 21 along, and the DLQ's own limit is 20. The next abnormal return, a single one, made the DLQ give up on it too, and since the DLQ has no dead-letter exchange of its own, the message was dropped.

It does not even take a crash. In a second run we only looked: one basic.get followed by basic.nack with requeue=true, and once more through the management UI's "Get messages" with requeue. Both times the DLQ was empty afterwards. On RabbitMQ 4.3.6 a message whose count is already above the queue's limit is dropped the moment it is put back, abnormally or not; a count equal to the limit survives. So the usual first step of an investigation, peeking at the DLQ, is what loses the message.

Why the count travels

The delivery count is not just a header the queue writes for you to read: it is part of the message's annotations, and dead-lettering keeps them. The DLQ continues counting from where the work queue stopped. A header that a publisher sets is a different thing: we published a fresh message with x-delivery-count: 5 into a queue with a limit of 5, and RabbitMQ ignored it. The next delivery showed 1, and the message survived five abandoned deliveries. So the carried count comes from dead-lettering, not from the header itself.

It is not only delivery_limit messages. In the same test with explicit limits, a message that had been abandoned twice and then rejected arrived in the DLQ with a count of 3, and with a DLQ limit of 5 it was dropped after three more failures, not five. Every message rejected by a consumer arrives with at least 1.

Who abandons a delivery in a DLQ?

More often than you would think:

On 3.13 it was worse (every requeue counted, so even looking at messages in the management UI cost deliveries), but 4.x's default limit makes the carry-over bite with no configuration at all.

Three fixes

All three kept the message through three abandoned deliveries in the DLQ in our test.

1. No delivery limit on the DLQ. A DLQ is not consumed in a loop, so it does not need one. By policy:

rabbitmqctl set_policy dlq-no-limit '\.dlq$' '{"delivery-limit": -1}' --apply-to quorum_queues

or with x-delivery-limit: -1 when you declare the queue; a negative value means unlimited. Only one policy applies to a queue: if one matches your DLQs already, add the key there.

2. A much higher limit, if you want a backstop anyway: x-delivery-limit: 100.

3. A classic queue as DLQ. Classic queues have no delivery limit. You lose the replication of a quorum queue, which matters less for a queue that is mostly written and rarely read; decide by how much you care about dead letters surviving the loss of a node.

And in whatever reads the DLQ: settle every message you fetch, ack what you have handled and nack with requeue=true what you have not. On 4.x a nack with requeue costs nothing (a reject with requeue does count), except for a message that is already above the queue's limit: the requeue drops it as well. Fix the limit before anyone reads the queue.

Checking your own queues

Checklist

The RabbitMQ 4 upgrade check finds the quorum queues in a definitions export that the delivery limit affects, and the policy generator writes the delivery-limit policy. Both run in your browser.