Your dead letters, handled before the 3 a.m. call.
The management UI configures your broker and tells you a queue holds 300 messages. Warren runs the traffic on it: what those messages are, why they died, who should have consumed them, and one audited click to put them back.
Self-hosted next to your broker. RabbitMQ 3.13 and 4.x, classic and quorum queues.
The Community edition is available now, free: one compose file and you are in.
Queues
14 dead letters| Queue | Messages | Consumers | Dead-letters to |
|---|---|---|---|
| orders.dlqDLQ | 9 | 0 | – |
| billing.invoice.deadDLQ | 5 | 0 | – |
| orders.process | 3 | 4 | orders.dlx |
| shipping.label | 7 | 2 | – |
orders.dlq
9 of 9 shown| Message id | Last death | Original route | Payload | |
|---|---|---|---|---|
| order-4700 | expired ×5 wait.5m | orders → order.created | {"orderId":4700,"customer":"ACME… | |
| order-4701 | expired ×5 wait.5m | orders → order.created | {"orderId":4701,"customer":"ACME… | |
| order-4702 | expired ×5 wait.5m | orders → order.created | {"orderId":4702,"customer":"ACME… | |
| order-4703 | rejected ×1 process | orders → order.created | {"orderId":4703,"customer":"Nort… |
Replay messages
Replays
Audit log| When | By | From → To | Replayed | Status |
|---|---|---|---|---|
| just now | nina | orders.dlq → original route | 3 | COMPLETED |
| Tue 09:14 | ops-bot | billing.invoice.dead → billing.invoice | 12 | COMPLETED |
| Mon 17:02 | tomas | orders.dlq → orders / order.retry | 0 | FAILED |
Every replay: who, when, from where to where, per-message outcome, payload preview.
From 200 dead letters to a throttled replay. Half a minute, no mouse.
Recorded in the real app. Pick a recording below; the big one plays it.
The whole story in four minutes: a step-by-step walkthrough on YouTube, from 200 dead letters in the management UI to a throttled replay, with a note for the team, parking, the audit log and replay rules.
The management console configures your broker. Warren runs the traffic on it.
You keep the management UI, and you should. Warren does not compete with it and does not touch its job. It answers the four questions the console cannot, in the order on-call asks them.
Configure the broker
- Declare exchanges, queues and bindings
- Policies, limits, federation, shovels
- Broker users and permissions
- Cluster formation and node health
- Counts and rates per queue
Stays as it is. Warren never deletes queues, exchanges or bindings and never touches policies. The one queue it may declare is the parking queue an operator parks messages into. Emptying a queue is an operator's call, with a reason in the audit log.
Operate the traffic
- Read messages, payload and full death history
- Search across payloads, ids and routing keys
- Metrics history and alerting on the queues that matter
- Replay, edit, reset retries, all with an audit trail
- Park poison messages, discard or purge with a reason, publish a test message
- Roles and SSO so support can look and on-call can act
Reads through the management API, publishes over AMQP with a least-privilege user.
- 1Is something broken?
Dead-letter counts on top, history charts, rules like “DLQ above 100 for 5 min” to Slack, Teams, PagerDuty, Opsgenie, e-mail or a webhook.
- 2What exactly?
Open the queue, read the messages, search for the order id the customer just gave you.
- 3Why?
The
x-deathtimeline: which queue rejected it, how often, when, and the retry counter that would bounce it again. - 4What now?
Replay to the original route, fix the payload first if needed, and let the audit log answer the postmortem.
Built for the moment a queue turns red.
Browsing, grouping and replaying are in the recordings above. These are the parts that keep a team out of the DLQ in the first place. Each preview is drawn from the real screens; use the arrows to step through.
Know about the pile-up before the customer does.
Warren samples every queue around the clock. A rule like “orders.dlq above 100 for five minutes” fires once, reaches Slack, Teams, PagerDuty, Opsgenie, e-mail or any webhook with a link to the queue, and sends the all-clear when it recovers; in PagerDuty and Opsgenie that closes the incident. No flood, no silence. A rule on the inflow rate catches a mass failure even while a replay rule keeps draining the queue.
MESSAGES_ABOVE · NO_CONSUMERS · GROWTH · INFLOW_RATE · disk · memory · re-notify after 4 h| Rule | Condition | Channel | State |
|---|---|---|---|
| Orders DLQ growing | messages > 100 for 300 s | #ops-alerts | OK |
| Billing stuck | no consumers, ready > 0 | #billing | OK |
The 3 a.m. timeout replays itself. The broken message waits for you.
Tell Warren which dead letters are worth another try: why they died, a text such as TimeoutException, how long ago. It puts them back on a schedule, only while a consumer is there to take them, and leaves a message alone after its last attempt. Replays by hand count too. A rule that keeps failing stops itself and tells your channel.
reason · text · minimum age · max attempts · rate · live preview · audit log| Rule | Takes | Last run |
|---|---|---|
| Retry payment timeouts orders.dlq · every 10 min | rejected · “TimeoutException” after 15 min · up to 3× | replayed 12 4 min ago |
| Invoices after deploy billing.invoice.dead · every 5 min | expired · any text right away · up to 2× | no consumer nobody reads yet |
| Label retries shipping.label.dlq · every 30 min | republished · “503” after 1 h · up to 5× | stopped after 3 failures |
Give support the read access. Keep the replay button for on-call.
Three roles that build on each other: viewers browse queues, payloads and history, operators additionally replay and edit, admins additionally manage alerts and users. Create accounts in the UI, lock one in a second, or map roles from your identity provider's claims. Warren refuses to remove its last administrator.
local accounts · OIDC claim → role · bcrypt · last-admin guardUsers
| On | Username | Roles | Last login |
|---|---|---|---|
| nina | ADMIN | 2 min ago | |
| tomas | OPERATOR | yesterday | |
| support | VIEWER | 3 days ago |
What each role may do
| VIEWER | OPERATOR | ADMIN | |
|---|---|---|---|
| Browse queues, payloads, history, audit | ● | ● | ● |
| Replay, edit payload and headers | – | ● | ● |
| Alert rules and channels | – | – | ● |
| Users and roles | – | – | ● |
New user
Also in the box
- Edit before replay. Fix a field, drop the retry counter and stack trace, send it. Marked as edited in the audit log.
- Payloads you can read. JSON as a tree that folds, or as a table of field paths. PDFs, images, CSV and JSON sent as base64 show as files: open, download, or preview a CSV as a table. Dates say how long ago, Java class names shrink to their short name.
- Personal data stays out of sight. Passwords, tokens, IBANs, e-mail addresses and card numbers are hidden in the message view, in payloads, headers and exception texts. Operators can show one message's values; replays and downloads keep the real data. In Community every user is an admin and may show them; who may is a matter of roles, in Team and Pro.
- Notes for the team. "Known bug, fix in 4.2, do not replay" on a group of dead letters: everyone sees it on the group, on each message, and in the replay dialog.
- Before the disk runs full. Free disk and memory of every broker node on the overview, the queues taking up the space next to them, and an alert before RabbitMQ blocks your publishers.
- When is it empty? The queue list and every queue page say "empty in ≈ 14 min" at the current pace, or that a queue grows. During a replay it tells you when the replay is done.
- Compare and filter by value. Select messages, and Warren lists only the fields that differ, with how many messages carry each value. Any value becomes the search in one click.
- Spring and MassTransit understood. Next to the broker's x-death, Warren reads Spring's exception headers and MassTransit's _error and _skipped queues: the exception, the consumer, the retries, and a replay back to the endpoint.
- Search 200 dead letters. Payload, ids, routing key and fingerprint in one field.
- Park the poison messages. One click moves them to
<queue>.parking, out of the way of replays and rules, with their death history intact. Replay them to the original route once the fix is out. - Discard or purge with a reason. Operators only; the reason lands in the audit log.
- Exactly the messages you saw. "The first 500" or "every match" is checked first: Warren freezes those messages and shows how many, and the confirm takes exactly them. What fails in between stays in the queue; a purge stops if the queue grew.
- An audit log you can search. Filter by action, queue, user or rule and time range, and share the filtered view as a link. In Pro: export it as CSV and forward every entry to your SIEM.
- Four eyes on what cannot be undone. In Pro a replay, discard or purge can wait for a second operator. Both names land in the audit log.
- Export, import, publish. NDJSON out before a purge, back in later or on staging, or one test message by hand.
- Several clusters, one Warren. Every broker and vhost in one switcher, and one overview of all of them that you can narrow to the clusters you care about; SSO through your identity provider in Pro.
- Keyboard all the way. j k x r, ⌘K for any action, f puts a letter on every button.
Warren reads the whole journey, not just the last stop.
A dead letter usually has a story. RabbitMQ writes it into x-death, the consumer adds retry counters and exception headers, and nobody reads any of it at 3 a.m. Warren turns them into one sentence, a timeline and the exception, and tells you what a replay would actually do.
Replaying this message as-is would send it straight back to the DLQ: the consumer reads x-retry-count: 5 and gives up. Warren points that out, resets it in the replay dialog on request, and records that you did.
| Message | Death | Original route | Dead since | |
|---|---|---|---|---|
| › | order-4708 | expired ×5 …wait.5m | order.created via orders | 12 min ago |
| › | order-4711 | expired ×5 …wait.5m | order.created via orders | 7 min ago |
| › | order-4713 | republished | order.updated via orders | ≥ 2 h |
No message is ever lost. Here is the exact mechanism.
Dedicated channel, confirms on
Warren opens its own AMQP channel with publisher confirms. Nothing is shared with your consumers.
Fetch without acknowledging
Messages are taken with basic.get and left unacknowledged, so each fetch returns the next one. Unselected messages are released back in place.
Publish, then confirm, then ack
A selected message is published with mandatory=true. Only after the broker confirms does Warren acknowledge the source; at full speed a hundred publishes share one wait, each tracked by its sequence number. Unroutable publishes come back as failures.
Release everything else
What was not moved is requeued in its original position. Closing the channel releases the rest, even if Warren crashes mid-way.
Runs next to your broker. Your payloads never leave your network.
One container with an embedded database, or point Pro at your own PostgreSQL. Configure clusters with environment variables or a YAML file, put it behind your ingress, sign in with SSO. Air-gapped installations work: the licence is checked offline.
- Docker image, versioned releases with a changelog, database migrations run on upgrade
- Reads through the management API, replays over AMQP with a least-privilege user
- Metrics and audit stay in your network, embedded or in your Postgres, retention is yours to set; Pro forwards the audit log to your SIEM
- No telemetry, no phone-home, no third-party scripts
A RabbitMQ, Warren and three dead-letter queues with real dead letters. Only Docker needed.
- 1
curl -O https://warrenops.io/docker-compose.try.yml - 2
docker compose -f docker-compose.try.yml up -d - 3Open localhost:8080, sign in as admin / admin, open orders.dlq.
Done? docker compose -f docker-compose.try.yml down -v removes it all. See the file
services: warren: image: ghcr.io/viba88/warren:0.11 ports: ["8080:8080"] volumes: [warren-data:/app/data] environment: WARREN_RABBIT_MANAGEMENT_URL: http://rabbitmq:15672 WARREN_RABBIT_HOST: rabbitmq WARREN_RABBIT_USERNAME: warren WARREN_RABBIT_PASSWORD: ${RABBIT_PASSWORD} WARREN_ADMIN_PASSWORD: ${WARREN_ADMIN_PASSWORD} # Team/Pro: WARREN_LICENSE_KEY # Pro: WARREN_DB_URL, WARREN_OIDC_ISSUER_URI volumes: warren-data:
docker compose up -d, then sign in on port 8080 as admin with the password you set. The RabbitMQ user needs the management tag and read/write on the vhost; add monitoring to see disk and memory of the nodes. Full examples, also with PostgreSQL, are on GitHub.
Free to browse and replay. Pay when Warren runs your on-call.
The Community edition is free forever, no signup, no time limit. Team and Pro are bought once and never expire: twelve months of updates are included, renewing them is up to you. Community is available now. Team and Pro are in development; their prices follow with the release, and the forms below tell you when.
- One broker, up to 3 of its vhosts
- Browse dead letters with full x-death history, breakdown and search
- Replay to the original route, throttled, with payload and header edits
- Park, discard, purge, export, import and publish, every action logged
- Metrics history of the last hour, broker disk and memory
- Audit log, local users, keyboard shortcuts
- Embedded database: one container, one volume
Image ghcr.io/viba88/warren, no account, no key. Start with the demo or next to your own broker.
- Everything in Community
- Up to 3 brokers, any number of vhosts each
- Roles: viewer, operator, admin; viewers never see masked values
- Metrics history, 15 minutes to 7 days
- Alerting to Slack, Teams, PagerDuty, Opsgenie, e-mail, webhooks; quiet hours per channel
- Prometheus metrics for Grafana, with ready-made alert rules
- Replay rules: retry dead letters on a schedule, with attempt limits
- Embedded database, offline licence
Used only to tell you when it ships. Privacy
For a small team on one broker that wants to hear about a full DLQ first.
- Everything in Team
- Unlimited brokers and vhosts, one flat price
- Four-eyes approval: a replay, discard or purge runs once a second operator approves it
- Audit log as CSV, forwarded to your SIEM by webhook or syslog, kept at least as long as you set
- Metrics history up to 90 days
- OIDC single sign-on
- Bring your own PostgreSQL
- Offline licence, also for air-gapped installations
Used only to tell you when it ships. Privacy
Tell us what you would need from Pro. We reply personally.
Questions operators ask first.
Is Warren a replacement for the management UI?
No, and it is not trying to be. The management UI configures the broker and will keep doing that. Warren is for the people who run the traffic on it: reading messages, finding out why they died, alerting, replaying, and proving afterwards what was done. Most teams have both open.
Will Warren ever declare queues or edit policies?
No policies, no exchanges, no bindings, by design. A tool that can replay production messages should not also be able to reshape the broker. Warren reads, publishes and, when an operator asks with a reason, removes messages. The single exception is Park: the first time an operator parks messages from a queue, Warren creates <queue>.parking as a plain durable queue if it does not exist yet. That is what lets you give it to a wider circle than the admin UI.
Does Warren change my broker configuration?
No. It reads queues, bindings and messages through the management API and publishes over AMQP. It never deletes queues, exchanges or bindings; the only queue it creates is a parking queue, when an operator parks messages. The only destructive actions are discarding messages and purging a queue: operators only, with a reason, recorded in the audit log.
What permissions does the RabbitMQ user need?
The management tag plus read and write on the vhosts you configure, and the monitoring tag if Warren should show disk and memory of the nodes. Configure permission is only needed if Warren should create parking queues itself; without it, create <queue>.parking once and Warren uses it as it is.
Does peeking at messages have side effects?
RabbitMQ marks peeked messages as redelivered. That is a property of the broker and is visible to consumers as a flag; the messages stay in place and in order. On RabbitMQ 3.13 a peek of a quorum queue with a delivery limit also counts as a delivery; Warren asks before it reads such a queue.
Will support see customers' personal data?
Not by default. The message view hides values whose field name looks sensitive (password, token, IBAN, e-mail and more, configurable with WARREN_MASKING_FIELDS) and anything that looks like an e-mail address, an IBAN or a card number. Operators and admins can show one message's values, viewers cannot; in Community, where every user is an admin, anyone signed in can. This is display masking: replays, downloads and exports keep the real data, and so does the API for anyone with access.
Which RabbitMQ versions are supported?
3.13 and 4.x, classic and quorum queues. Streams are read-only for now.
Team or Pro?
Team is for a team that runs its own brokers: up to three, with any number of vhosts each, roles so support can look while on-call acts, history, alerts, and rules that retry dead letters by themselves. Pro is for a company that has to show what happened: any number of brokers, four-eyes approval for replays, discards and purges, the audit log as CSV and in your SIEM, 90 days of history, sign-in through your identity provider, and your own PostgreSQL. A Team licence upgrades to Pro in place; nothing is reinstalled. Both are priced per installation: one running Warren, however many brokers and vhosts it watches, so prod, staging and dev on one Warren count once.
Is Team or Pro a subscription?
No. You pay once and the licence never expires. It includes twelve months of updates: every Warren version released in that time runs with your key for good. After that you can renew updates for another year, or keep the version you have for as long as you like. A version released after your update date runs as Community until you install a renewed key; Warren shows the date a few weeks ahead. Nothing switches off because a payment was missed, and the key is checked offline.
Is Warren open source?
Not at the moment. Warren is distributed as a ready-to-run container image under a commercial licence; the Community edition is free to use for any purpose, without registration or time limit. The source may open up later; if that matters to you, tell us. The public repository on GitHub holds the README, the compose files, the changelog, and the issue tracker.
Where is my data?
In your network. Warren stores replay audit entries, queue metrics and users in its embedded database on a volume you own, or in your own PostgreSQL with Pro. The audit log keeps the first 16 KiB of each moved message by default; WARREN_AUDIT_PAYLOAD_BYTES sets how much.
Your first replay is three commands away.
Download the compose file, start it, open localhost:8080. A RabbitMQ with real dead letters is included, so there is nothing to prepare. Free, no account.
Team and Pro follow. Leave your address in the pricing section to hear when they ship.