Skip to content

Operations / Field guide

How to monitor BullMQ in production

A worker can be alive while its queue makes no useful progress. Effective monitoring connects the queue inventory to finished-job trends, failure evidence, Redis capacity, and an action the on-call engineer can take. This guide covers Redis-backed BullMQ queues, which are the queues Durabull currently inspects.

By DurabullPublished 4 min read11 primary sources
The short answer

Monitor customer waiting time, processing progress, failures, and Redis health together. A queue count alone cannot tell you whether work is on time.

The production feedback loop
  1. 01
    Observe

    Queue age, failures, completion rate, worker availability, and Redis health.

  2. 02
    Investigate

    Inspect a representative job, its logs, stack trace, and dependency health.

  3. 03
    Recover & verify

    Fix the cause, retry only safe work, then verify completions and side effects.

A count is a signal. A recovery is complete only when the workload is healthy again.

1. Establish what should be running

Start with an inventory: environment, Redis endpoint and logical database, queue prefix, queue name, owning service, and expected workload. Include quiet queues and periodic work. An empty overnight queue may be healthy; a daily report queue that never receives its expected job may represent an incident.

Durabull discovers queue metadata in Redis using the configured prefix. Match that prefix to the producer and worker configuration, and check network access before interpreting an empty list. Discovery covers the connected database; it is not proof that every service or environment has been connected.

BullMQ v6 also ships an optional PostgreSQL backend. This runbook covers Redis only, and Durabull does not currently support PostgreSQL-backed queues. Confirm the backend and library version before selecting a dashboard.

2. Enable historical metrics deliberately

BullMQ native throughput history records completed and failed jobs in minute buckets. Enable metrics.maxDataPoints on every worker for a queue, using the same retention setting. The example retains one week. Replace the sample processor with your real handler; creating an extra worker in production changes who consumes jobs.

Live state counts and historical throughput answer different questions. A failed-set count includes retained jobs; it is not the number of failures in the last five minutes. Native getMetrics() history also differs from Prometheus state-count export, which can expose current counts without that history. Do not treat missing historical data as zero failures.

import { MetricsTime, Worker } from 'bullmq'

const worker = new Worker('emails', async (job) => {
  await job.log('Starting email delivery')
  await deliverEmail(job.data) // Your application handler
}, {
  connection: { host: '127.0.0.1', port: 6379 },
  metrics: { maxDataPoints: MetricsTime.ONE_WEEK },
})

worker.on('error', (error) => console.error(error))
Add metrics to your existing worker configuration.

3. Diagnose backlog before adding workers

Choose a customer-facing target, such as transactional mail beginning within a minute. Track the age of the oldest waiting job alongside waiting, active, delayed, and completed counts. A large scheduled workload is different from ready work that cannot start. Record the observation window so a brief traffic spike is not confused with persistent overload.

Use a short decision sequence: confirm the queue is not intentionally paused; inspect worker availability; check rate limits and concurrency; inspect representative active and failed jobs; then compare Redis health with application logs. If completions continue but arrivals are faster, capacity may be the issue. If downstream requests time out, adding workers can increase pressure on the dependency.

For repeated stalls, inspect CPU-heavy processing and event-loop blocking. A worker must keep renewing its job lock; a stalled job returns to waiting and is processed again, and moves to failed once it exceeds the stall limit. Close workers with await worker.close() on SIGTERM so routine deploys do not create stalls. Increasing a timeout without finding the cause can hide the symptom while extending recovery time.

4. Alert on actionable conditions and missing evidence

Pair a backlog or failure alert with an owner, a runbook, and a relevant environment. Start with thresholds based on your workload, observe normal peaks, then tune. An example policy is sustained waiting work with no completions during an expected processing window. Durabull’s queue-stalled rule implements that pattern, but the window is yours to choose and validate; there is no universal threshold.

BullMQ requires Redis maxmemory-policy set to noeviction; evicting BullMQ keys can break queue state, so treat any key eviction as an incident. Durabull Redis health rules evaluate memory, client capacity, eviction rates, and other INFO-derived signals. Missing INFO fields remain unavailable; they do not prove recovery. Percentage memory alerts need an explicit Redis maxmemory limit; use an absolute allocation threshold when that denominator is unavailable.

Test the entire notification path with a controlled staging failure. Verify the recipient, delivery result, incident resolution, and what happens when monitoring itself stops. A configured destination is not evidence that somebody receives the alert.

5. Close the loop with controlled recovery

Before a retry, identify the failed side effect and check whether it may already have succeeded. Payment, email, and provisioning jobs need business-level idempotency. Fix the cause, retry one representative job, and inspect its terminal result before widening recovery.

Retain enough failed-job evidence to investigate, while bounding completed and failed history to your storage budget. Record a recovery note with the original failure, deployed fix, affected job IDs, and observed outcome. A job returning to waiting means it was re-enqueued; only a later result establishes that processing succeeded.

Common questions

Do I need to change workers to use Durabull?

Queue and job inspection connects to Redis without a worker SDK. Historical completion charts need BullMQ metrics enabled, and job logs require application calls to job.log().

Why are my BullMQ charts empty?

Check worker metrics configuration, consistent retention across workers, recent completed or failed work, and the selected Redis connection and prefix. An empty history is not necessarily zero traffic.

Does a larger failed count mean the current failure rate increased?

Not necessarily. The failed set retains jobs according to your cleanup policy. Use a defined observation window and failure events or native metric buckets to assess recent failures.

Sources & review

Written by Durabull. Reviewed against the references below and Durabull’s implementation. Pricing examples are documented models, not measured benchmarks or vendor quotes.

Suggest a correction if a feature, price, or documented behavior has changed.

Compare the options

Keep reading

Connect Redis. Queues appear.

Inspect your Redis-backed BullMQ queues in one place. Hosted Durabull is free to start.