bullmq-specialist
BullMQ expert for Redis-backed job queues, background processing, and reliable async execution in Node.js/TypeScript applications.
By sickn33 · 1,904 installs
npx skills add sickn33/agentic-awesome-skills --skill bullmq-specialist
Source repository · Upstream listing
BullMQ Specialist
BullMQ expert for Redis backed job queues, background processing, and
reliable async execution in Node.js/TypeScript applications.
Principles
Jobs are fire and forget from the producer side let the queue handle delivery
Always set explicit job options defaults rarely match your use case
Idempotency is your responsibility jobs may run more than once
Backoff strategies prevent thundering herds exponential beats linear
Dead letter queues are not optional failed jobs need a home
Concurrency limits protect downstream services start conservative
Job data should be small pass IDs, not payloads
Graceful shutdown prevents orphaned jobs handle SIGTERM properly
Capabilities
bullmq queues
job scheduling
delayed jobs
repeatable jobs
job priorities
rate limiting jobs
job events
worker patterns
flow producers
job dependencies
Scope
redis infrastructure redis specialist
serverless queues upstash qstash
workflow orchestration temporal craftsman
event sourcing event architect
email delivery email systems
Tooling
Core
bullmq
ioredis
Hosting
upstash
redis cloud
elasticache
railway
Monitoring
bull board
arena
bullmq pro
Patterns
delayed jobs
repeatable jobs
job flows
rate limiting
sandboxed processors
Patterns
Basic Queue Setup
Production ready BullMQ queue with proper configuration
When to use : Starting any new queue implementation
import { Queue, Worker, QueueEvents } from 'bullmq';
import IORedis from 'ioredis';
// Shared connection for all queues
const connection = new IORedis(process.env.REDIS URL, {
maxRetriesPerRequest: null, // Required for BullMQ
enableReadyCheck: false,
});
// Create queue with sensible defaults
const emailQueue = new Queue('emails', {
connection,
defaultJobOptions: {
attempts: 3,
backoff: {
type: 'exponential',
delay: 1000,
},
removeOnComplete: { count: 1000 },
removeOnFail: { count: 5000 },
},
});
// Worker with concurrency limit
const worker = new Worker('emails', async (job) = {
await sendEmail(job.data);
}, {
connection,
concurrency: 5,
limiter: {
max: 100,
duration: 60000, // 100 jobs per minute
},
});
// Handle events
worker.on('failed', (job, err) = {
console.error( Job ${job?.id} failed: , err);
});
Delayed and Scheduled Jobs
Jobs that run at specific times or after delays
When to use : Scheduling future tasks, reminders, or timed actions
// Delayed job runs once after delay
await queue.add('reminder', { userId: 123 }, {
delay: 24 60 60 1000, // 24 hours
});
// Repeatable job runs on schedule
await queue.add('daily digest', { type: 'summary' }, {
repeat: {
pattern: '0 9 ', // Every day at 9am
tz: 'America/New York',
},
});
// Remove repeatable job
await queue.removeRepeatable('daily digest', {
pattern: '0 9 ',
tz: 'America/New York',
});
Job Flows and Dependencies
Complex multi step job processing with parent child relationships
When to use : Jobs depend on other jobs completing first
import { FlowProducer } from 'bullmq';
const flowProducer = new FlowProducer({ connection });
// Parent waits for all children to complete
await flowProducer.add({
name: 'process order',
queueName: 'orders',
data: { orderId: 123 },
children: [
{
name: 'validate inventory',
queueName: 'inventory',
data: { orderId: 123 },
},
{
name: 'charge payment',
queueName: 'payments',
data: { orderId: 123 },
},
{
name: 'notify warehouse',
queueName: 'notifications',
data: { orderId: 123 },
},
],
});
Graceful Shutdown
Properly close workers without losing jobs
When to use : Deploying or restarting workers
const shutdown = async () = {
console.log('Shutting down gracefully...');
// Stop accepting new jobs
await worker.pause();
// Wait for current jobs to finish (with timeout)
await worker.close();
// Close queue connection
await queue.close();
process.exit(0);
};
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
Bull Board Dashboard
Visual monitoring for BullMQ queues
When to use : Need visibility into queue status and job states
import { createBullBoard } from '@bull board/api';
import { BullMQAdapter } from '@bull board/api/bullMQAdapter';
import { ExpressAdapter } from '@bull board/express';
const serverAdapter = new ExpressAdapter();
serverAdapter.setBasePath('/admin/queues');
createBullBoard({
queues: [
new BullMQAdapter(emailQueue),
new BullMQAdapter(orderQueue),
],
serverAdapter,
});
app.use('/admin/queues', serverAdapter.getRouter());
Validation Checks
Redis connection missing maxRetriesPerRequest
Severity: ERROR
BullMQ requires maxRetriesPerRequest null for proper reconnection handling
Message: BullMQ queue/worker created without maxRetriesPerRequest: null on Redis connection. This will cause workers to stop on Redis connection issues.
No stalled job event handler
Severity: WARNING
Workers should handle stalled events to detect crashed workers
Message: Worker created without 'stalled' event handler. Stalled jobs indicate worker crashes and should be monitored.
No failed job event handler
Severity: WARNING
Workers should handle failed events for monitoring and alerting
Message: Worker created without 'failed' event handler. Failed jobs should be logged and monitored.
No graceful shutdown handling
Severity: WARNING
Workers should gracefully shut down on SIGTERM/SIGINT
Message: Worker file without graceful shutdown handling. Jobs may be orphaned on deployment.
Awaiting queue.add in request handler
Severity: INFO
Queue additions should be fire and forget in request handlers
Message: Queue.add awaited in request handler. Consider fire and forget for faster response.
Potentially large data in job payload
Severity: WARNING
Job data should be small pass IDs not full objects
Message: Job appears to have large inline data. Pass IDs instead of full objects to keep Redis memory low.
Job without timeout configuration
Severity: INFO
Jobs should have timeouts to prevent infinite execution
Message: Job added without explicit timeout. Consider adding timeout to prevent stuck jobs.
Retry without backoff strategy
Severity: WARNING
Retries should use exponential backoff to avoid thundering herd
Message: Job has retry attempts but no backoff strategy. Use exponential backoff to prevent thundering herd.
Repeatable job without explicit timezone
Severity: WARNING
Repeatable jobs should specify timezone to avoid DST issues
Message: Repeatable job without explicit timezone. Will use server local time which can drift with DST.
Potentially high worker concurrency
Severity: INFO
High concurrency can overwhelm downstream services
Message: Worker concurrency is high. Ensure downstream services can handle this load (DB connections, API rate limits).
Collaboration
Delegation Triggers
redis infrastructure redis cluster memory tuning redis specialist (Queue needs Redis infrastructure)
serverless queue edge queue no redis upstash qstash (Need queues without managing Redis)
complex workflow saga compensation long running temporal craftsman (Need workflow orchestration beyond simple jobs)
event sourcing CQRS event streaming event architect (Need event driven architecture)
deploy kubernetes scaling infrastructure devops (Queue needs infrastructure)
monitor metrics alerting dashboard performance hunter (Queue needs monitoring)
Email Queue Stack
Skills: bullmq specialist, email systems, redis specialist
Workflow:
Background Processing Stack
Skills: bullmq specialist, backend, devops
Workflow:
AI Processing Pipeline
Skills: bullmq specialist, ai workflow automation, performance hunter
Workflow:
Scheduled Tasks Stack
Skills: bullmq specialist, backend, redis specialist
Workflow:
Related Skills
Works well with: redis specialist , backend , nextjs app router , email systems , ai workflow automation , performance hunter
When to Use
User mentions or implies: bullmq
User mentions or implies: bull queue
User mentions or implies: redis queue
User mentions or implies: background job
User mentions or implies: job queue
User mentions or implies: delayed job
User mentions or implies: repeatable job
User mentions or implies: worker process
User mentions or implies: job scheduling
User mentions or implies: async processing
Limitations
Use this skill only when the task clearly matches the scope described above.
Do not treat the output as a substitute for environment specific validation, testing, or expert review.
Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.