bullmq-specialist

BullMQ expert for Redis-backed job queues, background processing, and reliable async execution in Node.js/TypeScript applications.

By sickn33 · 1,904 installs

npx skills add sickn33/agentic-awesome-skills --skill bullmq-specialist

Source repository · Upstream listing

BullMQ Specialist BullMQ expert for Redis backed job queues, background processing, and reliable async execution in Node.js/TypeScript applications. Principles Jobs are fire and forget from the producer side let the queue handle delivery Always set explicit job options defaults rarely match your use case Idempotency is your responsibility jobs may run more than once Backoff strategies prevent thundering herds exponential beats linear Dead letter queues are not optional failed jobs need a home Concurrency limits protect downstream services start conservative Job data should be small pass IDs, not payloads Graceful shutdown prevents orphaned jobs handle SIGTERM properly Capabilities bullmq queues job scheduling delayed jobs repeatable jobs job priorities rate limiting jobs job events worker patterns flow producers job dependencies Scope redis infrastructure redis specialist serverless queues upstash qstash workflow orchestration temporal craftsman event sourcing event architect email delivery email systems Tooling Core bullmq ioredis Hosting upstash redis cloud elasticache railway Monitoring bull board arena bullmq pro Patterns delayed jobs repeatable jobs job flows rate limiting sandboxed processors Patterns Basic Queue Setup Production ready BullMQ queue with proper configuration When to use : Starting any new queue implementation import { Queue, Worker, QueueEvents } from 'bullmq'; import IORedis from 'ioredis'; // Shared connection for all queues const connection = new IORedis(process.env.REDIS URL, { maxRetriesPerRequest: null, // Required for BullMQ enableReadyCheck: false, }); // Create queue with sensible defaults const emailQueue = new Queue('emails', { connection, defaultJobOptions: { attempts: 3, backoff: { type: 'exponential', delay: 1000, }, removeOnComplete: { count: 1000 }, removeOnFail: { count: 5000 }, }, }); // Worker with concurrency limit const worker = new Worker('emails', async (job) = { await sendEmail(job.data); }, { connection, concurrency: 5, limiter: { max: 100, duration: 60000, // 100 jobs per minute }, }); // Handle events worker.on('failed', (job, err) = { console.error( Job ${job?.id} failed: , err); }); Delayed and Scheduled Jobs Jobs that run at specific times or after delays When to use : Scheduling future tasks, reminders, or timed actions // Delayed job runs once after delay await queue.add('reminder', { userId: 123 }, { delay: 24 60 60 1000, // 24 hours }); // Repeatable job runs on schedule await queue.add('daily digest', { type: 'summary' }, { repeat: { pattern: '0 9 ', // Every day at 9am tz: 'America/New York', }, }); // Remove repeatable job await queue.removeRepeatable('daily digest', { pattern: '0 9 ', tz: 'America/New York', }); Job Flows and Dependencies Complex multi step job processing with parent child relationships When to use : Jobs depend on other jobs completing first import { FlowProducer } from 'bullmq'; const flowProducer = new FlowProducer({ connection }); // Parent waits for all children to complete await flowProducer.add({ name: 'process order', queueName: 'orders', data: { orderId: 123 }, children: [ { name: 'validate inventory', queueName: 'inventory', data: { orderId: 123 }, }, { name: 'charge payment', queueName: 'payments', data: { orderId: 123 }, }, { name: 'notify warehouse', queueName: 'notifications', data: { orderId: 123 }, }, ], }); Graceful Shutdown Properly close workers without losing jobs When to use : Deploying or restarting workers const shutdown = async () = { console.log('Shutting down gracefully...'); // Stop accepting new jobs await worker.pause(); // Wait for current jobs to finish (with timeout) await worker.close(); // Close queue connection await queue.close(); process.exit(0); }; process.on('SIGTERM', shutdown); process.on('SIGINT', shutdown); Bull Board Dashboard Visual monitoring for BullMQ queues When to use : Need visibility into queue status and job states import { createBullBoard } from '@bull board/api'; import { BullMQAdapter } from '@bull board/api/bullMQAdapter'; import { ExpressAdapter } from '@bull board/express'; const serverAdapter = new ExpressAdapter(); serverAdapter.setBasePath('/admin/queues'); createBullBoard({ queues: [ new BullMQAdapter(emailQueue), new BullMQAdapter(orderQueue), ], serverAdapter, }); app.use('/admin/queues', serverAdapter.getRouter()); Validation Checks Redis connection missing maxRetriesPerRequest Severity: ERROR BullMQ requires maxRetriesPerRequest null for proper reconnection handling Message: BullMQ queue/worker created without maxRetriesPerRequest: null on Redis connection. This will cause workers to stop on Redis connection issues. No stalled job event handler Severity: WARNING Workers should handle stalled events to detect crashed workers Message: Worker created without 'stalled' event handler. Stalled jobs indicate worker crashes and should be monitored. No failed job event handler Severity: WARNING Workers should handle failed events for monitoring and alerting Message: Worker created without 'failed' event handler. Failed jobs should be logged and monitored. No graceful shutdown handling Severity: WARNING Workers should gracefully shut down on SIGTERM/SIGINT Message: Worker file without graceful shutdown handling. Jobs may be orphaned on deployment. Awaiting queue.add in request handler Severity: INFO Queue additions should be fire and forget in request handlers Message: Queue.add awaited in request handler. Consider fire and forget for faster response. Potentially large data in job payload Severity: WARNING Job data should be small pass IDs not full objects Message: Job appears to have large inline data. Pass IDs instead of full objects to keep Redis memory low. Job without timeout configuration Severity: INFO Jobs should have timeouts to prevent infinite execution Message: Job added without explicit timeout. Consider adding timeout to prevent stuck jobs. Retry without backoff strategy Severity: WARNING Retries should use exponential backoff to avoid thundering herd Message: Job has retry attempts but no backoff strategy. Use exponential backoff to prevent thundering herd. Repeatable job without explicit timezone Severity: WARNING Repeatable jobs should specify timezone to avoid DST issues Message: Repeatable job without explicit timezone. Will use server local time which can drift with DST. Potentially high worker concurrency Severity: INFO High concurrency can overwhelm downstream services Message: Worker concurrency is high. Ensure downstream services can handle this load (DB connections, API rate limits). Collaboration Delegation Triggers redis infrastructure redis cluster memory tuning redis specialist (Queue needs Redis infrastructure) serverless queue edge queue no redis upstash qstash (Need queues without managing Redis) complex workflow saga compensation long running temporal craftsman (Need workflow orchestration beyond simple jobs) event sourcing CQRS event streaming event architect (Need event driven architecture) deploy kubernetes scaling infrastructure devops (Queue needs infrastructure) monitor metrics alerting dashboard performance hunter (Queue needs monitoring) Email Queue Stack Skills: bullmq specialist, email systems, redis specialist Workflow: Background Processing Stack Skills: bullmq specialist, backend, devops Workflow: AI Processing Pipeline Skills: bullmq specialist, ai workflow automation, performance hunter Workflow: Scheduled Tasks Stack Skills: bullmq specialist, backend, redis specialist Workflow: Related Skills Works well with: redis specialist , backend , nextjs app router , email systems , ai workflow automation , performance hunter When to Use User mentions or implies: bullmq User mentions or implies: bull queue User mentions or implies: redis queue User mentions or implies: background job User mentions or implies: job queue User mentions or implies: delayed job User mentions or implies: repeatable job User mentions or implies: worker process User mentions or implies: job scheduling User mentions or implies: async processing Limitations Use this skill only when the task clearly matches the scope described above. Do not treat the output as a substitute for environment specific validation, testing, or expert review. Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.