bullmq users: how do you deal with stalled jobs and retries on expensive jobs?
i run bullmq for background jobs that call llm apis (for content generation) and i keep hitting the same two things: stalled jobs, and retries that already cost money.
how are you handling it? custom dedupe keys? saving progress between steps? just eating the cost?
not selling anything, no library to plug. just want to hear real setups.
- last stalled / double-run job you had, what happened
- how you stop a retry from redoing finished expensive work
- anything you built around bullmq to fix it
trying to figure out what works best for this in my own prod
0
6 comments
by migrating to trigger.dev
Not sure why downvoted, my other advice would be to consider a workflow service or library instead. There are a lot, Temporal, Trigger.dev, etc.
what were you on before and what made u move? how long did the migration take?
bullmq was the initial queuing solution on this project. We wanted something with more visibility and scalability. Chose trigger.dev and migrated it in the span of like 10 days last year at the company I work for. We would crash and burn by now if we had stayed on bullmq. Background jobs are the heart and soul of this company
10 days is quickkkk. what couldn't you see in bullmq that you can now? and is there anything you still can't see or wish trigger.dev did differently? would be great to hear more in a quick chat if you're open to it
yeah those were the days. Today I would have the PR ready in 30 min with opus 5.5 and deployed in next 10 min.