Help for owners
Emails, uploads and other background tasks are piling up and not finishing
Short answer
Apps hand slow work such as emails, imports and image processing to a background queue and a worker that runs it. When the worker stops, crashes or is too slow, jobs pile up. Do not restart servers or clear the queue yourself, since that can delete jobs. Tell your developer what is waiting and since when.
Does this sound like your situation?
- Emails, notifications or receipts arrive hours late or never.
- Uploaded files stay in processing.
- Reports and exports never finish.
- It started after an update, a deploy or a server move.
- My developer says the queue is backed up.
Why does this happen?
The worker process stopped
Queue workers are long-running programs. Laravel's documentation says a worker that exits is typically restarted by a process manager configured on the server. Without one, a single crash or reboot leaves the queue unprocessed.
Confirmed by laravel.com
Workers keep running old code after an update
A long-running worker loads the app once. After a deploy it can keep running the old version, or fail on jobs that were created by the new one, until it is restarted on purpose.
A job fails and is retried or given up
A job that keeps failing is attempted again up to its limit and then marked failed. Laravel's documentation notes that a job exceeding its maximum attempts is considered failed. If the limit is set to retry forever, one bad job can keep the line busy.
Confirmed by laravel.com
The queue service itself is down or full
The queue lives in a database or in a service like Redis. If it is out of memory or unreachable, jobs cannot be added or fetched.
Technical detail: node-redis throws The client is closed when you send a command
Too much work for too few workers
A busy period, a big import or a slow outside service can add jobs faster than they finish, so the line only grows.
Technical detail: Celery in a Django project does not connect to Redis or ignores the broker URL
What can I do about it today?
- Write down what is waitingNote what kind of work is delayed, how many items you know of and when it last worked normally.
- Do not clear or purge the queueClearing it deletes customer emails, orders and uploads for good. Only a developer who has checked what is in it should touch it.
- Do not resubmit the same itemsUploading again or resending creates duplicates when the original jobs finally run.
- Ask whether a deploy or server change happenedA recent update or reboot is the most common starting point. Send the date to your developer.
- Tell customers about the delayA short message that emails or processing are delayed prevents repeat requests.
When should I bring someone in?
- The delayed jobs include payments, orders or confirmations.
- The pile keeps growing after a restart.
- Nobody knows how the queue is run on the server.
How do I handle it?
- Look before touchingI check whether the worker is alive, what is in the queue and what the failed jobs say, without deleting anything.
- Find why work stopped or slowedA crashed worker, old code, a failing job type or a full queue service: the logs show which.
- Restore processing and drain safelyI fix the cause, start workers under a process manager, and let the backlog drain at a pace the system can take, retrying failed jobs only when it is safe.
- Prevent a repeatWorkers restart themselves, deploys restart them on purpose, and an alert fires when the queue grows or a worker is missing.
The price depends on what I find, so I quote in writing after a short discovery call. Tell me what is happening and I reply within 24 hours.
What do owners ask about this?
Are the waiting jobs lost?
Usually not. They sit in the queue until a worker takes them, which is why clearing the queue is the one thing to avoid.
Why did the problem show up only now?
A stopped worker can go unnoticed until the delayed work is something visible, like an email a customer expected.
Is this about tasks that run on a schedule?
No, that is a separate problem, described on the scheduled tasks page. A queue holds work that was triggered by a user action.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Found a mistake? Tell me.
For your developer: the technical fixes
ModuleNotFoundError: No module named 'redis'Celery in a Django project does not connect to Redis or ignores the broker URLClientClosedError: The client is closednode-redis throws The client is closed when you send a commandError: connect ECONNREFUSED 127.0.0.1:6379Node.js cannot reach Redis: ECONNREFUSED on port 6379cache.set returns without error but redis-cli KEYS * is emptyNestJS cache.set works but Redis shows no keys