Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
Node.js, PHP and APIs

Celery in a Django project does not connect to Redis or ignores the broker URL

ErrorModuleNotFoundError: No module named 'redis'

Short answer

Celery talks to Redis through a client library and reads its broker URL from your Django settings. It fails when the redis extra is not installed, when the setting is not prefixed to match the namespace, when the URL points at the wrong host, or when Redis evicts Celery's keys. Check each in that order.

  • Applies to Celery 5.5, Redis 7, Django 5.2, Python 3.12
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Ask Celery which broker it is really using. The first lines of the worker output show the transport and URL.

    celery -A proj worker -l INFO
  2. 2Match what you see, then jump to its fix

    1. 1ModuleNotFoundError: No module named 'redis'Celery was installed without the Redis supportFix
    2. 2The worker banner shows an amqp:// transport instead of redis://The settings are not read, so Celery falls back to its default brokerFix
    3. 3Connection errors from the worker while the redis CLI works from another shell or containerThe URL points at the wrong placeFix
    4. 4InconsistencyError mentioning _kombu.binding.celeryRedis is evicting Celery's keysFix

What does the error look like?

  • ModuleNotFoundError: No module named 'redis'
  • kombu.exceptions.InconsistencyError: ... _kombu.binding.celery
  • The worker banner shows an amqp:// transport, Celery's default, instead of your Redis URL

Cause 1 of 4

Celery was installed without the Redis support

1Check

Redis support comes from the redis extra. Plain celery does not install the Redis client.

pip show redis

2Fix

Install Celery with the extra.

pip install -U "celery[redis]"

3Verify

The import now works in the same environment as the worker.

python -c "import redis; print(redis.__version__)"

Confirmed by docs.celeryq.dev

Did this fix it?

Cause 2 of 4

The settings are not read, so Celery falls back to its default broker

1Check

With config_from_object('django.conf:settings', namespace='CELERY'), every Celery option must be upper case and start with CELERY_. So broker_url becomes CELERY_BROKER_URL; a plain BROKER_URL in settings.py is ignored. Also check that proj/__init__.py imports the app.

grep -n -E "CELERY_|config_from_object" proj/settings.py proj/celery.py

2Fix

Use the namespaced name, and import the app in the package's __init__.py so it loads when Django starts.

# proj/settings.py
CELERY_BROKER_URL = "redis://localhost:6379/0"

# proj/__init__.py
from .celery import app as celery_app
__all__ = ("celery_app",)

3Verify

Start the worker from the project root and read the banner.

celery -A proj worker -l INFO

Confirmed by docs.celeryq.dev

Did this fix it?

Cause 3 of 4

The URL points at the wrong place

1Check

The URL form is redis://:password@hostname:port/db_number. Missing parts default to localhost, port 6379 and database 0, which is wrong in a container or a managed Redis.

redis-cli -u redis://HOST:6379/0 ping

2Fix

Write the full URL, with the password if one is set, and use the service name in Docker.

CELERY_BROKER_URL = "redis://:mypassword@redis:6379/0"

3Verify

Restart the worker and send a task.

python manage.py shell -c "from proj.tasks import add; print(add.delay(2, 3).id)"

Confirmed by docs.celeryq.dev

Did this fix it?

Cause 4 of 4

Redis is evicting Celery's keys

1Check

Celery's docs say this can happen when Redis evicts keys under memory pressure. Look at the policy.

redis-cli CONFIG GET maxmemory-policy

2Fix

Set maxmemory and a maxmemory-policy of noeviction or allkeys-lru for the Redis instance used as a broker.

redis-cli CONFIG SET maxmemory-policy noeviction

3Verify

Read the setting back.

redis-cli CONFIG GET maxmemory-policy

Confirmed by docs.celeryq.dev

Did this fix it?

How do I stop it coming back?

  • Give Celery its own Redis database or instance, with a policy that does not evict its keys.
  • Print the broker URL (without the password) in your startup log.
  • Keep the broker URL in an environment variable so containers and laptops differ only by config.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Back end

npm install fails inside a node:alpine Docker build

Docker Node Alpine Image Build Fails on node-gyp

The node alpine images ship almost nothing beyond Node.js, so a native module that node-gyp must compile has no Python, make or C++ compiler.

2 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.