Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
Servers, nginx and Docker

Creating a Kafka topic against a Docker broker times out waiting for a node assignment

ErrorTimed out waiting for a node assignment

Short answer

The client could not open a working connection to any broker. With Kafka in Docker this is almost always a listener problem: the client connects to bootstrap.servers, then follows the advertised.listeners address the broker returns, and that address is not reachable from where the client runs. Give the broker one listener per network and point each client at the matching one.

  • Applies to Apache Kafka 4.1, apache/kafka Docker image, Docker Engine 28
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Print the listener settings the broker container is running with, then compare them with the address your client uses.

    docker inspect my-kafka --format '{{range .Config.Env}}{{println .}}{{end}}' | grep -i listener
    docker port my-kafka
  2. 2Match what you see, then jump to its fix

    1. 1The client reaches the port but still times out; advertised.listeners shows localhost, an internal container name or an IP of another networkadvertised.listeners points to an address the client cannot reachFix
    2. 2The first connection already fails: nothing listens on the address in bootstrap.serversThe client uses the wrong bootstrap address or portFix

What does the error look like?

  • Timed out waiting for a node assignment

Cause 1 of 2

advertised.listeners points to an address the client cannot reach

1Check

advertised.listeners is the set of addresses the broker tells clients and other brokers to use. The client first connects to bootstrap.servers, then switches to what the broker advertises. A client on your host cannot resolve a container name, and a client in another container cannot use localhost.

docker inspect my-kafka --format '{{range .Config.Env}}{{println .}}{{end}}' | grep ADVERTISED

2Fix

Create one listener for containers and one for the host. In the apache/kafka image, set the options as environment variables with the KAFKA_ prefix, with dots replaced by underscores, and publish the host port.

environment:
  KAFKA_LISTENERS: PLAINTEXT://:19092,PLAINTEXT_HOST://:9092
  KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker-1:19092,PLAINTEXT_HOST://localhost:29092
  KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
  KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
ports:
  - "29092:9092"

3Verify

From the host use localhost:29092; from another container use broker-1:19092.

docker compose up -d --force-recreate
docker compose logs --tail 20

Confirmed by hub.docker.com

Did this fix it?

Cause 2 of 2

The client uses the wrong bootstrap address or port

1Check

bootstrap.servers is the host and port list used for the first connection. If the port is not published, or you use the container-side port from the host, the client never reaches a broker. Check what is published.

docker port my-kafka

2Fix

Use the published host port when the client runs on the host, and the container name with the internal listener port when it runs in another container. With a single broker and a localhost listener, publish 9092 and use localhost:9092 from the host.

# client on the host
--bootstrap-server localhost:29092
# client in another container
--bootstrap-server broker-1:19092

3Verify

Run the topic command again from the same place the client lives; it lists topics instead of timing out.

kafka-topics.sh --bootstrap-server localhost:29092 --list

Confirmed by kafka.apache.org

Did this fix it?

How do I stop it coming back?

  • Never advertise 0.0.0.0: Kafka does not allow it in advertised.listeners, only in listeners.
  • Name your listeners after the network they serve (internal, host) so the compose file shows which client uses which.
  • Test the bootstrap address from the same place the application runs, not only from your laptop.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.