Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
Servers, nginx and Docker

GitLab CI job cannot reach the docker service: 'lookup docker ... no such host'

Errorerror during connect: Post "https://docker:2376/v1.24/auth": dial tcp: lookup docker ... no such host

Short answer

The job's Docker client tries to reach the dind service by the hostname docker, and DNS finds nothing. Either the service never started (usually a runner without privileged mode), or its image name changed the hostname the runner derives. Read the dind service log in the job output first, then fix privileged mode or add the alias.

  • Applies to GitLab Runner 17, Docker executor, docker 24 dind image
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Run docker info as the first command of the job and read the job log above the failure: the dind service writes its own start-up errors there.

    before_script:
      - docker info
  2. 2Match what you see, then jump to its fix

    1. 1The service log shows 'mount: permission denied (are you root?)' and the error names docker:2376The dind service did not start because the runner is not privilegedFix
    2. 2The service is declared as registry.example.com/library/docker:24.0.5-dind and starts fine, but docker is not foundThe service image name includes a registry hostFix
    3. 3Cannot connect to the Docker daemon at tcp://docker:2375, or builds fail with 'cgroups: cgroup mountpoint does not exist'An old dind image, or a TLS setting mismatch with the clientFix
    4. 4Cannot connect to the Docker daemon at unix:///var/run/docker.sockDocker variables are missing in a job that relied on legacy linksFix

What does the error look like?

  • error during connect: Post "https://docker:2376/v1.24/auth": dial tcp: lookup docker ... no such host
  • docker: error during connect: ... lookup docker ... no such host
  • Cannot connect to the Docker daemon at tcp://docker:2375

Cause 1 of 4

The dind service did not start because the runner is not privileged

1Check

Docker-in-Docker needs a runner with privileged mode. Look at the runner's config.toml, and at the tags your job selects.

sudo grep -n -E "privileged|volumes" /etc/gitlab-runner/config.toml

2Fix

Set privileged = true for the runner (and keep the /certs/client volume), and make sure the job uses tags that select that runner.

[runners.docker]
  privileged = true
  volumes = ["/certs/client", "/cache"]

3Verify

Restart the runner, rerun the job, and check that docker info prints server details.

sudo gitlab-runner restart
# in the job
docker info

Confirmed by docs.gitlab.com

Did this fix it?

Cause 2 of 4

The service image name includes a registry host

1Check

The hostname the job uses for the service comes from the image name. A registry hostname in the name changes it, but the Docker client expects docker.

grep -n -A3 "services:" .gitlab-ci.yml

2Fix

Give the service the alias docker.

services:
  - name: registry.example.com/library/docker:24.0.5-dind
    alias: docker

3Verify

Rerun the job: docker info reaches the daemon.

docker info

Confirmed by docs.gitlab.com

Did this fix it?

Cause 3 of 4

An old dind image, or a TLS setting mismatch with the client

1Check

Docker-in-Docker 19.03 and later starts with TLS on (port 2376), so a client pointed at tcp://docker:2375 cannot connect. A dind image older than 20.10 fails on hosts running Docker Engine 20.10 or newer.

grep -n -E "DOCKER_HOST|DOCKER_TLS_CERTDIR|dind" .gitlab-ci.yml

2Fix

Use a current dind image and the documented TLS setup: set DOCKER_TLS_CERTDIR to /certs. For the TLS-enabled Docker executor setup the docs do not set DOCKER_HOST, because Docker 19.03 and later set it automatically.

image: docker:24.0.5-cli
services:
  - docker:24.0.5-dind
variables:
  DOCKER_TLS_CERTDIR: "/certs"

3Verify

docker info in before_script prints the server version.

docker info

Confirmed by docs.gitlab.com

Did this fix it?

Cause 4 of 4

Docker variables are missing in a job that relied on legacy links

1Check

The job has no Docker connection variables. Old setups relied on DOCKER_PORT_2375_TCP from the deprecated --link flag, which stops working when FF_NETWORK_PER_BUILD is true.

grep -n -E "DOCKER_HOST|FF_NETWORK_PER_BUILD" .gitlab-ci.yml

2Fix

Set DOCKER_HOST explicitly, with the TLS variables if TLS is on. For the Kubernetes setup the docs use tcp://docker:2376.

variables:
  DOCKER_HOST: tcp://docker:2376
  DOCKER_TLS_CERTDIR: "/certs"
  DOCKER_TLS_VERIFY: 1
  DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"

3Verify

Rerun the job and run docker info first.

docker info

Confirmed by docs.gitlab.com

Did this fix it?

How do I stop it coming back?

  • Put docker info at the start of every dind job so a daemon problem fails there and not at docker login.
  • Keep the dind image at the same major version as the docker client image.
  • Use runner tags so jobs that need privileged mode only run on runners that allow it.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.