Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
Servers, nginx and Docker

Prometheus container cannot write to its data folder

Errorerr="open /prometheus/queries.active: permission denied"

Short answer

The prom/prometheus image runs as the user nobody and writes to /prometheus. If you mount a host folder there that nobody cannot write to, Prometheus stops with permission denied on queries.active. Use a named volume, or give the folder to the user the container runs as. On SELinux hosts add the :z label.

  • Applies to Docker Engine 27, Docker Compose 2, prom/prometheus image
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    See who owns the host folder you mounted, and which user the container is told to run as.

    ls -ldn ./prometheus-data
    docker compose config | grep -n -E "user:|prometheus"
  2. 2Match what you see, then jump to its fix

    1. 1The failing path is under /prometheus and your compose file bind-mounts a host directory there.The mounted host folder belongs to another userFix
    2. 2You need the data in a host folder you control, for backups or disk layout.You must keep the bind mount, but the container user cannot write to itFix
    3. 3Ownership looks right and the error stays, on Fedora, RHEL, Rocky or Alma.SELinux blocks the bind mountFix

What does the error look like?

  • err="open /prometheus/queries.active: permission denied"
  • open /prometheus/queries.active: permission denied

Cause 1 of 3

The mounted host folder belongs to another user

1Check

The image's Dockerfile switches to USER nobody and sets /prometheus as the data volume. A bind mount replaces that folder with your host folder, which keeps its own owner (often root or your login user).

ls -ldn ./prometheus-data

2Fix

Let Docker manage the data with a named volume. The Prometheus installation guide recommends one for persistent storage.

services:
  prometheus:
    image: prom/prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus-data:/prometheus
volumes:
  prometheus-data:

3Verify

Recreate the container and read its log.

docker compose up -d --force-recreate prometheus
docker compose logs --tail 20 prometheus

Confirmed by raw.githubusercontent.com

Did this fix it?

Cause 2 of 3

You must keep the bind mount, but the container user cannot write to it

1Check

The compose service attribute user overrides the user the image defines, so you can run the container as the owner of the folder.

id -u && id -g
ls -ldn ./prometheus-data

2Fix

Run the service as the numeric owner of the host folder (the output of id above), or change the folder's owner to match.

services:
  prometheus:
    image: prom/prometheus
    user: "1000:1000"   # replace with the owner of ./prometheus-data
    volumes:
      - ./prometheus-data:/prometheus

3Verify

Recreate and confirm the log has no permission error.

docker compose up -d --force-recreate prometheus
docker compose logs --tail 20 prometheus

Confirmed by docs.docker.com

Did this fix it?

Cause 3 of 3

SELinux blocks the bind mount

1Check

Docker's bind mount docs describe the z and Z options, which relabel the host path so the container may use it. Without a label SELinux can refuse access.

getenforce

2Fix

Add :z to the mount. It relabels the host folder, so use it only on a folder made for this.

volumes:
  - ./prometheus-data:/prometheus:z

3Verify

Recreate the container and check the log.

docker compose up -d --force-recreate prometheus && docker compose logs --tail 20 prometheus

Confirmed by docs.docker.com

Did this fix it?

How do I stop it coming back?

  • Prefer named volumes for databases and time-series data; use bind mounts for config files.
  • When you pick a bind mount for data, set the folder owner before the first start.
  • Never relabel system folders such as /home with :Z; use a dedicated data directory.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Servers

A container cannot read or write a mounted host folder

Files in the mounted folder cannot be listed or opened from inside the container

When a bind-mounted host folder is denied inside the container, look at two documented causes first: SELinux on Fedora, RHEL and similar systems blocks the container until the mount carries a :z or :Z label, and a mount flagged ro or readonly can never be written.

2 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.