Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
Databases and Redis

psycopg2 fails with: server closed the connection unexpectedly

Errorpsycopg2.OperationalError: server closed the connection unexpectedly

Short answer

The client was talking to Postgres and the connection vanished. The three causes with a documented fix: an idle connection cut by something on the network path, one connection shared between forked worker processes, or the database container killed for using too much memory.

  • Applies to psycopg2 2.9, PostgreSQL 17, Docker Engine 28, Docker Compose 2
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Check whether the database container itself restarted around the time of the error. If its uptime is short, go to the third cause; if not, read on.

    docker ps -a
    docker compose logs db --tail 50
  2. 2Match what you see, then jump to its fix

    1. 1The error shows up on the first query after the app has been quiet for a while, and later queries work.An idle connection was dropped somewhere between app and databaseFix
    2. 2The error appears under gunicorn, Celery or multiprocessing workers, often soon after startup, and not with a single process.One connection is shared by several forked processesFix
    3. 3docker ps -a shows the db container restarted recently, and the error hit every open connection at the same moment.The database container was killed for using too much memoryFix

What does the error look like?

  • psycopg2.OperationalError: server closed the connection unexpectedly
  • OperationalError: server closed the connection unexpectedly (psycopg2, Flask, SQLAlchemy)

Cause 1 of 3

An idle connection was dropped somewhere between app and database

1Check

A connection that sits idle can be discarded by a NAT, firewall or VM network layer. libpq can send TCP keepalives to prevent it: keepalives is on by default, but the idle time uses the system default unless you set it.

grep -rn "keepalives" . --include=*.py

2Fix

Set explicit keepalive values in the connection, so traffic flows before the idle drop happens. Also retry once on this error in code that holds long-lived connections.

import psycopg2

conn = psycopg2.connect(
    "postgresql://app:secret@db:5432/appdb",
    keepalives=1,
    keepalives_idle=30,
    keepalives_interval=10,
    keepalives_count=5,
)

3Verify

Leave the app idle for longer than before and repeat a query.

python -c "import psycopg2; c=psycopg2.connect('postgresql://app:secret@db:5432/appdb', keepalives_idle=30); print(c.closed)"

Confirmed by postgresql.org

Did this fix it?

Cause 2 of 3

One connection is shared by several forked processes

1Check

Psycopg warns that libpq connections should not be used by forked processes. A connection opened before the fork is then used by every child at once.

grep -rn "connect(" . --include=*.py

2Fix

Open the connection after the fork, inside each worker, or per request. Do not create it at import time in a module that is loaded before workers start.

def get_conn():
    # called inside the worker or request, never at import time
    return psycopg2.connect(DATABASE_URL)

3Verify

Run the app with several workers and send parallel requests.

docker compose logs web --tail 30

Confirmed by psycopg.org

Did this fix it?

Cause 3 of 3

The database container was killed for using too much memory

1Check

When the host runs low on memory the kernel starts killing processes, and a container is more likely to be chosen. A --memory limit on the container sets a ceiling for it.

docker ps -a
docker compose logs db --tail 50

2Fix

Give the container more memory (raise or remove its limit, or free memory on the host) and reduce what each query asks for.

# plain docker: raise the -m limit of the container
docker run -d --name db -m 2g -e POSTGRES_PASSWORD=example-password postgres:17

3Verify

The container stays up through the workload that used to kill it.

docker ps

Confirmed by docs.docker.com

Did this fix it?

How do I stop it coming back?

  • Create database connections inside workers or per request, not at import time.
  • Set TCP keepalive parameters for any app that keeps connections open for hours.
  • Wrap long-lived connections with a reconnect-and-retry on OperationalError.
  • Give the database container a memory ceiling that fits the host and watch it.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.