psycopg2 fails with SSL SYSCALL error: EOF detected on a PostgreSQL connection
psycopg2.OperationalError: SSL SYSCALL error: EOF detectedShort answer
The server or something between you and it closed the SSL connection while your code still held it. Pooled connections that sat idle, an idle TCP link dropped by a network device, and a pool shared with a forked worker are the three usual sources. Test connections on checkout, send TCP keepalives, and give each process its own engine.
Fix it fast
1Find out which cause you have
Note when it happens. After a quiet period it is an idle connection (causes 1 and 2). Right after a worker starts, or only under a forking process manager, it is cause 3. Then see which engine options you already set.
grep -rn "create_engine\|pool_pre_ping\|pool_recycle\|keepalives" --include=*.py .2Match what you see, then jump to its fix
- 1
The first query after a quiet period fails, and a retry worksA pooled connection went stale while idleFix - 2
Long queries or long idle gaps fail on a cloud host or behind a NAT or firewall, and nothing is wrong in the server logA network device drops the idle TCP connectionFix - 3
The error shows up in gunicorn, Celery or multiprocessing workers, soon after they start, and the connections were opened before the forkA forked worker is using the parent's connectionFix
- 1
What does the error look like?
psycopg2.OperationalError: SSL SYSCALL error: EOF detectedSSL SYSCALL error: EOF detected
Cause 1 of 3
A pooled connection went stale while idle
1Check
Pools keep connections open between requests. If the server or a proxy closed one meanwhile, the pool hands out a dead one. Look at the engine options: with no pre-ping and no recycle, nothing checks the connection before use.
grep -rn "create_engine" --include=*.py .2Fix
Turn on pool_pre_ping so every checkout tests the connection, and set pool_recycle below the idle limit of the server, proxy or load balancer in front of the database.
from sqlalchemy import create_engine
engine = create_engine(
"postgresql+psycopg2://user:password@db.example.com/mydb",
pool_pre_ping=True,
pool_recycle=1800,
)3Verify
Leave the app idle past your usual gap, then send a request. It should succeed on the first try.
python -c "from sqlalchemy import text; print(engine.connect().execute(text('select 1')).scalar())"Confirmed by docs.sqlalchemy.org
Cause 2 of 3
A network device drops the idle TCP connection
1Check
libpq can send TCP keepalives so a quiet connection looks alive. keepalives is on by default, but the idle, interval and count values default to the operating system settings, which may be long.
grep -rn "keepalives" --include=*.py .2Fix
Pass keepalive settings through connect_args. They go straight to the driver's connect call.
engine = create_engine(
"postgresql+psycopg2://user:password@db.example.com/mydb",
connect_args={
"keepalives": 1,
"keepalives_idle": 30,
"keepalives_interval": 10,
"keepalives_count": 5,
},
)3Verify
Run a query that sleeps longer than the gap that used to fail.
psql "postgresql://user@db.example.com/mydb?keepalives_idle=30" -c "select pg_sleep(120)"Confirmed by postgresql.org
Cause 3 of 3
A forked worker is using the parent's connection
1Check
A pooled connection must not be shared by two processes, because both would use the same socket. Check whether the engine is created at import time in the parent and used in children.
grep -rn "create_engine" --include=*.py .2Fix
In each child, discard the inherited pool without closing the parent's sockets. SQLAlchemy documents engine.dispose(close=False) for this.
def init_worker():
engine.dispose(close=False)3Verify
Start the workers and send requests to each one.
for i in 1 2 3 4; do curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8000/health; doneConfirmed by docs.sqlalchemy.org
How do I stop it coming back?
- Enable pool_pre_ping on every long-lived engine.
- Choose pool_recycle lower than the shortest idle timeout on the path to the database.
- Create engines after the process fork, or call dispose(close=False) in the worker start-up hook.
- Retry a read request once on OperationalError; pre-ping cannot save a connection that breaks in the middle of a transaction.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
- stegnor.comMy company site: 804 pages from a Python and Jinja generator, with 15 free browser tools.
- MenuloA restaurant discovery platform in Germany, kept current by a Python ingestion pipeline.
- ShopGoAn e-commerce aggregator with a Python scraping engine behind its catalog, past 10K Google Play downloads.
Related fixes
Databases
psycopg2 fails with: server closed the connection unexpectedly
OperationalError: server closed the connection unexpectedly (psycopg2, Flask, SQLAlchemy)
The client was talking to Postgres and the connection vanished.
3 causes, each with a fixDatabases
python manage.py migrate finishes but the tables for my app are missing
relation "myapp_book" does not exist
migrate only applies migration files that exist.
3 causes, each with a fixDatabases
Django save fails on PostgreSQL: null value in a column violates a not-null constraint
django.db.utils.IntegrityError: null value in column "price" of relation "shop_product" violates not-null constraint
A model field is NOT NULL in the database unless it has null=True.
3 causes, each with a fixDatabases
Django migration on PostgreSQL stops with 'pending trigger events'
django.db.utils.OperationalError: cannot ALTER TABLE "mytable" because it has pending trigger events
PostgreSQL runs a Django migration in one transaction.
2 causes, each with a fixDatabases
psycopg2 cannot connect: connection refused on port 5432
Is the server running on that host and accepting TCP/IP connections?
Connection refused means nothing accepted the connection at the address psycopg2 tried.
3 causes, each with a fixDatabases
psycopg2 cannot resolve the host name db in Docker Compose
could not translate host name "db" to address
Compose registers each service in its own DNS under the service name, but only containers on the same Compose network can look it up.
4 causes, each with a fix