Npgsql throws 'Exception while reading from stream' against PostgreSQL
Npgsql.NpgsqlException: Exception while reading from streamShort answer
This exception means the PostgreSQL connection was closed under Npgsql while it waited for data. Either something between the app and the server dropped an idle socket, or the server ended the session itself. Turn on Npgsql keepalive for the first, and check the server's idle session timeouts for the second.
Fix it fast
1Find out which cause you have
Ask the server whether it has a timeout that ends sessions. If either value is not 0, the server may be the one closing the connection.
SHOW idle_session_timeout; SHOW idle_in_transaction_session_timeout;2Match what you see, then jump to its fix
- 1
The error appears on the first query after the app has been quiet for a while, and the next retry worksA firewall, load balancer or proxy dropped the idle connectionFix - 2
idle_session_timeout or idle_in_transaction_session_timeout shows a value other than 0The server ended the session because of an idle timeoutFix - 3
The error appears in the middle of one slow query, not after an idle periodA long query runs with no traffic and the socket is silently lostFix
- 1
What does the error look like?
Npgsql.NpgsqlException: Exception while reading from streamSystem.IO.EndOfStreamException: Attempted to read past the end of the stream
Cause 1 of 3
A firewall, load balancer or proxy dropped the idle connection
1Check
Npgsql keeps pooled connections open between requests. Network equipment that forgets idle TCP connections leaves Npgsql holding a socket the other side has already discarded. Check whether Keepalive is set in your connection string; it is off by default (0).
grep -R -n -i "keepalive" appsettings*.json2Fix
Turn on Npgsql's application-level Keepalive, in seconds, with a value lower than the idle limit of the network device. On Linux, Tcp Keepalive=true also asks the OS to send TCP probes, which is meant for long-running queries.
Host=db.example.com;Database=app;Username=app;Password=...;Keepalive=30;Tcp Keepalive=true3Verify
Leave the app idle for longer than before and then run a query. The first query should succeed.
dotnet runConfirmed by npgsql.org
Cause 2 of 3
The server ended the session because of an idle timeout
1Check
Both settings default to 0 (disabled), so a non-zero value was set on purpose for the server, database or role. A session that sits idle longer than the limit is terminated, and pooled connections look idle.
SELECT rolname, rolconfig FROM pg_roles WHERE rolconfig IS NOT NULL;2Fix
Remove the limit for the application role, or raise it above the time a pooled connection can sit idle. The PostgreSQL manual advises applying these limits to interactive users and warns that poolers may not cope with sudden closure.
ALTER ROLE app_user RESET idle_session_timeout;
ALTER ROLE app_user RESET idle_in_transaction_session_timeout;3Verify
Open a new connection as that role and check the value.
SHOW idle_session_timeout;Confirmed by postgresql.org
Cause 3 of 3
A long query runs with no traffic and the socket is silently lost
1Check
PostgreSQL only notices a lost client when it next touches the socket. With the server's TCP keepalive settings at 0 it uses the operating system defaults, which are often slow to detect a dead link.
SHOW tcp_keepalives_idle;2Fix
Set the server's TCP keepalive values so dead connections are detected sooner, and use Npgsql's Tcp Keepalive for the client side.
# postgresql.conf
tcp_keepalives_idle = 60
tcp_keepalives_interval = 10
tcp_keepalives_count = 63Verify
Reload the configuration and read the value back.
SELECT pg_reload_conf();
SHOW tcp_keepalives_idle;Confirmed by postgresql.org
How do I stop it coming back?
- Set Keepalive in every production connection string that goes through a firewall, load balancer or cloud NAT.
- Keep server-side idle timeouts for people, not for the application role.
- Open connections late and dispose them early, so pooled connections are reused rather than left idle.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
Related fixes
Databases
Npgsql refuses a local DateTime for a timestamp with time zone column
A save that worked before the Npgsql 6 upgrade now fails on a DateTime property
Since Npgsql 6, a DateTime is written as timestamptz only when its Kind is Utc.
3 causes, each with a fixDatabases
Dapper cannot find a constructor that fits the query columns
The error appears when Dapper maps rows into a record or a class whose only constructor takes arguments
Dapper looks for a public parameterless constructor first.
3 causes, each with a fixDatabases
DBeaver cannot run pg_restore: the utility is not found in the client home
Utility 'pg_restore' not found in client home '/usr/lib' (/usr/lib)
DBeaver runs pg_restore from a local PostgreSQL client folder; it does not bundle one.
3 causes, each with a fixDatabases
DBeaver cannot download the PostgreSQL driver from Maven
The PostgreSQL driver files are missing and the download does not complete
DBeaver downloads the PostgreSQL JDBC driver from a Maven repository the first time you connect.
3 causes, each with a fixDatabases
python manage.py migrate finishes but the tables for my app are missing
relation "myapp_book" does not exist
migrate only applies migration files that exist.
3 causes, each with a fixDatabases
Django save fails on PostgreSQL: null value in a column violates a not-null constraint
django.db.utils.IntegrityError: null value in column "price" of relation "shop_product" violates not-null constraint
A model field is NOT NULL in the database unless it has null=True.
3 causes, each with a fix