Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
AI and LLM integration

ollama Python library fails with httpx.ConnectError

Errorhttpx.ConnectError

Short answer

httpx.ConnectError means the client could not open a connection to the address it used. Ollama listens on 127.0.0.1 port 11434 by default. Check the server answers there, point the Python Client at the real host if it runs elsewhere, and set OLLAMA_HOST on the server when a container or another machine must reach it.

  • Applies to Python 3.12, ollama-python, Ollama on Linux, macOS, Windows
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Ask the server directly, from the same machine and shell as your Python code. If this fails, nothing is listening there.

    curl http://localhost:11434/api/generate -d '{"model": "mistral"}'
  2. 2Match what you see, then jump to its fix

    1. 1curl on localhost also failsThe Ollama server is not running at the address the client usesFix
    2. 2Ollama runs on another machine or port, and the code uses the default clientThe client uses a different host than the serverFix
    3. 3Your code runs in Docker or on another computer, and the server runs on the hostThe server only accepts local connectionsFix

What does the error look like?

  • httpx.ConnectError

Cause 1 of 3

The Ollama server is not running at the address the client uses

1Check

The Ollama FAQ says it listens on 127.0.0.1 port 11434. ollama ps talks to the same server, so it fails when the server is down.

ollama ps

2Fix

Start the Ollama app, or restart the service on Linux, then repeat the curl call.

sudo systemctl restart ollama

3Verify

curl returns a JSON reply.

curl http://localhost:11434/api/generate -d '{"model": "mistral"}'

Confirmed by docs.ollama.com

Did this fix it?

Cause 2 of 3

The client uses a different host than the server

1Check

The ollama-python README shows creating a Client with an explicit host, and says extra arguments go to httpx.Client.

grep -rn "Client(\|ollama\." --include=*.py .

2Fix

Create a client with the real address and use it for every call.

from ollama import Client

client = Client(host="http://192.168.1.50:11434")
print(client.list())

3Verify

The call returns the model list.

python app.py

Confirmed by raw.githubusercontent.com

Did this fix it?

Cause 3 of 3

The server only accepts local connections

1Check

By default Ollama binds to 127.0.0.1, which other machines and containers cannot reach. The FAQ says to change the bind address with the OLLAMA_HOST environment variable.

ss -ltn | grep 11434

2Fix

On Linux with systemd, add the variable to the service, reload and restart.

sudo systemctl edit ollama.service
# add under [Service]:
# Environment="OLLAMA_HOST=0.0.0.0:11434"
sudo systemctl daemon-reload
sudo systemctl restart ollama

3Verify

From the other machine or container, request the server address.

curl http://HOST_IP:11434/api/generate -d '{"model": "mistral"}'

Confirmed by docs.ollama.com

Did this fix it?

How do I stop it coming back?

  • Keep the Ollama address in one setting, not scattered through the code.
  • On macOS and Windows set OLLAMA_HOST the way the FAQ describes, then restart the app.
  • Expose the port only to networks you trust, and keep it closed to the internet.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

AI

LlamaIndex cannot import Ollama from llama_index.llms

ImportError: cannot import name 'Ollama' from 'llama_index.llms' (unknown location)

Since LlamaIndex 0.10, each integration is its own package and the old top-level llama_index.llms path is gone.

3 causes, each with a fix

AI

Ollama is slow and the GPU is not being used

Replies arrive a few tokens per second on a machine with a GPU

Run ollama ps and read the PROCESSOR column: 100% GPU means the model is on the card, anything with CPU means it is not fully there.

4 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.