LlamaIndex gets 404 Not Found from Ollama at /api/chat
Client error '404 Not Found' for url 'http://localhost:11434/api/chat'Short answer
Ollama's error docs describe 404 as a resource that was not found, such as a model that does not exist. A LlamaIndex agent that sends a model name Ollama has not downloaded gets exactly that on /api/chat. List the local models, pull the one you named, and use the same name, including the tag, in Ollama(model=...).
Fix it fast
1Find out which cause you have
Ask the Ollama server which models it really has. The name you pass to LlamaIndex must match one entry exactly.
curl http://localhost:11434/api/tags2Match what you see, then jump to its fix
What does the error look like?
Client error '404 Not Found' for url 'http://localhost:11434/api/chat'
Cause 1 of 2
The model was never pulled
1Check
Compare the name in Ollama(model=...) with the names returned by /api/tags.
curl http://localhost:11434/api/tags
grep -rn "Ollama(" --include=*.py .2Fix
Download the model through the pull endpoint, wait for it to finish, then retry the agent.
curl http://localhost:11434/api/pull -d '{ "model": "llama3.2" }'3Verify
The model now appears in the list.
curl http://localhost:11434/api/tagsConfirmed by github.com
Cause 2 of 2
The model name or tag in the code differs from the local one
1Check
The LlamaIndex integration selects a model as name:tag, such as llama3.1:latest, and looks for the latest version when you give only a family name.
curl http://localhost:11434/api/tags2Fix
Copy the exact name from /api/tags into your code.
from llama_index.llms.ollama import Ollama
llm = Ollama(model="llama3.1:latest", request_timeout=120.0)3Verify
Run the agent again; the chat call returns a reply rather than a 404.
python -c "print(llm.complete('hi'))"Confirmed by pypi.org
How do I stop it coming back?
- Keep the model name in one config value, shared by the pull step and the app.
- Print the model list at startup and stop with a clear message if the model is missing.
- Set request_timeout explicitly for large models.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
- stegnor.comMy company site: 804 pages from a Python and Jinja generator, with 15 free browser tools.
- MenuloA restaurant discovery platform in Germany, kept current by a Python ingestion pipeline.
- ShopGoAn e-commerce aggregator with a Python scraping engine behind its catalog, past 10K Google Play downloads.
Related fixes
AI
LlamaIndex cannot import Ollama from llama_index.llms
ImportError: cannot import name 'Ollama' from 'llama_index.llms' (unknown location)
Since LlamaIndex 0.10, each integration is its own package and the old top-level llama_index.llms path is gone.
3 causes, each with a fixAI
ollama Python library fails with httpx.ConnectError
httpx.ConnectError
httpx.ConnectError means the client could not open a connection to the address it used.
3 causes, each with a fixAI
Azure OpenAI says the API deployment for this resource does not exist
The API deployment for this resource does not exist
Azure OpenAI does not take a model name in the model field.
2 causes, each with a fixAI
Azure OpenAI fails with an SSL certificate verification error
httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED]
The client checks the endpoint's certificate against its CA bundle, which is certifi by default.
2 causes, each with a fixAI
LangChain says it could not import the chromadb package
ImportError: Could not import chromadb python package. Please install it with `pip install chromadb`.
The message comes from LangChain's old Chroma class, which cannot import chromadb in the Python that runs your code.
2 causes, each with a fixAI
Gemini API returns 429 RESOURCE_EXHAUSTED on the free tier
429 RESOURCE_EXHAUSTED
Gemini limits are applied per Google Cloud project, not per API key, so a new key in the same project does not reset anything.
3 causes, each with a fix