Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
AI and LLM integration

LlamaIndex gets 404 Not Found from Ollama at /api/chat

ErrorClient error '404 Not Found' for url 'http://localhost:11434/api/chat'

Short answer

Ollama's error docs describe 404 as a resource that was not found, such as a model that does not exist. A LlamaIndex agent that sends a model name Ollama has not downloaded gets exactly that on /api/chat. List the local models, pull the one you named, and use the same name, including the tag, in Ollama(model=...).

  • Applies to Ollama 0.x, llama-index-llms-ollama, Python 3.12
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Ask the Ollama server which models it really has. The name you pass to LlamaIndex must match one entry exactly.

    curl http://localhost:11434/api/tags
  2. 2Match what you see, then jump to its fix

    1. 1/api/tags does not list the model name used in your codeThe model was never pulledFix
    2. 2A similar model is listed, but not under the exact name you passThe model name or tag in the code differs from the local oneFix

What does the error look like?

  • Client error '404 Not Found' for url 'http://localhost:11434/api/chat'

Cause 1 of 2

The model was never pulled

1Check

Compare the name in Ollama(model=...) with the names returned by /api/tags.

curl http://localhost:11434/api/tags
grep -rn "Ollama(" --include=*.py .

2Fix

Download the model through the pull endpoint, wait for it to finish, then retry the agent.

curl http://localhost:11434/api/pull -d '{ "model": "llama3.2" }'

3Verify

The model now appears in the list.

curl http://localhost:11434/api/tags

Confirmed by github.com

Did this fix it?

Cause 2 of 2

The model name or tag in the code differs from the local one

1Check

The LlamaIndex integration selects a model as name:tag, such as llama3.1:latest, and looks for the latest version when you give only a family name.

curl http://localhost:11434/api/tags

2Fix

Copy the exact name from /api/tags into your code.

from llama_index.llms.ollama import Ollama
llm = Ollama(model="llama3.1:latest", request_timeout=120.0)

3Verify

Run the agent again; the chat call returns a reply rather than a 404.

python -c "print(llm.complete('hi'))"

Confirmed by pypi.org

Did this fix it?

How do I stop it coming back?

  • Keep the model name in one config value, shared by the pull step and the app.
  • Print the model list at startup and stop with a clear message if the model is missing.
  • Set request_timeout explicitly for large models.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

AI

LlamaIndex cannot import Ollama from llama_index.llms

ImportError: cannot import name 'Ollama' from 'llama_index.llms' (unknown location)

Since LlamaIndex 0.10, each integration is its own package and the old top-level llama_index.llms path is gone.

3 causes, each with a fix

AI

LangChain says it could not import the chromadb package

ImportError: Could not import chromadb python package. Please install it with `pip install chromadb`.

The message comes from LangChain's old Chroma class, which cannot import chromadb in the Python that runs your code.

2 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.