Ollama is slow and the GPU is not being used
ollama ps shows 100% CPU or a split such as 48%/52% CPU/GPUShort answer
Run ollama ps and read the PROCESSOR column: 100% GPU means the model is on the card, anything with CPU means it is not fully there. Then check the usual blocks: a driver older than Ollama needs, a GPU lost after suspend, a Docker container started without GPU access, or an AMD card that needs an override.
Fix it fast
1Find out which cause you have
Load a model, then ask Ollama where it put it. The PROCESSOR column is the answer.
ollama run llama3.2 "hello" ollama ps2Match what you see, then jump to its fix
- 1
ollama ps shows CPU only on a supported cardThe NVIDIA driver is too old for OllamaFix - 2
It was fast before the laptop or server slept, and is CPU-only sinceThe GPU disappeared after suspend and resume (Linux, NVIDIA)Fix - 3
Ollama works in Docker but only on CPUThe Docker container was started without GPU accessFix - 4
Ollama runs on CPU with a Radeon card (Linux)An AMD card Ollama does not recognise, or lacks permission to useFix
- 1
What does the error look like?
ollama ps shows 100% CPU or a split such as 48%/52% CPU/GPUReplies arrive a few tokens per second on a machine with a GPU
Cause 1 of 4
The NVIDIA driver is too old for Ollama
1Check
Ollama needs compute capability 5.0 or higher and driver 550 or newer. Cards with compute capability 5.0 to 6.2 need driver 570 or newer.
nvidia-smi --query-gpu=name,driver_version --format=csv2Fix
Update the NVIDIA driver to a version that meets the requirement, then restart Ollama.
sudo systemctl restart ollama3Verify
Load the model again and look at the PROCESSOR column.
ollama run llama3.2 "hello"
ollama psConfirmed by docs.ollama.com
Cause 2 of 4
The GPU disappeared after suspend and resume (Linux, NVIDIA)
1Check
Ollama's GPU guide lists this case: the UVM driver must be reloaded after resume.
nvidia-smi -L2Fix
Reload the nvidia_uvm kernel module, then restart Ollama.
sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm
sudo systemctl restart ollama3Verify
Run a model and read ollama ps again.
ollama psConfirmed by docs.ollama.com
Cause 3 of 4
The Docker container was started without GPU access
1Check
The CPU-only run command has no --gpus flag. NVIDIA GPUs also need the NVIDIA Container Toolkit configured for Docker.
docker inspect ollama --format '{{.HostConfig.DeviceRequests}}'2Fix
Configure Docker for the NVIDIA runtime, restart Docker, and recreate the container with --gpus=all. For AMD use the rocm image with the /dev/kfd and /dev/dri devices.
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
docker rm -f ollama
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama3Verify
Run a model inside the container and check where it loaded.
docker exec -it ollama ollama run llama3.2 "hello"
docker exec -it ollama ollama psConfirmed by docs.ollama.com
Cause 4 of 4
An AMD card Ollama does not recognise, or lacks permission to use
1Check
Supported AMD cards need the ROCm v7 driver. For a card outside the supported list, Ollama documents an override, and on some distributions the ollama user must be in the render group.
rocminfo | grep -i gfx2Fix
Add the ollama user to the render group. If the card is unsupported, try HSA_OVERRIDE_GFX_VERSION with a close supported version, for example 10.3.0, set on the Ollama service.
sudo usermod -aG render ollama
sudo systemctl edit ollama
# in the editor add:
# [Service]
# Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"
sudo systemctl restart ollama3Verify
Run a model and read the PROCESSOR column.
ollama psConfirmed by docs.ollama.com
How do I stop it coming back?
- Look at ollama ps after every model change, not only when it feels slow.
- Keep the GPU driver at or above the version Ollama lists, and recheck after OS upgrades.
- Keep the Docker run command in a compose file or script so the GPU flag is never forgotten.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
Related fixes
AI
The shell says ollama: command not found
command not found: ollama
The shell can only run ollama if the binary is installed in the same place the shell runs.
2 causes, each with a fixAI
Open WebUI or another Docker app cannot connect to Ollama
Connect call failed ('127.0.0.1', 11434)
Inside a container, localhost is the container itself, not your machine, so a call to 127.0.0.1:11434 never reaches Ollama.
3 causes, each with a fixAI
ollama Python library fails with httpx.ConnectError
httpx.ConnectError
httpx.ConnectError means the client could not open a connection to the address it used.
3 causes, each with a fixAI
Chroma data is empty after a restart in LangChain
The collection is empty after the app restarts
Chroma only keeps data when you ask it to.
3 causes, each with a fixAI
TextLoader fails on a text file with an encoding error
UnicodeDecodeError: codec can't decode byte
TextLoader opens the file with the encoding you give it, or the system default if you give none.
3 causes, each with a fixAI
LlamaIndex cannot import Ollama from llama_index.llms
ImportError: cannot import name 'Ollama' from 'llama_index.llms' (unknown location)
Since LlamaIndex 0.10, each integration is its own package and the old top-level llama_index.llms path is gone.
3 causes, each with a fix