How to fix Ollama not using your GPU

Ollama is running, models answer, everything works — just slowly. Somewhere along the way you accepted that local AI is like this.

It is not. Your GPU is probably not being used.

Confirm it in five seconds

With a model loaded:

ollama ps

Look at PROCESSOR. It should say 100% GPU. Anything mentioning CPU means part or all of the model is running on your processor.

And check CONTEXT while you are there. If it says 4096, that is the same bug wearing a different hat: Ollama sizes the default context window from the total VRAM it detected at startup, and a GPU it cannot see contributes nothing. No GPU found means the bottom tier — 4,096 tokens. See why Ollama gives you a 4K context window.

So the same failure makes your model slow and forgetful, which is why it gets misdiagnosed as "local models are bad".

Read the log

Ollama tells you what it found at startup. Where to look depends on your OS:

If you ran ollama serve yourself, the logs are in that terminal.

For more detail, set OLLAMA_DEBUG=1 before starting.

Cause 1: your card is not supported

Ollama publishes hard requirements for Nvidia, and they are stricter than people assume:

• Compute capability 5.0 or newer
• Driver 550 or newer
• Cards with compute capability 5.0–6.2 need driver 570 or newer

That last line catches people with older cards who dutifully updated to 550 and still get nothing. Check your card's compute capability against Nvidia's list before assuming a software problem.

AMD cards are supported through ROCm, with additional coverage via Vulkan.

Cause 2: the Linux suspend/resume bug

This one is documented, specific, and infuriating: on Linux, after a suspend/resume cycle Ollama sometimes fails to discover your Nvidia GPU and falls back to CPU. It is a driver bug, not an Ollama bug.

Ollama publishes the workaround — reload the UVM driver:

sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm

If your machine is fast after a reboot and slow after a lid-close, this is your answer, and you have probably been rebooting to fix it.

Cause 3: the wrong library got picked

Ollama ships multiple inference libraries compiled for different GPUs and CPU instruction sets, and picks one by autodetection. The log line listing them looks roughly like:

Dynamic LLM libraries [rocm_v6 cpu cpu_avx cpu_avx2 cuda_v11 rocm_v5]

If autodetection picks badly, you can override it:

OLLAMA_LLM_LIBRARY="cpu_avx2" ollama serve

Ollama labels this experimental. Treat it as a diagnostic — if forcing a library fixes things, you have found the problem, not the cure.

Cause 4: the GPU is excluded

With multiple Nvidia GPUs, CUDA_VISIBLE_DEVICES selects which Ollama may use. It takes numeric IDs or UUIDs — and Ollama's docs recommend UUIDs, because numeric ordering can vary. Get them with nvidia-smi -L.

Worth knowing: an invalid GPU ID such as -1 forces CPU. If that variable is set somewhere in your shell profile from an old experiment, it will silently do exactly what you asked.

The order to work through

1. ollama ps → is PROCESSOR 100% GPU? If yes, your problem is elsewhere.
2. Read the server log. Ollama says what it discovered.
3. Check compute capability and driver version against the requirements above.
4. On Linux after a suspend: reload nvidia_uvm.
5. Check CUDA_VISIBLE_DEVICES is not set to something silly.
6. Re-check ollama ps — and confirm CONTEXT climbed off 4096 too.

If step 6 shows a bigger context window than before, you did not just make it faster. You made it usable.