Why Ollama gives you a 4K context window — and how to fix it

Your local model keeps losing the thread. You paste a long file and it answers about the first page. You assumed the model was weak.

It probably is not the model. Ollama chose a 4,096-token context window for it, and 4,096 tokens is about six pages.

Why it happens

When ollama serve starts, it inventories your GPUs, adds up their total memory, and picks a default context length from the total. It does this once, at startup, before any model is loaded — so the default has nothing to do with which model you run.

The tiers are:

Total VRAM detected	Default context
Under ~23 GiB	4,096 (4K)
~23 – 47 GiB	32,768 (32K)
~47 GiB and up	262,144 (256K)

Most consumer graphics cards have 8, 12 or 16 GB. They all land in the bottom tier. If you have never touched this setting, you are almost certainly on 4K.

The documented thresholds and the real ones differ slightly

Ollama's documentation states the tiers as under 24 GiB, 24–48 GiB, and 48 GiB and up. The source uses 23 and 47 GiB, with a comment explaining that the thresholds were lowered deliberately to allow for small differences in reported values.

This matters at two places, and both are popular card sizes. Cards do not report their full nominal size — after driver overhead a "24 GB" card may report 23.5 GiB, and a "48 GB" card may report 47.5.

Your card reports	Documented rule says	The code actually gives
23.5 GiB (a "24 GB" card)	4K — you are under 24	32K — you clear 23
47.5 GiB (a "48 GB" card)	32K — you are under 48	256K — you clear 47

In both cases the code is more forgiving than the documentation suggests, which is the direction you want to be surprised in. But if you sized your expectations from the docs, you may be leaving a much larger window on the table than you think.

The important diagnostic: 4K may mean your GPU is invisible

Look at how the total is built. Ollama sums the memory of the GPUs it discovered. If discovery fails, it finds no GPUs, the total is zero — and zero falls into the bottom tier.

Ollama's own troubleshooting docs note that GPU discovery can fail, usually from driver problems. Put those two facts together and you get the thing worth knowing:

A 4K default does not mean your card is too small. It may mean Ollama cannot see your card at all.

If you have a 24 GB GPU and you are getting 4K, do not raise the context length and move on. Find out why your GPU is missing, because you are probably also running on the CPU and wondering why it is slow.

How to check what you actually got

Ask Ollama. With a model loaded:

ollama ps

NAME             ID              SIZE      PROCESSOR    CONTEXT    UNTIL
gemma4:latest    c6eb396dbd59    9.6 GB    100% GPU     131072     2 minutes from now

Two columns matter. CONTEXT is what you actually got. PROCESSOR should say 100% GPU — anything mentioning CPU means part of the model is running on the processor, which is the other half of "why is this so slow".

Read the log. At startup Ollama logs a line reporting the total VRAM it found and the context length it chose from it. If the VRAM figure is far below your card's size, that is your real problem.

The fix

Set the context length explicitly when you serve:

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

If you use the desktop app, there is a context length slider in settings.

Ollama's docs recommend at least 64,000 tokens for anything agentic — web search, coding tools, document work. The 4K default is not sized for those jobs.

The catch

Context costs memory. The window is not free: it is allocated alongside the model weights, and it grows with length. Ask for 128K on a card that cannot hold it and you will either fail to load or spill onto the CPU and crawl.

So raise it deliberately, then check ollama ps again:

PROCESSOR still 100% GPU → good, keep it.
CPU appearing → you have overshot. Back the context off, or use a more heavily quantized model to free room.

There is a real trade here. A smaller model with a big window often beats a bigger model with a tiny one — a 4K window makes even an excellent model useless for anything involving a document.

The short version

Run ollama ps and look at CONTEXT. If it says 4096, that is your problem.
Check PROCESSOR says 100% GPU. If not, fix GPU detection first — the context default is a symptom, not the disease.
Serve with OLLAMA_CONTEXT_LENGTH=64000 (or use the app slider).
Re-run ollama ps. If the processor split moved to CPU, you asked for too much.

What we addedThe VRAM tier table as actually implemented in source (47/23 GiB) versus as documented (48/24 GiB), and the diagnostic that a 4K default usually means GPU discovery failed rather than that your card is small — derived by reading the tiering code against the troubleshooting docs.

This article was researched and drafted with AI assistance from the sources listed below, then checked and edited by Fiqhro Dedhen before publication. How we work.