How much VRAM do you need to run a local LLM?
Work it out in three steps, then check you were right — because the model file is not the whole bill.
Local & Self-Hosted AI
Running models on your own hardware — requirements, performance and trade-offs.
7 articles
What is a mixture of experts (MoE)?
Why a "large" model can be fast: most of it sits out any given word.
How to fix Ollama not using your GPU
It is slow, and it also silently gave your model a 4K context window. Both are the same bug.
Why Ollama gives you a 4K context window — and how to fix it
It is not the model. Ollama picks your context length from your VRAM at startup, and the bottom tier is very small.
What is a GGUF file?
The file format behind every local model you have downloaded — and how to read its name.
What is quantization?
Shrinking a model so it fits on hardware you own — and what you give up.
Ollama vs LM Studio vs llama.cpp: which local AI runner?
They are the same engine. You are choosing an interface and a licence, not a speed.