AI PC vs GPU vs Mac: what actually runs a local LLM?

If you want to run a language model on your own machine, three kinds of hardware get marketed at you: an "AI PC" with a neural processing unit, a desktop with a discrete graphics card, and an Apple-Silicon Mac. The pitch usually centres on the NPU and its TOPS rating. That pitch is aimed at the wrong part of the machine.

Here is the finding, up front: the NPU is not what runs a local LLM today. The tools people actually use for this — Ollama, LM Studio, llama.cpp — run the model on the GPU, or on an Apple Mac's unified memory through Apple's Metal. The NPU sits unused. What decides how big a model you can run is memory, not TOPS.

Why the NPU is not the answer (yet)

Two things, both checkable.

First, what the runtimes target. llama.cpp — the engine underneath both Ollama and LM Studio — ships mature backends for Nvidia GPUs (CUDA), Apple Silicon (Metal), AMD GPUs (HIP) and generic GPUs (Vulkan). Backends for consumer laptop NPUs do exist — Intel's through OpenVINO, Qualcomm's Snapdragon through Hexagon — but both are marked in progress in the project's own README. They are experiments, not the path your download takes today.

Second, what the NPU was built for. Microsoft's own developer guide describes the NPU as the chip for Windows' built-in AI features — live captions, image cleanup, background effects — running small models that have been quantized to low-precision integer maths. That is a different job from running a general 13B-parameter chatbot, and a different software stack (ONNX Runtime, not llama.cpp).

None of this makes an AI PC useless for local AI. It means the NPU is not doing the work — the CPU or the integrated GPU is, and neither is quick at it.

What actually sets the ceiling: memory

A model has to fit in memory to run, and that memory has to be fast. A discrete GPU brings its own dedicated, very fast VRAM. An Apple-Silicon Mac uses unified memory — one pool shared by CPU and GPU, so the GPU can address most of the system's RAM without copying data across (Apple's MLX framework is built around this idea: its arrays "live in shared memory"). An AI PC's NPU has neither advantage for this task.

Path	What a local-LLM runtime actually uses	Memory ceiling	Roughly fits
AI PC (e.g. a 40-TOPS NPU laptop)	The CPU or integrated GPU — not the NPU	Shared system RAM, lower bandwidth	Small models, slowly
Discrete GPU (e.g. a 16 GB card)	The GPU, via CUDA / Vulkan / ROCm	Its dedicated VRAM — fast, but fixed	Around a 13B model at 4-bit
Apple-Silicon Mac (e.g. 32 GB unified)	The GPU, via Apple Metal	Unified memory, shared with the whole system	A 30B-class model at 4-bit

The sizes are deliberately soft — the real number depends on quantization and how much context you want. For the arithmetic, see how much VRAM you actually need. The shape holds regardless: for this workload, more memory and faster memory matter more than a higher TOPS figure.

Which should you get?

Framed by what you are optimising for, not by price:

Already have a gaming PC with a discrete GPU? You are most of the way there. VRAM capacity is the thing to check — a card's GGUF-sized headroom decides which models load.
Buying a Mac anyway? Unified memory makes Apple Silicon a genuinely strong local-AI machine, and the amount of memory is the spec that matters. A 30B-class model is comfortable on 32 GB of unified memory; a 70B wants considerably more — 48 GB or beyond.
Tempted by an AI PC specifically to run chatbots locally? That is the mismatch to avoid. The NPU is good at the always-on features it was built for, but it will not be the chip running your local LLM — and the laptop's shared memory and bandwidth are the limiting factors, not the TOPS on the sticker.

The hype-free summary: for local LLMs, buy memory, not TOPS. The NPU is a real and useful chip pointed at a real and different job — see what an NPU is for.

What we addedA table mapping each hardware path (AI-PC NPU, discrete GPU, Apple-Silicon Mac) to what a local-LLM runtime actually uses, its memory ceiling, and the rough model size that fits — with the finding that the NPU is not the part doing the work.

This article was researched and drafted with AI assistance from the sources listed below, then checked and edited by Fiqhro Dedhen before publication. How we work.