What is a context window?

A context window is the maximum amount of text a language model can consider at one time. It is measured in tokens, not words, and it has to fit everything: your instructions, any documents you paste or attach, the whole conversation up to this point, and the answer the model is currently writing.

When people say a model has a "200K context window", they mean roughly 200,000 tokens of room, shared across all of that.

Why it exists at all

Models based on the transformer architecture compare every token against every other token to work out what matters. That comparison is what makes them good, and it is also what makes long inputs expensive: the work grows much faster than the length of the text does. The context window is where the vendor draws the line.

What it means in practice

Token counts are hard to picture, so here is a rough translation, using the old convention of about three-quarters of a word per token.

Treat that ratio as a loose convention rather than a constant, because it moves with the tokenizer — and it has moved. Anthropic's newer models produce roughly 30% more tokens for the same text, which puts their 1M window nearer 555,000 words than the 750,000 below. Code, tables and non-English text tokenize differently again.

The only number you can rely on is the one your vendor's own tokenizer returns. For a case where the same nominal window quietly shrank by a quarter, see which Claude model should you use.

The three things that actually bite you

Running out. When a conversation exceeds the window, something has to go. Most chat products silently drop or summarise the oldest turns. That is why a long session starts "forgetting" what you told it at the beginning — the model isn't being forgetful, the text is simply no longer there.

Paying for it. API pricing is per token, and the entire window is re-read on every single turn. A long conversation doesn't just cost more at the end; it costs more on every message, because all of it gets sent again.

The important caveat: if the repeated part is identical each turn, you can cache it and pay roughly 10% of the rate for that portion. That changes the arithmetic enormously, and most people leave it switched off. See prompt caching.

Waiting for it. Larger inputs take longer to process. A 500K-token prompt is not free just because it fits.

Big window vs. good retrieval

A large context window is not the same as the model using it well. Vendors publish "needle in a haystack" tests precisely because performance across a long window is not uniform — material in the middle of a very long input tends to get less reliable attention than material at the start or end.

This is why RAG hasn't gone away. Feeding a model the ten paragraphs that matter is usually faster, cheaper and more accurate than feeding it a thousand pages and hoping.

What to do about it

• For long chats, start a new conversation when you change topic. You get the full window back and stop paying to re-read irrelevant history.
• If you need the model to remember something specific, restate it. Don't assume it survived 200 turns ago.
• If you're building on the API, measure your token use before optimising it. The bill is usually context, not output.