What is a transformer?

A transformer is a neural network architecture introduced in 2017. The "T" in GPT stands for it. Essentially every large language model in use today — whichever company made it — is a transformer or a close relative.

The idea it replaced

Earlier language models read text in order, one word at a time, carrying a summary of what came before. That approach had two problems: it was slow to train because it couldn't be parallelised, and it tended to lose the thread over long passages.

The transformer's contribution was attention: instead of reading sequentially, look at every token in the input simultaneously and learn which ones are relevant to which. In "the trophy didn't fit in the suitcase because it was too big", attention is what lets the model connect "it" to "the trophy" rather than "the suitcase".

Because every position is processed at once, training parallelises across thousands of chips. That property — not raw cleverness — is most of why models got big so fast.

The catch

Comparing every token to every other token means the work grows with the square of the input length. Double the input, quadruple the comparisons. This is the direct cause of context window limits, and the reason long prompts cost real money. A great deal of research since 2017 has gone into making attention cheaper without making it worse.

How we got from there to here

What it means for you

Very little, day to day — and that is the point. You don't need to understand attention to use a model, any more than you need to understand TCP to load a web page.

It's worth knowing exactly one implication: transformers predict likely continuations of text. They are not looking anything up unless something has explicitly given them a document to look at. That single fact explains most of what people find confusing about model behaviour, including hallucination.