What is chain-of-thought prompting?

Chain-of-thought prompting is asking a model to show its working before giving an answer. In 2022 researchers found that this simple change — "let's think step by step" — substantially improved performance on problems requiring several steps.

The finding was notable because nothing about the model changed. The same weights, asked to work out loud, got more answers right.

Why it works

A model generates one token at a time, and each token gets a roughly fixed amount of computation. Demanding an immediate answer to a multi-step problem asks it to do all the work in one step, which it cannot.

Reasoning out loud spreads the work across many tokens. It also means each step conditions on the previous one — the model reads its own partial reasoning as context and builds on it. In effect, generating text is the thinking.

When it helps and when it doesn't

Task	Worth it?
Multi-step arithmetic or logic	Yes — the original result
Problems with intermediate steps	Yes
Anything where you'll check the working	Yes — the working is the value
Simple factual lookup	No. Wastes tokens, adds latency
Creative writing	Usually not
Classification into obvious buckets	No

The costs are real: reasoning tokens are billed, they consume the context window, and they add latency. Reflexively appending "think step by step" to every prompt makes cheap tasks expensive for nothing.

Note that many current models do this internally whether you ask or not — the technique has largely been absorbed into how models are trained and served. The explicit instruction matters most with smaller or older models.

The important caveat

The reasoning it prints is not necessarily the reasoning it used.

The visible chain is generated text, subject to all the usual behaviour of generated text. A model can produce impeccable-looking reasoning and a wrong conclusion; it can also reach the right answer and then construct a tidy justification that had nothing to do with it. Research on faithfulness has repeatedly found that stated reasoning and actual influences on the answer come apart.

So treat a chain of thought as a checkable artifact, not an explanation. Its value is that you can read step three and notice it's wrong. Its value is not that it tells you why the model said what it said. Confident, well-formatted reasoning is not evidence of a correct answer — same as everything else these models produce.

What we addedA guide to when step-by-step prompting helps versus wastes tokens, and why the printed reasoning should not be trusted as an explanation.

This article was researched and drafted with AI assistance from the sources listed below, then checked and edited by Fiqhro Dedhen before publication. How we work.