How to Use Chain of Thought Prompting Without Wasting Tokens

Chain of thought prompting means asking a model to work through a problem in visible steps before it commits to an answer, instead of producing the answer directly. The minimal version is one line: "Think step by step, then answer." The useful version also says what to reason about, where the reasoning goes, and where the final answer goes so your code can extract it.

It helps on tasks with dependent steps: date and unit math, rules with several conditions, debugging, and classification with awkward edge cases. It wastes tokens on lookups, reformatting, and extraction, and it's mostly redundant on models that already reason in a built-in thinking phase before they reply. Below: how it works, templates you can paste, code for building the prompt and pulling out the answer, a comparison with few-shot, ReAct, and tree of thought, and the mistakes that quietly cost money.

How chain of thought prompting works#

A language model produces one token at a time, and each token is conditioned on everything before it. If the model answers "no" immediately, that one token has to carry the whole computation. If it first writes "06:40 plus 2 hours is 08:40, plus 35 minutes is 09:15," each intermediate value becomes context that the next step can read. The reasoning text is scratch paper.

Two things follow. The reasoning has to come before the answer, because an explanation written after the answer is a justification, not a computation, and it won't catch a wrong answer. And the quality of the steps matters more than their presence. Three vague sentences followed by a guess isn't reasoning. Prompts that say what to check produce better steps than prompts that only say "think."

How to write a chain of thought prompt#

Start small and add structure only when the output needs it.

[TASK DESCRIPTION]

Think through the problem step by step before answering. Then give your final answer inside <answer></answer> tags, with nothing after the closing tag.

[INPUT]

Why this wording: "step by step" triggers the behavior, "before answering" fixes the order, and the answer tags give your code a reliable string to pull out. Anthropic's chain of thought guidance recommends this tag pattern for manual chain of thought on Claude when thinking is off, and the same prompt works with ChatGPT and Gemini.

When the task has known failure modes, name the checks rather than hoping the model finds them:

You are reviewing [KIND OF INPUT, e.g. a SQL migration].

Work through these in order and write a short note for each:
1. What the input is trying to do.
2. [SPECIFIC CHECK 1, e.g. any column dropped or renamed that other tables reference]
3. [SPECIFIC CHECK 2, e.g. any operation that locks a large table]
4. Anything that contradicts [CONSTRAINT].

Then give your verdict inside <answer></answer> tags as exactly one of: [LABEL A], [LABEL B], [LABEL C].

<input>
[INPUT]
</input>

Why this wording: the numbered list makes order and completeness explicit, which Anthropic's docs recommend whenever the steps matter. The fixed label set keeps the answer parseable. Putting the input in its own tags separates data from instructions, so a sentence inside the input that reads like a command is less likely to be obeyed.

The third pattern is a self-check at the end, which the same docs describe as a reliable way to catch errors in coding and math:

[TASK]

Before you finish, verify your answer against [TEST CRITERIA, e.g. the three sample inputs and expected outputs below]. If a check fails, fix the answer and say what changed.

[INPUT]

Why this wording: concrete criteria give the model something to reason against, which beats a general "be careful." One caveat from the docs: Claude Opus 5 already verifies its own work and can over-verify when told to, so on that model the advice is to remove the instruction rather than reword it.

For the surrounding structure, including roles, constraints, and examples, the CoreMesh overview of prompt engineering techniques that work in any AI chat goes deeper. If your main use is code review or debugging, the copy-paste prompts for coding already bake in a reason-then-answer shape.

Chain of thought prompting examples you can run#

The code around a chain of thought prompt is short: build the prompt, send it, extract the answer. This script does the first and last steps against a sample response, so you can see the extraction logic without an API key.

import re

TEMPLATE = """{task}

Think through the problem step by step before answering.
Then give your final answer inside <answer></answer> tags,
with nothing after the closing tag.

{input}"""

def build_prompt(task, user_input):
    return TEMPLATE.format(task=task, input=user_input)

def extract_answer(text):
    match = re.search(r"<answer>(.*?)</answer>", text, re.DOTALL)
    return match.group(1).strip() if match else None

prompt = build_prompt(
    "Decide whether the shipment arrives before the store opens.",
    "Shipment leaves at 06:40 and takes 2 h 35 min. Store opens at 09:00.",
)
print(prompt)
print("-" * 40)

sample_response = """06:40 plus 2 h is 08:40. Plus 35 min is 09:15.
09:15 is after 09:00, so the shipment arrives after opening.
<answer>No, it arrives at 09:15, 15 minutes after the store opens.</answer>"""

print("Extracted:", extract_answer(sample_response))
print("Missing tag ->", extract_answer("Probably no."))

The None case is the important one. If the model skips the tags, you want to detect that and retry, not parse a paragraph of reasoning as if it were the answer.

With the Claude API, the current advice is to let the model's built-in thinking do the reasoning and to control depth through the effort setting instead of prompt text. Anthropic's docs show this shape:

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=16000,
    thinking={"type": "adaptive"},
    output_config={"effort": "high"},
    messages=[{
        "role": "user",
        "content": "Decide whether the shipment arrives before the store opens. "
                   "Shipment leaves at 06:40 and takes 2 h 35 min. Store opens at 09:00.",
    }],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

Two details from the same page: pick content blocks by type rather than by position, because a response can begin with thinking blocks, and leave room in max_tokens for the thinking itself.

Chain of thought vs few shot, ReAct, and tree of thought#

These names get mixed up because they overlap. Few-shot is about examples. Chain of thought is about visible reasoning. ReAct and tree of thought are control patterns built on top of reasoning.

Few-shot and chain of thought combine well. Anthropic's docs say to wrap examples in example tags so the model can tell them from instructions, and that worked examples still shape how Claude approaches problems even when it reasons in thinking blocks. Present each example as the problem, the method to apply, and the expected answer. For how many examples to include, see how few shot prompting works and how many examples to use.

ReAct is what most tool-using agents do: the model writes a short thought, picks a tool, reads the result, and repeats until it has what it needs. If you're calling an API with tools enabled, you're getting this already, and adding "think step by step" to the system prompt mostly makes each thought longer.

Tree of thought is rarely worth building yourself. It means generating several reasoning branches, scoring them, and dropping the weak ones, which multiplies the cost of every answer. It's a research technique for puzzles with dead ends, not a default for production prompts.

When chain of thought helps and when it wastes tokens#

So the question isn't whether chain of thought improves reasoning in general. It's whether this particular task has any reasoning in it.

That last row is the one that changed most recently. According to Anthropic's docs, thinking is on by default on Claude Opus 5, Sonnet 5.5, Sonnet 5, and Haiku 5.5 when you omit the thinking parameter, and always on for Claude Fable 5.1, Mythos 5.1, Fable 5, Mythos 5, and Opus 5.5. On those models the docs say to rely on extended thinking rather than manual chain of thought, and to pick a lower effort level if cost or latency matters. They also warn that a prompt asking Claude to write out its reasoning, for example in thinking tags, may be declined on Fable 5.1, Fable 5, Opus 5.5, Opus 5, and Sonnet 5.5. On Claude Opus 4.6 through 4.8 and Sonnet 4.6, thinking is off unless you turn it on, and manual chain of thought still applies.

The same page makes a point that holds across vendors: general instructions tend to beat prescriptive ones. A prompt like "think thoroughly" often produces better reasoning than a hand-written step-by-step plan, because the model's own decomposition frequently goes further than a human would specify. Save numbered steps for cases where a specific check must happen, like the review template above.

When you want a cheap, direct answer, say so:

Answer with only [FORMAT, e.g. a JSON object with the keys "label" and "confidence"]. Do not explain your reasoning.

[INPUT]

Why this wording: models tend to add explanation by default. Naming the exact output and ruling out the explanation keeps tokens down on tasks that have nothing to reason about.

Checking that the model actually reasoned#

Once chain of thought is in production, the risk is drift. The model starts answering first, or writes two words of reasoning, or adds a sign-off after the answer tag that breaks your parser. A few string checks catch these shapes in evals and logs.

const checks = {
  hasAnswerTag: (t) => /<answer>[\s\S]*?<\/answer>/.test(t),
  answerIsLast: (t) => /<\/answer>\s*$/.test(t),
  reasonsBeforeAnswer: (t) => {
    const i = t.indexOf("<answer>");
    return i > 0 && t.slice(0, i).trim().split(/\s+/).length >= 15;
  },
};

const samples = {
  good:
    "06:40 plus 2 h is 08:40. Plus 35 min is 09:15. 09:15 is after 09:00, " +
    "so it arrives after opening and the answer is no.\n<answer>No</answer>",
  answerFirst:
    "<answer>No</answer>\nBecause 06:40 plus 2 h 35 min is 09:15, which is after 09:00.",
  noTag: "It arrives at 09:15, so no.",
  trailingChatter:
    "08:40, then 09:15, which is after opening.\n<answer>No</answer>\n" +
    "Let me know if you need anything else!",
};

for (const [name, text] of Object.entries(samples)) {
  const results = Object.entries(checks).map(
    ([k, fn]) => `${k}=${fn(text) ? "ok" : "FAIL"}`
  );
  console.log(name.padEnd(16), results.join("  "));
}

The checks are crude, and a word count says nothing about quality. But they catch the shapes that break pipelines, and they cost nothing to run on every response. For a real quality check, Anthropic's docs describe a chaining pattern: generate a draft, have the model review it against criteria in a second call, then refine in a third. Each step is a separate call you can log, evaluate, or branch on.

Common mistakes with chain of thought prompting#

• Asking for the answer first and the reasoning second. The reasoning becomes a rationalization. Put the answer at the end, every time.
• Parsing the whole response as the answer. Use tags or a fixed final line, and handle the case where the tag never shows up.
• Showing the reasoning to end users. Scratch work belongs to the model and your logs. Strip it before display or collapse it.
• Stacking "think step by step" on a model with built-in thinking. Redundant at best. On several current Claude models a request for reasoning in thinking tags may be declined; use the effort setting instead.
• Writing a rigid ten-step plan where one general instruction would do. Over-prescription boxes in reasoning that would have been better on its own.
• Too small a max_tokens. Reasoning plus answer can get cut off, and if the closing tag never arrives the answer is gone. Raise the limit, and on Claude leave room for thinking blocks as well.
• Mixing the input into the instructions. Wrap it in its own tags so a command-shaped sentence in the data doesn't get followed.
• Carrying self-check instructions across models. Anthropic's docs single out Claude Opus 5 as a model where verification prompts tuned for earlier models cause over-verification and should be removed.
• Treating one good run as proof. Run the prompt against a small set of inputs with known answers, with and without the reasoning instruction, and compare both accuracy and output length.

Quick reference#

• Use chain of thought for math, dates, multi-condition rules, debugging, and ambiguous classification.
• Skip it for extraction, reformatting, tone changes, and facts the model already knows.
• Reasoning first, answer last, answer inside tags.
• Prefer "think thoroughly" over a prescriptive plan; add numbered steps only for checks that must happen.
• Add a self-check with concrete criteria for code and math, except on Claude Opus 5.
• On models with thinking on by default or always on, drop manual chain of thought and tune effort instead.
• Select content blocks by type, and leave room in max_tokens for thinking.
• Validate the response shape: tag present, tag last, reasoning before it.
• Measure accuracy and token count on a fixed test set before shipping.

Quick answers