How Reasoning Models Differ
Internal chain-of-thought vs standard models: what changes for the prompt author.
How Reasoning Models Differ is a free AI Prompt Engineering lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
What Are Reasoning Models?
Reasoning models are LLMs specifically trained and configured to run extended internal deliberation before producing a response. Examples include OpenAI's o1/o3/o4 series and Anthropic's Claude with extended thinking enabled.
Unlike standard models that generate text token-by-token directly from your prompt, reasoning models first produce a lengthy internal chain-of-thought, then summarize it into a final answer.
Standard vs Reasoning: What You See
From the API user's perspective, the difference is:
- Standard model: Input → Output (fast, direct)
- Reasoning model: Input → [Internal thinking, hidden or streamed] → Output (slower, more accurate on hard problems)
The thinking process is the model's private scratchpad. It may contain wrong turns, self-corrections, and intermediate calculations that never appear in the final answer.
OpenAI o-Series: API Behavior
OpenAI's o1/o3/o4 models handle thinking internally — you don't see the reasoning tokens by default. You can observe the thinking token count in usage metadata, but not the content.
import openai
client = openai.OpenAI(api_key='sk-...')
# o3 — reasoning happens internally
response = client.chat.completions.create(
model='o3',
messages=[
{'role': 'user', 'content': 'Solve: A train leaves Chicago at 9am going 60mph. Another leaves NYC at 10am going 80mph. If the distance is 790 miles, when do they meet?'}
],
# reasoning_effort='high' # Optional: 'low', 'medium', 'high'
)
print(response.choices[0].message.content)
# Check how many tokens were used for thinking:
print('Input tokens:', response.usage.prompt_tokens)
print('Output tokens:', response.usage.completion_tokens)Claude Extended Thinking: API Behavior
Anthropic's Claude exposes extended thinking tokens via the API. You can stream and observe the model's reasoning process in real time. The thinking content appears before the final response.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-...')
# Enable extended thinking
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=16000,
thinking={
'type': 'enabled',
'budget_tokens': 10000 # Max tokens for thinking
},
messages=[{
'role': 'user',
'content': 'What is the 100th prime number?'
}]
)
# Response contains both thinking blocks and text blocks
for block in response.content:
if block.type == 'thinking':
print('THINKING:', block.thinking[:200], '...')
elif block.type == 'text':
print('ANSWER:', block.text)Streaming Thinking Tokens
You can stream extended thinking in real time, seeing the model's reasoning as it unfolds. This is useful for UX — showing a 'thinking' animation or letting advanced users observe the reasoning process.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-...')
def stream_with_thinking(question):
with client.messages.stream(
model='claude-opus-4-5',
max_tokens=16000,
thinking={'type': 'enabled', 'budget_tokens': 8000},
messages=[{'role': 'user', 'content': question}]
) as stream:
current_block_type = None
for event in stream:
# Track which block type we're in
if hasattr(event, 'type'):
if 'thinking' in str(event.type):
current_block_type = 'thinking'
elif 'text' in str(event.type):
current_block_type = 'text'
# Print text delta
if hasattr(event, 'delta') and hasattr(event.delta, 'text'):
prefix = '[THINK] ' if current_block_type == 'thinking' else '[ANS] '
print(prefix + event.delta.text, end='', flush=True)
stream_with_thinking('Explain why P != NP is unproven.')What Happens Internally: The Scratchpad
The model's internal thinking is a scratchpad where it can:
- Explore multiple approaches before committing
- Make calculations and verify them
- Identify its own errors and backtrack
- Consider edge cases
- Plan multi-step solutions
This internal deliberation is why reasoning models vastly outperform standard models on hard math, coding, and strategic planning — they essentially do a literature review before writing.
budget_tokens: Controlling Thinking Depth
budget_tokens (Claude) or reasoning_effort (OpenAI) controls how much thinking the model does. More thinking = higher accuracy on hard problems, but more cost and latency.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-...')
def ask_with_budget(question, budget):
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=budget + 2048, # must exceed budget_tokens
thinking={'type': 'enabled', 'budget_tokens': budget},
messages=[{'role': 'user', 'content': question}]
)
thinking_tokens = sum(
len(b.thinking.split()) * 1.3 # rough estimate
for b in r.content if b.type == 'thinking'
)
answer = next(b.text for b in r.content if b.type == 'text')
return answer, int(thinking_tokens)
# Same hard question with different budgets
q = 'Prove that the square root of 2 is irrational.'
ans_small, tok_small = ask_with_budget(q, 1024)
ans_large, tok_large = ask_with_budget(q, 8000)
print(f'Small budget ({tok_small} thinking tokens): {ans_small[:100]}')
print(f'Large budget ({tok_large} thinking tokens): {ans_large[:100]}')Temperature and Reasoning Models
Reasoning models behave differently with temperature settings:
- For OpenAI o-series: temperature defaults to 1 and cannot always be changed
- For Claude extended thinking: temperature is typically set to 1 during thinking
Don't try to use temperature to control reasoning model behavior — use budget_tokens or reasoning_effort instead.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-...')
# For Claude with extended thinking, temperature=1 is the default
# and best practice
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=10000,
temperature=1, # Required to be 1 when extended thinking is enabled
thinking={
'type': 'enabled',
'budget_tokens': 5000
},
messages=[{
'role': 'user',
'content': 'Write a Python function to find all prime numbers up to N.'
}]
)
print(response.content[-1].text[:300])Performance Benchmarks: Where Reasoning Wins
Reasoning models outperform standard models most dramatically on:
- Competition math: AIME, AMC (o3 near human expert)
- Complex coding: Competitive programming (Codeforces)
- Scientific reasoning: GPQA (graduate-level science)
- Multi-step logic: Problems requiring sequential deduction
On simple tasks (grammar, summarization, factual Q&A), standard models perform equally well at 10-100x lower cost.
The Latency Reality
Reasoning models are slow. A hard problem with high reasoning effort can take 30-60 seconds. Plan your UX accordingly:
- Show 'thinking...' progress indicators
- Use streaming to show partial results as available
- Don't use reasoning models for real-time chat where latency matters
- Pre-compute reasoning-model outputs for known hard questions
import time
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-...')
def timed_reasoning_call(question, budget):
start = time.time()
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=budget + 2000,
thinking={'type': 'enabled', 'budget_tokens': budget},
messages=[{'role': 'user', 'content': question}]
)
elapsed = time.time() - start
answer = next(b.text for b in response.content if b.type == 'text')
print(f'Latency: {elapsed:.1f}s | Answer: {answer[:100]}')
return answer
# Hard problem — expect 20-45 seconds
timed_reasoning_call(
'Design a database schema for a multi-tenant SaaS billing system.',
budget=8000
)Reasoning Models and System Prompts
Reasoning models respond to system prompts differently than standard models. Because they deliberate internally before answering, they can follow complex multi-step instructions in system prompts more reliably.
However, very long, constraining system prompts can conflict with internal reasoning. Best practice: keep system prompts concise for reasoning models — define the role and output format, then let the model's internal reasoning handle strategy.
Knowledge Check: Reasoning Model Thinking
What is the key difference between what the prompt author provides and what happens internally in a reasoning model?
Recap: How Reasoning Models Differ
Reasoning models like o1/o3 and Claude with extended thinking run an internal chain-of-thought before responding. The prompt author provides a problem; the model deliberates internally, exploring approaches and self-correcting, then produces a final answer. You control thinking depth with budget_tokens (Claude) or reasoning_effort (OpenAI). Reasoning models excel at complex math, coding, and multi-step logic but are 10-100x more expensive and 10-100x slower than standard models. Use streaming and progress indicators to manage latency in your UX.
Frequently asked questions
Is the “How Reasoning Models Differ” lesson free?
Yes — the full text of “How Reasoning Models Differ” is free to read here on the web, and the AI Prompt Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.
What will I learn in “How Reasoning Models Differ”?
Internal chain-of-thought vs standard models: what changes for the prompt author. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Prompt Engineering?
No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “How Reasoning Models Differ” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Prompt Engineering lesson?
Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- How Reasoning Models Differ
- Effective Prompts for Extended Thinking
- When to Use Reasoning vs Standard Models
- Cost and Latency Tradeoffs