Chain-of-Thought Prompting
Eliciting step-by-step reasoning.
Reasoning as Token Budget
Chain-of-thought (CoT) prompting elicits intermediate reasoning steps before the final answer. The key insight: transformers do a fixed amount of computation per token, so allowing the model to emit reasoning tokens effectively grants it more serial compute to reach the answer.
Forcing an immediate answer caps the compute available for a hard problem; CoT removes that cap by spreading the computation across generated tokens.
# Direct: model must compute everything before the first token
DIRECT = 'Q: ' + q + '\nA:'
# CoT: model can use tokens as scratch space
COT = 'Q: ' + q + '\nA: Let us reason step by step.'Zero-Shot CoT
The famous trigger phrase Let us think step by step (Kojima et al., 2022) elicits reasoning with no examples. It works because instruction-tuned models have internalized the pattern that such a phrase precedes a worked solution.
Zero-shot CoT is cheap and surprisingly effective, but its reasoning quality is less controllable than few-shot CoT with curated exemplars.
def zero_shot_cot(question):
stage1 = llm('Q: ' + question + '\nA: Let us think step by step.')
# Extract the final answer in a second, constrained call
stage2 = llm(stage1 + '\nTherefore, the final answer is:')
return parse_answer(stage2)All lessons in this course
- Chain-of-Thought Prompting
- Self-Consistency Sampling
- Tree-of-Thought Exploration
- When Reasoning Prompts Help