Million-Token Context Windows
Opportunities and pitfalls.
What a Million Tokens Buys
Million-token windows let you place entire codebases, document corpora, or multi-hour transcripts directly in context — no retrieval system required. The promise is in-context everything: the model reasons over the raw material rather than over a lossy summary.
- Whole-repository reasoning and refactors.
- Cross-document synthesis without chunking.
- Long-horizon conversations that retain early detail.
But capacity is not the same as effective use.
Capacity vs Effective Context
The advertised window is an upper bound, not a guarantee of uniform recall. Effective context — the span over which the model reliably retrieves and reasons — is usually smaller and uneven across positions.
Treat the window as a budget with non-uniform value: tokens near the edges are recalled better than tokens buried in the middle.
All lessons in this course
- Million-Token Context Windows
- Lost in the Middle
- Structuring Huge Prompts
- Caching Long Prefixes