Root Cause Analysis for Prompts
Isolating whether failure is in context, instruction, format, or model capability.
Why Root Cause Analysis Matters
When a prompt fails, there are multiple possible causes. Randomly tweaking wording wastes time and may fix the symptom without fixing the root cause — leading to the same failure on slightly different inputs.
Root cause analysis (RCA) is a systematic process to isolate the specific reason a prompt fails, so the fix addresses the actual problem.
The Four Root Cause Categories
Every prompt failure traces back to one of four root causes:
- Context problem: the model lacks information needed to answer correctly
- Instruction ambiguity: the instruction has multiple valid interpretations, and the model chose the wrong one
- Format conflict: two parts of the prompt give contradictory formatting instructions
- Model capability limit: the task requires reasoning or knowledge beyond what this model can reliably do
All lessons in this course
- Diagnosing Unexpected Outputs
- Root Cause Analysis for Prompts
- Systematic Debugging Approach
- Logging and Documentation Strategies