Rubric-Based Scoring Prompts
Structured evaluation criteria: accuracy, fluency, relevance, safety (1-5 scales).
What Is Rubric-Based Scoring?
Rubric-based scoring gives the LLM judge a structured set of criteria with explicit definitions for each score level. Instead of asking 'how good is this?', you ask 'how does this score on each of these specific dimensions?'
Rubrics reduce bias, increase consistency, and make evaluation results interpretable and actionable.
Anatomy of a Scoring Rubric
A well-designed rubric has three components:
- Criteria names: What dimensions to evaluate (Accuracy, Completeness, Clarity)
- Score anchors: Explicit definitions of what each score means for that criterion
- Weight or priority: Which criteria matter most for this use case
Each component makes the judge's work more constrained and reproducible.
All lessons in this course
- Using LLM to Evaluate LLM Outputs
- Rubric-Based Scoring Prompts
- Comparative Judging: A vs B
- Calibration and Bias in LLM Judges