Calibration and Bias in LLM Judges
Position bias, verbosity bias, and how to mitigate them in judge prompts.
Calibration and Bias in LLM Judges is a free AI Prompt Engineering lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Judge Calibration Matters
An LLM judge that systematically scores one type of response higher than it deserves produces misleading evaluation results. You might ship a worse model because the judge preferred its verbose style — not its actual quality.
Calibration means the judge's scores accurately reflect true quality. A calibrated judge agrees with human raters at a measurable rate and doesn't systematically favor any one attribute unrelated to quality.
Position Bias: Deep Dive
Position bias is the strongest and most studied LLM judge bias. In pairwise comparison, judges prefer the first option 60-65% of the time independent of quality. This is equivalent to a coin that comes up heads 60% of the time — significant at scale.
The bias exists because LLMs are trained to generate continuations — seeing 'Response A:' first primes them toward A before they read B.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-...')
def measure_position_bias(question, n_pairs=20):
"""
Measure position bias by comparing IDENTICAL responses.
If both responses are the same, wins should be 50/50.
Any deviation from 50/50 is pure position bias.
"""
response = 'Machine learning is a subset of AI that learns from data.'
first_wins = 0
for _ in range(n_pairs):
prompt = (
f'Which response is better?\nQ: {question}\n'
f'Response A: {response}\n'
f'Response B: {response}\n'
f'Reply with A or B.'
)
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=5,
messages=[{'role': 'user', 'content': prompt}]
)
if 'A' in r.content[0].text:
first_wins += 1
bias = first_wins / n_pairs
print(f'First-position win rate with IDENTICAL responses: {bias:.0%}')
print(f'Expected (no bias): 50%')
print(f'Measured bias: {(bias - 0.5) * 100:+.0f}%')
return biasVerbosity Bias: Deep Dive
Verbosity bias causes judges to prefer longer responses even when shorter ones are more accurate and complete. Research shows LLM judges rate longer responses 1.5-2x more often as 'better' in pairwise comparisons, controlling for content quality.
This bias likely stems from training data where human raters also conflate length with quality.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-...')
def measure_verbosity_bias(question, correct_answer):
"""
Compare a concise correct answer against a verbose one with filler.
A good judge should prefer the concise version or call it a tie.
"""
concise = correct_answer
verbose = (
f'That is a great question! I am happy to help. '
f'{correct_answer} '
f'I hope this comprehensive explanation addresses all your needs. '
f'Please feel free to ask if you need any further clarification!'
)
for label, resp in [('Concise', concise), ('Verbose', verbose)]:
prompt = (
f'Rate this response 1-5 for quality.\n'
f'Q: {question}\nA: {resp}\n'
f'Return only a number 1-5.'
)
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=5,
messages=[{'role': 'user', 'content': prompt}]
)
score = r.content[0].text.strip()
print(f'{label} response score: {score}')
print(f' ({len(resp.split())} words)')Self-Preference Bias: Deep Dive
Claude judges rate Claude-generated responses higher on average. GPT-4 judges rate GPT-4-generated responses higher. This has been demonstrated in multiple independent studies.
The mechanism: each model has a distinctive style — sentence structure, hedging patterns, vocabulary choices. The same model recognizes and prefers its own style.
import anthropic
import openai
anthropic_client = anthropic.Anthropic(api_key='sk-ant-...')
openai_client = openai.OpenAI(api_key='sk-...')
def test_self_preference(question):
# Generate one response per model
claude_answer = anthropic_client.messages.create(
model='claude-opus-4-5', max_tokens=100,
messages=[{'role': 'user', 'content': question}]
).content[0].text
gpt_answer = openai_client.chat.completions.create(
model='gpt-4o', max_tokens=100,
messages=[{'role': 'user', 'content': question}]
).choices[0].message.content
judge_prompt = (
f'Which response is better?\nQ: {question}\n'
f'Response A: {claude_answer}\n'
f'Response B: {gpt_answer}\n'
f'Reply: A or B'
)
# Claude judges the comparison
claude_verdict = anthropic_client.messages.create(
model='claude-opus-4-5', max_tokens=5,
messages=[{'role': 'user', 'content': judge_prompt}]
).content[0].text.strip()
print(f'Claude judge verdict: {claude_verdict}')
print('(A=Claude response, B=GPT response)')
print('Self-preference: did Claude prefer its own response?')Mitigation 1: Swap Order
The single most effective mitigation for position bias: run every pairwise comparison twice with swapped order and use only consistent results. Implement this as a standard function that all evaluations pass through.
import anthropic
import json
client = anthropic.Anthropic(api_key='sk-ant-...')
def debiased_compare(question, response_a, response_b, label_a='A', label_b='B'):
def single_pass(first, second, first_label, second_label):
prompt = (
f'Q: {question}\n\n'
f'{first_label}: {first}\n\n'
f'{second_label}: {second}\n\n'
f'Which is better? Reply with {first_label}, {second_label}, or TIE.'
)
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=10,
messages=[{'role': 'user', 'content': prompt}]
)
return r.content[0].text.strip()
# Pass 1: A first
p1 = single_pass(response_a, response_b, label_a, label_b)
# Pass 2: B first — but labels stay the same (we just swap presentation order)
p2 = single_pass(response_b, response_a, label_b, label_a)
# Normalize p2: if judge said label_b in pass 2, that means they preferred first-shown
# Need to map back: if p2 = label_b, the 'first' won; if p2 = label_a, the 'second' won
if p1 == p2 and p1 != 'TIE':
return p1, 'Consistent result'
else:
return 'TIE', f'Inconsistent: pass1={p1}, pass2={p2}'
winner, reason = debiased_compare(
'What is Docker?',
'Docker is a containerization platform.',
'Docker allows you to package applications into containers for consistent deployment.'
)
print(f'{winner}: {reason}')Mitigation 2: Multiple Diverse Judges
Self-preference bias is reduced by using multiple different models as judges and aggregating their verdicts. When Claude, GPT-4, and Gemini all agree, the result is much more trustworthy than any single model's verdict.
import anthropic
import openai
anthropic_client = anthropic.Anthropic(api_key='sk-ant-...')
openai_client = openai.OpenAI(api_key='sk-...')
def multi_model_pairwise(question, response_a, response_b):
results = {}
# Judge 1: Claude
r1 = anthropic_client.messages.create(
model='claude-opus-4-5', max_tokens=5,
messages=[{'role': 'user', 'content':
f'Q: {question}\nA: {response_a}\nB: {response_b}\n'
f'Which is better? Reply A, B, or TIE.'
}]
)
results['Claude'] = r1.content[0].text.strip()
# Judge 2: GPT-4o
r2 = openai_client.chat.completions.create(
model='gpt-4o', max_tokens=5,
messages=[{'role': 'user', 'content':
f'Q: {question}\nA: {response_a}\nB: {response_b}\n'
f'Which is better? Reply A, B, or TIE.'
}]
)
results['GPT4o'] = r2.choices[0].message.content.strip()
print(f'Claude judge: {results["Claude"]}')
print(f'GPT-4o judge: {results["GPT4o"]}')
# Aggregate: majority vote
votes = list(results.values())
if votes.count('A') >= 2: return 'A'
if votes.count('B') >= 2: return 'B'
return 'TIE'
final = multi_model_pairwise(
'Explain recursion.',
'A function that calls itself.',
'Recursion is when a function solves a problem by solving a smaller version of the same problem.'
)
print(f'Final verdict: {final}')Mitigation 3: Anti-Verbosity Instructions
Directly instruct the judge to penalize verbosity and reward conciseness. This counteracts the verbosity bias that would otherwise make longer responses appear better.
ANTI_VERBOSITY_JUDGE = (
'Evaluate this response. Apply these corrections for known judge biases:\n\n'
'VERBOSITY CORRECTION: Do NOT rate a response higher simply because it is longer. '
'A concise, accurate 20-word answer is better than a 200-word answer that says the same thing. '
'Actively penalize unnecessary padding, filler phrases like "Great question!", '
'and repetition.\n\n'
'LENGTH PENALTY: If the response contains introductory filler, closing remarks, '
'or restates the question, subtract 1 point from your score.\n\n'
'Question: {question}\n'
'Response: {response}\n\n'
'Score 1-5 (apply verbosity correction above):'
)
# This instruction significantly reduces verbosity inflation in practice
print('Anti-verbosity instructions reduce the length-quality conflation')Calibrating Against Human Raters
The gold standard for judge calibration: compare your LLM judge's scores against human rater scores on a calibration set. Measure agreement and identify systematic divergences.
import anthropic
import json
from scipy import stats # pip install scipy
client = anthropic.Anthropic(api_key='sk-ant-...')
def calibrate_judge(calibration_set):
"""
calibration_set: list of dicts with:
{'question': str, 'response': str, 'human_score': float}
"""
llm_scores = []
human_scores = []
for item in calibration_set:
prompt = (
f'Rate this response 1-5.\n'
f'Q: {item["question"]}\nA: {item["response"]}\n'
f'Return only a number.'
)
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=5,
messages=[{'role': 'user', 'content': prompt}]
)
try:
llm_score = float(r.content[0].text.strip())
except ValueError:
llm_score = 3.0 # Default on parse error
llm_scores.append(llm_score)
human_scores.append(item['human_score'])
correlation, p_value = stats.pearsonr(llm_scores, human_scores)
print(f'LLM-Human correlation: {correlation:.3f} (p={p_value:.4f})')
print(f'LLM mean score: {sum(llm_scores)/len(llm_scores):.2f}')
print(f'Human mean score: {sum(human_scores)/len(human_scores):.2f}')
return correlationDetecting Systematic Bias Patterns
Analyze your judge's output across different response categories to detect systematic bias. Does the judge consistently score responses from one model higher? Does it penalize responses on certain topics?
import json
from collections import defaultdict
def analyze_judge_bias(log_file='pairwise_log.jsonl'):
"""
Analyze pairwise logs to detect systematic bias.
"""
wins_by_label = defaultdict(int)
total_by_label = defaultdict(int)
with open(log_file) as f:
for line in f:
entry = json.loads(line)
winner = entry['winner']
label_a = entry['label_a']
label_b = entry['label_b']
if winner == 'A':
wins_by_label[label_a] += 1
elif winner == 'B':
wins_by_label[label_b] += 1
total_by_label[label_a] += 1
total_by_label[label_b] += 1
print('Win rate by model:')
for label in sorted(total_by_label):
total = total_by_label[label]
wins = wins_by_label[label]
print(f' {label}: {wins}/{total} = {wins/total:.0%}')
# Flag if any model wins >60% — likely systematic bias
for label in total_by_label:
rate = wins_by_label[label] / total_by_label[label]
if rate > 0.65 or rate < 0.35:
print(f'WARNING: {label} win rate {rate:.0%} suggests systematic bias')Bias Mitigation Checklist
A checklist to apply before deploying an LLM judge in production:
- Run all pairwise comparisons twice with swapped order
- Include explicit anti-verbosity instructions in the rubric
- Use at least 2 different models as judges for high-stakes evaluations
- Anchor score levels explicitly to prevent inflation
- Calibrate against human raters on a representative sample
- Log and monitor win rates by model label to detect drift
- Add structured JSON output to prevent free-text score hiding
Audit Trails and Explainability
A calibrated judge must also be explainable. If the judge awards a score of 4/5 to a response, you should be able to trace why — which criteria were satisfied, which fell short.
Requiring the judge to return JSON with per-criterion scores and a brief reason creates a natural audit trail. Stakeholders can review why specific responses scored as they did, and you can spot recurring patterns in low scores.
Knowledge Check: Self-Preference Mitigation
Which mitigation strategy MOST directly addresses self-preference bias in LLM judges?
Recap: Calibration and Bias in LLM Judges
LLM judges have four main systematic biases: position bias (prefer first option), verbosity bias (prefer longer responses), self-preference (prefer their own style), and scoring inflation (cluster at 4-5/5). Mitigate each specifically: swap order for position bias, add anti-verbosity instructions for verbosity bias, use diverse model judges for self-preference, and use anchored rubrics for inflation. Calibrate your judge against human raters on a labeled set and measure Pearson correlation. Monitor win rates by label over time to catch emerging bias patterns before they distort your evaluation pipeline.
Frequently asked questions
Is the “Calibration and Bias in LLM Judges” lesson free?
Yes — the full text of “Calibration and Bias in LLM Judges” is free to read here on the web, and the AI Prompt Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.
What will I learn in “Calibration and Bias in LLM Judges”?
Position bias, verbosity bias, and how to mitigate them in judge prompts. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Prompt Engineering?
No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Calibration and Bias in LLM Judges” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Prompt Engineering lesson?
Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Using LLM to Evaluate LLM Outputs
- Rubric-Based Scoring Prompts
- Comparative Judging: A vs B
- Calibration and Bias in LLM Judges