基于评分标准的评分提示词
结构化评估标准：准确性、流畅性、相关性和安全性（1—5 分制）。
基于评分标准的评分提示词 是 CoddyKit 上的免费 AI Prompt Engineering 课时。 这是第 2 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 AI Prompt Engineering 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 AI Prompt Engineering 课程共包含 4 节课。
什么是基于评分标准的评分
基于评分标准的评分会向 LLM 评审模型提供一组结构化标准，并明确规定每个分数等级的定义。与其询问“这个回答有多好？”，不如询问“这个回答在以下每个具体维度上应获得多少分？”
评分标准可以减少偏差、提高一致性，并使评估结果更易于解释和执行。
评分标准的构成
设计良好的评分标准包含三个组成部分：
- 标准名称：需要评估的维度（准确性、完整性、清晰度）
- 分数锚点：明确规定该标准下每个分数的含义
- 权重或优先级：哪些标准对于当前使用场景最重要
每个组成部分都会让评审模型的工作受到更多约束，并且更具可复现性。
三项标准评分提示词
下面是一个具体的评分标准提示词模板，用于从准确性、完整性和清晰度方面评估人工智能的回答。评审模型会返回包含各项标准分数的结构化 JSON。
import anthropic
import json
client = anthropic.Anthropic(api_key='sk-ant-...')
RUBRIC_PROMPT = (
'Score this response on a scale of 1-5 for each criterion:\n\n'
'ACCURACY: Is the response factually correct?\n'
' 1=Contains significant factual errors\n'
' 3=Mostly correct with minor inaccuracies\n'
' 5=Completely accurate with no errors\n\n'
'COMPLETENESS: Did it answer everything asked?\n'
' 1=Missed most of the question\n'
' 3=Answered the main question but missed sub-parts\n'
' 5=Addressed every part of the question\n\n'
'CLARITY: Is it easy to understand?\n'
' 1=Confusing, hard to follow\n'
' 3=Understandable but could be clearer\n'
' 5=Exceptionally clear and well-organized\n\n'
'Question: {question}\n'
'Response: {response}\n\n'
'Return JSON: {{"accuracy": N, "completeness": N, "clarity": N, '
'"overall": N, "notes": "one sentence"}}'
)
def rubric_judge(question, response):
prompt = RUBRIC_PROMPT.format(question=question, response=response)
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=200,
messages=[{'role': 'user', 'content': prompt}]
)
return json.loads(r.content[0].text)
result = rubric_judge(
question='What is a REST API?',
response='A REST API is a way for applications to communicate over HTTP.'
)
print(result)加权评分
并非所有标准都同等重要。对于客户支持机器人而言，帮助性比文学风格更重要。加权评分可以在最终分数计算中体现这些优先级。
import anthropic
import json
client = anthropic.Anthropic(api_key='sk-ant-...')
def weighted_rubric_judge(question, response, weights):
"""
weights: dict of criterion -> weight (should sum to 1.0)
Example: {'accuracy': 0.5, 'completeness': 0.3, 'clarity': 0.2}
"""
RUBRIC = (
'Score this response 1-5 on:\n'
'Accuracy: Is it factually correct?\n'
'Completeness: Does it cover the full question?\n'
'Clarity: Is it easy to understand?\n\n'
'Q: {q}\nA: {a}\n\n'
'Return JSON: {{"accuracy":N,"completeness":N,"clarity":N}}'
)
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=100,
messages=[{'role': 'user', 'content': RUBRIC.format(q=question, a=response)}]
)
scores = json.loads(r.content[0].text)
# Calculate weighted average
weighted_score = sum(
scores[criterion] * weight
for criterion, weight in weights.items()
if criterion in scores
)
print(f'Individual scores: {scores}')
print(f'Weighted score: {weighted_score:.2f}/5')
return weighted_score
weighted_rubric_judge(
'How do I reverse a string in Python?',
'Use slicing: s[::-1]',
weights={'accuracy': 0.5, 'completeness': 0.3, 'clarity': 0.2}
)评判标准：事实准确性
对于知识密集型任务，事实准确性是最关键的标准。专门的准确性评分标准会要求评审模型重点检查错误事实、过时信息和缺乏依据的声明。
ACCURACY_RUBRIC = (
'Evaluate FACTUAL ACCURACY of the following response.\n\n'
'Score 1-5:\n'
'1 = Multiple factual errors that fundamentally mislead the reader\n'
'2 = At least one significant factual error (wrong date, number, or core fact)\n'
'3 = Factually correct but includes minor imprecisions or over-generalizations\n'
'4 = Factually correct with appropriate hedging of uncertain claims\n'
'5 = Factually precise, no errors, and correctly acknowledges uncertainty where present\n\n'
'Check specifically for:\n'
'- Wrong dates, statistics, or numerical values\n'
'- Misattributed quotes or inventions\n'
'- Outdated information presented as current\n'
'- Claims stated with false confidence (should be hedged)\n\n'
'Q: {question}\nA: {response}\n\n'
'Score and list any errors found:'
)
print(ACCURACY_RUBRIC[:400])评判标准：完整性
完整性用于检查回答是否涵盖多部分问题的每个部分。该标准可以发现只回答第一个问题却忽略后续问题的回答，也可以发现问题要求具体内容时却只提供概括性回答的情况。
COMPLETENESS_RUBRIC = (
'Evaluate COMPLETENESS of this response.\n\n'
'First, list every distinct question or requirement in the original query.\n'
'Then, check whether the response addressed each one.\n\n'
'Score 1-5:\n'
'1 = Only addressed 0-20% of what was asked\n'
'2 = Addressed 20-50% — missed major components\n'
'3 = Addressed 50-80% — answered main question but missed sub-parts\n'
'4 = Addressed 80-95% — minor omissions only\n'
'5 = Addressed 100% — every requirement was met\n\n'
'Q: {question}\nA: {response}\n\n'
'Requirements checklist and completeness score:'
)
# This rubric forces the judge to decompose the question first,
# which is much more reliable than asking 'was it complete?'
print(COMPLETENESS_RUBRIC[:400])评判标准：清晰度
清晰度评估会检查可读性、结构，以及回答是否真正向目标受众传达了其含义。该标准可以发现技术上正确但解释不佳的回答。
CLARITY_RUBRIC = (
'Evaluate CLARITY of this response for a {audience} audience.\n\n'
'Score 1-5:\n'
'1 = Incomprehensible — cannot extract meaning\n'
'2 = Very hard to follow — excessive jargon, poor structure\n'
'3 = Understandable with effort — some confusing parts\n'
'4 = Clear and well-organized — easy to read\n'
'5 = Exceptionally clear — ideal structure, appropriate vocabulary, '
'no unnecessary complexity\n\n'
'Consider:\n'
'- Is the vocabulary appropriate for the audience?\n'
'- Is the response logically organized?\n'
'- Are sentences a readable length?\n'
'- Is the main point stated early and clearly?\n\n'
'Q: {question}\nA: {response}\n\n'
'Clarity score and key issues:'
)
# Parameterize the audience for context-aware clarity assessment
print(CLARITY_RUBRIC.format(
audience='non-technical business stakeholder',
question='What is an API?',
response='REST APIs use HTTP to transfer data between client and server.'
)[:300])任务专用评分标准
通用的准确性、完整性和清晰度评分标准适用范围较广，但任务专用评分标准能够为专业使用场景提供更好的评估。请根据应用真正重视的内容定制标准。
# Customer support response rubric
SUPPORT_RUBRIC = (
'Evaluate this customer support response:\n\n'
'EMPATHY (1-5): Does it acknowledge the customer emotion?\n'
'RESOLUTION (1-5): Does it provide a clear solution or next step?\n'
'TONE (1-5): Is it professional, warm, and not condescending?\n'
'EFFICIENCY (1-5): Does it avoid unnecessary words or boilerplate?\n\n'
'Customer message: {customer_message}\n'
'Support response: {support_response}\n\n'
'Scores and notes (JSON):'
)
# Code review rubric
CODE_REVIEW_RUBRIC = (
'Evaluate this code explanation:\n\n'
'CORRECTNESS (1-5): Is the code technically correct?\n'
'EDGE_CASES (1-5): Does it handle edge cases (empty input, errors)?\n'
'EFFICIENCY (1-5): Is it reasonably efficient (no obvious O(n^2) where O(n) is easy)?\n'
'READABILITY (1-5): Is the code easy to read and understand?\n\n'
'Task: {task}\n'
'Code: {code}\n\n'
'Scores and notes (JSON):'
)
print('Specialized rubrics produce better signal for your domain')评分标准一致性测试
通过使用略有不同的提示词，将同一个回答发送五次，并检查分数是否保持稳定，来测试您的评分标准是否一致。高方差意味着评分标准的规定不够明确。
import anthropic
import json
import statistics
client = anthropic.Anthropic(api_key='sk-ant-...')
def test_rubric_consistency(rubric_prompt, question, response, n_trials=5):
scores = []
for i in range(n_trials):
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=100,
messages=[{'role': 'user', 'content': rubric_prompt.format(
question=question, response=response
)}]
)
try:
data = json.loads(r.content[0].text)
overall = data.get('overall', sum(data.values()) / len(data))
scores.append(overall)
except Exception:
scores.append(None)
valid = [s for s in scores if s is not None]
if valid:
print(f'Scores: {valid}')
print(f'Mean: {statistics.mean(valid):.2f}')
print(f'Std dev: {statistics.stdev(valid):.2f}')
if statistics.stdev(valid) > 0.5:
print('WARNING: High variance — rubric may be underspecified')
return valid让评审模型返回 JSON
请始终要求评分标准评审模型返回结构化 JSON。这样可以让分数被机器读取，支持自动汇总，并防止评审模型把重要细微差异隐藏在您的流程会忽略的自由文本中。
import anthropic
import json
client = anthropic.Anthropic(api_key='sk-ant-...')
def structured_rubric_judge(question, response):
prompt = (
'Score this response 1-5 on three criteria and return JSON.\n\n'
'Q: {q}\nA: {a}\n\n'
'Return ONLY this JSON structure (no other text):\n'
'{{\n'
' "accuracy": <1-5>,\n'
' "completeness": <1-5>,\n'
' "clarity": <1-5>,\n'
' "overall": <1-5>,\n'
' "primary_issue": "<what most needs improvement>",\n'
' "primary_strength": "<what the response does best>"\n'
'}}'
).format(q=question, a=response)
r = client.messages.create(
model='claude-opus-4-5',
max_tokens=200,
messages=[{'role': 'user', 'content': prompt}]
)
text = r.content[0].text.strip()
# Strip markdown code fences if present
if text.startswith('###'):
text = text.split('###')[1]
if text.startswith('json'):
text = text[4:]
return json.loads(text.strip())
result = structured_rubric_judge(
'Explain big O notation.',
'Big O describes algorithm time complexity.'
)
print(json.dumps(result, indent=2))迭代评分标准设计
评分标准需要经过迭代才能表现良好。请从一个简单的三项标准评分标准开始，在 20—30 个 test 用例上运行，然后与人工评分进行比较。在评审模型与人工评估者分歧最大的地方改进标准定义。
常见的改进需求包括：准确性标准范围过于宽泛（拆分为事实准确性和逻辑一致性）；清晰度标准混淆了可读性和简洁性（将二者分开）；或者分数等级 3 的锚定不佳（大多数回答模糊地集中在这一等级）。
知识检查：评分锚点
为什么评分标准应明确说明每个分数等级的含义（分数锚点）？
回顾：基于评分标准的评分提示词
基于评分标准的评分会根据多个命名标准（准确性、完整性、清晰度）评估回答，并使用明确的分数锚点规定每个分数等级的含义。锚点可以减少评分膨胀并提高一致性。请使用加权评分，优先考虑对当前使用场景最重要的标准。对于专业应用，任务专用评分标准（客服、代码、创意）优于通用评分标准。请始终让评审模型返回 JSON，以获得机器可读取的结果。通过多次运行同一项评估来测试评分标准的一致性——高标准差表明评分标准的规定不够明确，需要进一步改进。
常见问题解答
「基于评分标准的评分提示词」课时是免费的吗？
是的 — 「基于评分标准的评分提示词」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 AI Prompt Engineering 课程的其余内容，请升级到 CoddyKit PRO。 AI Prompt Engineering 课程共包含 4 节课。
「基于评分标准的评分提示词」这节课中我会学到什么？
结构化评估标准：准确性、流畅性、相关性和安全性（1—5 分制）。 你通过在浏览器中直接运行的动手代码来练习 AI Prompt Engineering，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 AI Prompt Engineering 需要有经验吗？
无需任何先前经验。CoddyKit 上的 AI Prompt Engineering 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 2 节课，共 4 节。
「基于评分标准的评分提示词」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 AI Prompt Engineering 课中编写并运行代码吗？
能。每节 AI Prompt Engineering 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。
此课程中的所有课时
- 使用 LLM 评估 LLM 输出
- 基于评分标准的评分提示词
- 比较式评审：A 与 B
- LLM 评审中的校准与偏差