Claude Architect · 课时

使用示例进行迭代改进

提供 2–4 个输入/输出示例，并进行测试驱动的迭代。

第 4 / 4 课13 个步骤

使用示例进行迭代改进 是 CoddyKit 上的免费 Claude Architect 课时。这是第 4 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Claude Architect 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Claude Architect 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Why Examples Beat Adjectives

When a prompt under-performs, architects reach for vague fixes like "be more precise" or "try harder". These rarely move the needle. The reliable lever is explicit criteria plus concrete examples.

Compare: "be more precise" versus "flag a comment only when it contradicts the code". The second tells the model exactly where the decision boundary sits. In this lesson you'll learn to drive a prompt to production quality using 2-4 targeted input/output examples and a tight test-and-iterate loop.

How Few-Shot Actually Works

Few-shot prompting attaches a small set of worked examples to your instructions. The key insight: the model generalizes from the examples — it does not merely copy them. Given 3 representative cases, it infers the underlying rule and applies it to unseen inputs.

Few-shot is strongest for four jobs:

Consistency across many calls
Edge cases that words alone describe poorly
Output format the model should mirror
Reducing hallucination by anchoring behavior

Aim for 2-4 examples per ambiguity — enough to define the pattern, few enough to keep context lean.

Anatomy of a Good Example

An effective few-shot example is a paired input and the exact output you want back. The output must match the real schema or shape you'll consume in production — same fields, same casing, same structure.

Below, each example shows an input comment plus the precise verdict. The model learns the boundary: flag only genuine contradictions, not style nits.

examples = [
    {
        "input": "# returns the user's age\n def get_name(u): return u.name",
        "output": {"flag": True, "reason": "comment says age, code returns name"},
    },
    {
        "input": "# sort ascending\n items.sort()",
        "output": {"flag": False, "reason": "comment matches behavior"},
    },
]

system = (
    "Flag a comment ONLY when it contradicts the code. "
    "Style or wording issues are not contradictions.\n\n"
    "Examples:\n" + "\n".join(
        f"INPUT: {e['input']}\nOUTPUT: {e['output']}" for e in examples
    )
)

Pair Criteria With Examples

Examples and explicit criteria are partners, not substitutes. Criteria state the rule; examples calibrate the gray zone the rule can't fully capture in prose.

A common architect mistake is dumping examples with no governing instruction. The model then over-fits to surface features of the samples. Always lead with a crisp criterion — "flag only when X contradicts Y" — then let 2-4 examples pin down where X and Y blur.

Rule of thumb: if you can't write the criterion in one sentence, your task is still under-specified, and more examples won't fix that.

Build a Test Set First

Iterative refinement is test-driven. Before tuning the prompt, assemble a small labeled set of representative inputs with known-correct outputs. This is your ground truth — every prompt change is judged against it, not against a gut feeling.

Keep the test set separate from your few-shot examples. If you tune on the same cases you teach with, you're memorizing, not generalizing.

test_cases = [
    {"input": "# deletes the record\n def archive(r): r.archived = True",
     "expected": {"flag": True}},
    {"input": "# cache result for 60s\n cache.set(k, v, ttl=60)",
     "expected": {"flag": False}},
    {"input": "# returns count\n def total(rows): return sum(r.amt for r in rows)",
     "expected": {"flag": True}},
]

The Iteration Loop

The refinement loop is mechanical and repeatable:

Run the prompt over every test case
Compare output to the expected label
Inspect the failures — what boundary did the model miss?
Add or sharpen one example (or one criterion) that targets that failure
Re-run the full set and check nothing regressed

Change one thing per iteration. Batch edits make it impossible to know which tweak helped or hurt.

Scoring the Test Set

Automate the comparison so iteration is fast. A tiny harness runs each case, scores it, and prints the misses. With structured output you can compare fields directly instead of parsing prose.

Note tool_choice below: forcing a tool call guarantees the model returns schema-valid JSON every time, so your scorer never trips over free-text.

def score(client, system, cases):
    misses = []
    for c in cases:
        resp = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=256,
            system=system,
            tools=[verdict_tool],
            tool_choice={"type": "any"},  # must call a tool -> structured output
            messages=[{"role": "user", "content": c["input"]}],
        )
        out = resp.content[0].input
        if out["flag"] != c["expected"]["flag"]:
            misses.append((c["input"], out))
    return misses

Target Failures With New Examples

When you read a miss, ask: which ambiguity caused it? Then add an example that lives exactly on that boundary — not a random new case. One sharp example per failure mode generalizes far better than ten generic ones.

Suppose the model wrongly flagged a comment that paraphrased the code. You'd add a paired example showing paraphrase = not a contradiction. The model updates its internal boundary and the whole class of similar misses clears.

Resist the urge to keep piling on examples. Past 4-ish per ambiguity you bloat context and risk lost-in-the-middle, where the model under-attends to the center of a long prompt.

# Add ONE example aimed at the observed miss:
examples.append({
    "input": "# loop over each item\n for x in items: process(x)",
    "output": {"flag": False,
               "reason": "paraphrase of code, not a contradiction"},
})

Format Errors? Retry With Feedback

Some failures are format or structural, not reasoning errors — a malformed field, a bad arithmetic total, a missing bracket. For these, retry-with-feedback is the fix: send back the original input, the wrong output, and the exact validation error.

Crucial limit: retry helps when the model can produce the right answer but slipped. It does not help when the required information is simply absent from the source — no amount of retrying invents data that isn't there.

def retry_with_feedback(client, system, original, bad_output, error):
    return client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=512,
        system=system,
        messages=[
            {"role": "user", "content": original},
            {"role": "assistant", "content": str(bad_output)},
            {"role": "user", "content":
             f"That output failed validation: {error}. "
             "Return corrected output that satisfies the schema."},
        ],
    )

Lock Quality With Structured Output

Once examples have shaped the behavior, lock the shape with a JSON Schema via tool use. This eliminates syntax errors and enforces required fields. One discipline matters most: mark a field required only if it is always present.

Never require a field that might be absent — the model will fabricate a value to satisfy the schema. For optional or open-ended data, use an enum with an "other" value plus a free-text detail field, which keeps the contract extensible without forcing hallucination.

verdict_tool = {
    "name": "record_verdict",
    "description": "Record whether a comment contradicts its code.",
    "input_schema": {
        "type": "object",
        "properties": {
            "flag": {"type": "boolean"},
            "category": {"type": "string",
                         "enum": ["contradiction", "style", "other"]},
            "detail": {"type": "string"},
        },
        "required": ["flag"],  # only the always-present field
    },
}

Validate With a Fresh Reviewer

Before you trust the refined prompt, validate it with an independent, fresh instance — not the same session that produced the output. A same-session self-review is biased: the author retains its own reasoning and won't challenge itself.

And don't trust an aggregate score alone. A headline like 97% accuracy can hide a field or input type that's failing badly. Use stratified sampling and field-level confidence, calibrated on a labeled validation set, before you automate. Refinement isn't done when the average looks good — it's done when every slice clears the bar.

Quick Check: Fixing a Failing Prompt

A classifier prompt passes 92% of your labeled test set but consistently mislabels comments that paraphrase the code as contradictions. Which refinement move is most effective?

在交互式课程中回答

Recap: Refine With Examples, Prove With Tests

Key takeaways:

Examples beat adjectives. Explicit criteria plus 2-4 targeted examples outperform vague instructions like 'be more precise'.
The model generalizes from few-shot examples — best for consistency, edge cases, output format, and reducing hallucination.
Be test-driven: build a labeled set, score it, add one boundary example per failure mode, re-run, change one thing at a time.
Retry-with-feedback fixes format errors (send original + wrong output + exact error) — not missing-source data.
Lock the shape with a JSON Schema; require only always-present fields; use enum + 'other' + detail for extensibility.
Validate independently: fresh-instance review beats same-session self-review, and stratified field-level checks beat aggregate-only accuracy.

免费开始

用 AI 导师学习 Python — 免费

在浏览器中编写并运行真实代码，获得全天候 AI 导师的即时帮助，并在网页或应用中继续学习。

课程: 26
课程: 104

免费开始学习查看完整课程

常见问题解答

「使用示例进行迭代改进」课时是免费的吗？

是的 — 「使用示例进行迭代改进」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Claude Architect 课程的其余内容，请升级到 CoddyKit PRO。 Claude Architect 课程共包含 4 节课。

「使用示例进行迭代改进」这节课中我会学到什么？

提供 2–4 个输入/输出示例，并进行测试驱动的迭代。你通过在浏览器中直接运行的动手代码来练习 Claude Architect，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Claude Architect 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Claude Architect 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 4 节课，共 4 节。

「使用示例进行迭代改进」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Claude Architect 课中编写并运行代码吗？

能。每节 Claude Architect 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Claude Architect