예시를 활용한 반복적 개선
입력/출력 예시 2~4개와 테스트 주도 반복을 활용합니다
예시를 활용한 반복적 개선은(는) CoddyKit의 무료 Claude Architect 강의입니다. 이것은 4개 중 4번째 강의입니다. 아래에서 전체 강의를 무료로 읽을 수 있으며, 내장 코드 에디터와 24/7 AI 튜터와 함께 브라우저에서 직접 실습할 수 있습니다. 이 강의는 Claude Architect 학습 경로의 일부이며, 진행 상황이 웹과 CoddyKit 앱에 동기화됩니다. Claude Architect 강의에는 총 4개의 강의가 포함되어 있습니다.
이 강의의 일부는 아직 번역되지 않았으며 영어로 표시됩니다.
Why Examples Beat Adjectives
When a prompt under-performs, architects reach for vague fixes like "be more precise" or "try harder". These rarely move the needle. The reliable lever is explicit criteria plus concrete examples.
Compare: "be more precise" versus "flag a comment only when it contradicts the code". The second tells the model exactly where the decision boundary sits. In this lesson you'll learn to drive a prompt to production quality using 2-4 targeted input/output examples and a tight test-and-iterate loop.
How Few-Shot Actually Works
Few-shot prompting attaches a small set of worked examples to your instructions. The key insight: the model generalizes from the examples — it does not merely copy them. Given 3 representative cases, it infers the underlying rule and applies it to unseen inputs.
Few-shot is strongest for four jobs:
- Consistency across many calls
- Edge cases that words alone describe poorly
- Output format the model should mirror
- Reducing hallucination by anchoring behavior
Aim for 2-4 examples per ambiguity — enough to define the pattern, few enough to keep context lean.
Anatomy of a Good Example
An effective few-shot example is a paired input and the exact output you want back. The output must match the real schema or shape you'll consume in production — same fields, same casing, same structure.
Below, each example shows an input comment plus the precise verdict. The model learns the boundary: flag only genuine contradictions, not style nits.
examples = [
{
"input": "# returns the user's age\n def get_name(u): return u.name",
"output": {"flag": True, "reason": "comment says age, code returns name"},
},
{
"input": "# sort ascending\n items.sort()",
"output": {"flag": False, "reason": "comment matches behavior"},
},
]
system = (
"Flag a comment ONLY when it contradicts the code. "
"Style or wording issues are not contradictions.\n\n"
"Examples:\n" + "\n".join(
f"INPUT: {e['input']}\nOUTPUT: {e['output']}" for e in examples
)
)Pair Criteria With Examples
Examples and explicit criteria are partners, not substitutes. Criteria state the rule; examples calibrate the gray zone the rule can't fully capture in prose.
A common architect mistake is dumping examples with no governing instruction. The model then over-fits to surface features of the samples. Always lead with a crisp criterion — "flag only when X contradicts Y" — then let 2-4 examples pin down where X and Y blur.
Rule of thumb: if you can't write the criterion in one sentence, your task is still under-specified, and more examples won't fix that.
Build a Test Set First
Iterative refinement is test-driven. Before tuning the prompt, assemble a small labeled set of representative inputs with known-correct outputs. This is your ground truth — every prompt change is judged against it, not against a gut feeling.
Keep the test set separate from your few-shot examples. If you tune on the same cases you teach with, you're memorizing, not generalizing.
test_cases = [
{"input": "# deletes the record\n def archive(r): r.archived = True",
"expected": {"flag": True}},
{"input": "# cache result for 60s\n cache.set(k, v, ttl=60)",
"expected": {"flag": False}},
{"input": "# returns count\n def total(rows): return sum(r.amt for r in rows)",
"expected": {"flag": True}},
]The Iteration Loop
The refinement loop is mechanical and repeatable:
- Run the prompt over every test case
- Compare output to the expected label
- Inspect the failures — what boundary did the model miss?
- Add or sharpen one example (or one criterion) that targets that failure
- Re-run the full set and check nothing regressed
Change one thing per iteration. Batch edits make it impossible to know which tweak helped or hurt.
Scoring the Test Set
Automate the comparison so iteration is fast. A tiny harness runs each case, scores it, and prints the misses. With structured output you can compare fields directly instead of parsing prose.
Note tool_choice below: forcing a tool call guarantees the model returns schema-valid JSON every time, so your scorer never trips over free-text.
def score(client, system, cases):
misses = []
for c in cases:
resp = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=256,
system=system,
tools=[verdict_tool],
tool_choice={"type": "any"}, # must call a tool -> structured output
messages=[{"role": "user", "content": c["input"]}],
)
out = resp.content[0].input
if out["flag"] != c["expected"]["flag"]:
misses.append((c["input"], out))
return missesTarget Failures With New Examples
When you read a miss, ask: which ambiguity caused it? Then add an example that lives exactly on that boundary — not a random new case. One sharp example per failure mode generalizes far better than ten generic ones.
Suppose the model wrongly flagged a comment that paraphrased the code. You'd add a paired example showing paraphrase = not a contradiction. The model updates its internal boundary and the whole class of similar misses clears.
Resist the urge to keep piling on examples. Past 4-ish per ambiguity you bloat context and risk lost-in-the-middle, where the model under-attends to the center of a long prompt.
# Add ONE example aimed at the observed miss:
examples.append({
"input": "# loop over each item\n for x in items: process(x)",
"output": {"flag": False,
"reason": "paraphrase of code, not a contradiction"},
})Format Errors? Retry With Feedback
Some failures are format or structural, not reasoning errors — a malformed field, a bad arithmetic total, a missing bracket. For these, retry-with-feedback is the fix: send back the original input, the wrong output, and the exact validation error.
Crucial limit: retry helps when the model can produce the right answer but slipped. It does not help when the required information is simply absent from the source — no amount of retrying invents data that isn't there.
def retry_with_feedback(client, system, original, bad_output, error):
return client.messages.create(
model="claude-sonnet-4-5",
max_tokens=512,
system=system,
messages=[
{"role": "user", "content": original},
{"role": "assistant", "content": str(bad_output)},
{"role": "user", "content":
f"That output failed validation: {error}. "
"Return corrected output that satisfies the schema."},
],
)Lock Quality With Structured Output
Once examples have shaped the behavior, lock the shape with a JSON Schema via tool use. This eliminates syntax errors and enforces required fields. One discipline matters most: mark a field required only if it is always present.
Never require a field that might be absent — the model will fabricate a value to satisfy the schema. For optional or open-ended data, use an enum with an "other" value plus a free-text detail field, which keeps the contract extensible without forcing hallucination.
verdict_tool = {
"name": "record_verdict",
"description": "Record whether a comment contradicts its code.",
"input_schema": {
"type": "object",
"properties": {
"flag": {"type": "boolean"},
"category": {"type": "string",
"enum": ["contradiction", "style", "other"]},
"detail": {"type": "string"},
},
"required": ["flag"], # only the always-present field
},
}Validate With a Fresh Reviewer
Before you trust the refined prompt, validate it with an independent, fresh instance — not the same session that produced the output. A same-session self-review is biased: the author retains its own reasoning and won't challenge itself.
And don't trust an aggregate score alone. A headline like 97% accuracy can hide a field or input type that's failing badly. Use stratified sampling and field-level confidence, calibrated on a labeled validation set, before you automate. Refinement isn't done when the average looks good — it's done when every slice clears the bar.
Quick Check: Fixing a Failing Prompt
A classifier prompt passes 92% of your labeled test set but consistently mislabels comments that paraphrase the code as contradictions. Which refinement move is most effective?
Recap: Refine With Examples, Prove With Tests
Key takeaways:
- Examples beat adjectives. Explicit criteria plus 2-4 targeted examples outperform vague instructions like 'be more precise'.
- The model generalizes from few-shot examples — best for consistency, edge cases, output format, and reducing hallucination.
- Be test-driven: build a labeled set, score it, add one boundary example per failure mode, re-run, change one thing at a time.
- Retry-with-feedback fixes format errors (send original + wrong output + exact error) — not missing-source data.
- Lock the shape with a JSON Schema; require only always-present fields; use enum + 'other' + detail for extensibility.
- Validate independently: fresh-instance review beats same-session self-review, and stratified field-level checks beat aggregate-only accuracy.
자주 묻는 질문
“예시를 활용한 반복적 개선” 강의는 무료인가요?
네 — “예시를 활용한 반복적 개선” 전체 내용을 이 웹사이트에서 무료로 읽을 수 있습니다. 인터랙티브하게 실습하려면(내장 코드 에디터와 24/7 AI 튜터), CoddyKit PRO로 업그레이드하면 Claude Architect 강의 전체를 잠금 해제할 수 있습니다. Claude Architect 강의에는 총 4개의 강의가 포함되어 있습니다.
“예시를 활용한 반복적 개선”에서 뭘 배우나요?
입력/출력 예시 2~4개와 테스트 주도 반복을 활용합니다 브라우저에서 직접 실행하는 실습 코드로 Claude Architect을(를) 배우며, 24/7 AI 튜터가 강의를 진행하면서 질문에 답변해줍니다.
Claude Architect을(를) 시작하는 데 경험이 필요한가요?
사전 경험은 필요하지 않습니다. CoddyKit의 Claude Architect은(는) 초급자부터 고급 학습자까지를 위해 구성되어 있으므로, 여기서 시작하거나 처음부터 시작할 수 있으며 자신의 속도대로 진행할 수 있습니다. 이것은 4개 중 4번째 강의입니다.
“예시를 활용한 반복적 개선” 강의는 얼마나 걸리나요?
대부분의 CoddyKit 강의는 약 5~10분이 소요됩니다. 각 강의는 간결하고 인터랙티브하여 꾸준한 진행이 가능하며, 웹과 앱에서 중단한 부분부터 바로 시작할 수 있습니다.
이 Claude Architect 강의에서 코드를 작성하고 실행할 수 있나요?
네. 모든 Claude Architect 강의에는 내장 코드 에디터가 포함되어 있으므로, 브라우저에서 바로 실제 코드를 작성하고 실행한 후 즉시 AI 피드백을 받을 수 있습니다 — 로컬 설정이 필요 없습니다.
이 강의의 모든 강의
- 사용자 지정 명령과 Skills
- Skill Frontmatter
- 계획 모드와 직접 실행
- 예시를 활용한 반복적 개선