Self-Correction
Extract calculated_total and stated_total to catch drift.
Self-Correction is a free Claude Architect lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Claude Architect learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Trust Gap
When Claude extracts data from an invoice or report, the output can be perfectly valid JSON and still be wrong. A line item might be misread, a number transposed, or a subtotal silently drifted.
Schema validation catches structural problems: missing required fields, wrong types, bad enums. It does not catch a number that is well-formed but inconsistent with the rest of the document.
This lesson teaches self-correction: a technique where you extract enough information to let the system check its own arithmetic and catch drift before it reaches your database.
What 'Drift' Looks Like
Consider an invoice that lists line items and prints a total at the bottom. Two numbers are in play:
- stated_total — the total literally printed on the document
- calculated_total — the sum of the individual line items
On a clean document they match. But OCR noise, a misread digit, or a hallucinated line item makes them diverge. That divergence is drift.
The core idea of this lesson: if you only capture one of these two numbers, you can never detect the disagreement. Capture both, and the discrepancy becomes visible and machine-checkable.
Extract Both, Not One
The self-correction pattern starts in the schema. Instead of asking for a single total, you ask Claude to surface the raw materials for a consistency check.
Force structured output with a tool plus a JSON Schema so you eliminate syntax errors and guarantee the fields are present and typed. Use tool_choice set to any (or a forced tool) so the model must return the tool call rather than free-form prose.
import anthropic
client = anthropic.Anthropic()
extract_tool = {
"name": "extract_invoice",
"description": "Extract invoice line items plus BOTH the summed line-item total and the total printed on the document, so a downstream check can detect drift.",
"input_schema": {
"type": "object",
"properties": {
"line_items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"description": {"type": "string"},
"amount": {"type": "number"}
},
"required": ["description", "amount"]
}
},
"calculated_total": {"type": "number"},
"stated_total": {"type": "number"}
},
"required": ["line_items", "calculated_total", "stated_total"]
}
}
resp = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
tools=[extract_tool],
tool_choice={"type": "any"},
messages=[{"role": "user", "content": invoice_text}],
)Required Means Always Present
One subtle rule: mark a field required only if it is always present in the source. Both calculated_total and stated_total qualify here — every invoice has line items to sum and a printed total to read.
But never require a field that may be absent. If you force a field that isn't in the document, the model will fabricate a plausible value to satisfy the schema — and now you've manufactured the very drift you were trying to catch.
For optional structures, leave the field out of required and let it be omitted. For extensible categoricals, use an enum with an "other" value plus a free-text detail field.
The Validation Step
Once both numbers are in hand, validation is plain code — deterministic, fast, and free. Use a Pydantic-style validator so the check lives next to the schema and runs on every extraction.
The model decides what the numbers are; your code decides whether they agree. Keep the consistency rule in hard code, not in a prompt, because arithmetic agreement is exactly the kind of guarantee that deterministic logic provides and probabilistic prompting cannot.
from pydantic import BaseModel, model_validator
class Invoice(BaseModel):
line_items: list[dict]
calculated_total: float
stated_total: float
@model_validator(mode="after")
def check_drift(self):
summed = round(sum(li["amount"] for li in self.line_items), 2)
if abs(summed - self.calculated_total) > 0.01:
raise ValueError(
f"calculated_total {self.calculated_total} "
f"does not match line-item sum {summed}"
)
if abs(self.calculated_total - self.stated_total) > 0.01:
raise ValueError(
f"DRIFT: calculated_total {self.calculated_total} "
f"!= stated_total {self.stated_total}"
)
return selfRetry With Feedback
When the validator raises, don't just discard the output and re-run blind. Use retry-with-feedback: send the model the original document, its own wrong output, and the exact validation error.
This works precisely because the failure is an arithmetic / structural error — the information needed to fix it is present in the source; the model just has to recompute. Giving it the specific error turns a guess into a targeted correction.
def extract_with_retry(invoice_text, max_attempts=3):
messages = [{"role": "user", "content": invoice_text}]
for _ in range(max_attempts):
resp = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
tools=[extract_tool],
tool_choice={"type": "any"},
messages=messages,
)
data = resp.content[0].input
try:
return Invoice(**data)
except ValueError as err:
messages.append({"role": "assistant", "content": resp.content})
messages.append({"role": "user", "content":
f"That extraction failed validation: {err}. "
f"Re-read the document and correct the numbers."})
raise RuntimeError("extraction did not converge")When Retry Won't Help
Retry-with-feedback is powerful but it is not magic. It fixes format, structural, and arithmetic errors — cases where the right answer is recoverable from the source.
It does not help when information is simply absent from the document. If an invoice has no stated total at all, looping the model just pressures it to invent one. Re-reading the same blank space cannot produce a fact that was never there.
So branch your logic: discrepancy between two present numbers → retry. A genuinely missing field → mark it absent (leave it out of required) and escalate or flag, never fabricate.
# stated_total is OPTIONAL in the schema, not required
class Invoice(BaseModel):
line_items: list[dict]
calculated_total: float
stated_total: float | None = None # may be absent on the document
@model_validator(mode="after")
def check_drift(self):
if self.stated_total is None:
# Don't retry an absent fact -- flag for human review
self.flags = ["stated_total_missing"]
return self
if abs(self.calculated_total - self.stated_total) > 0.01:
raise ValueError("DRIFT detected") # recoverable -> retry
return selfCap Iterations As A Safety Net
Notice max_attempts in the retry loop. That cap is a safety net, not the primary control flow. The loop's real exit is success: the validator passes and you return.
This mirrors the agentic loop everywhere in Claude architecture — you terminate on a concrete condition (here, validation passing), and you keep a hard cap only to prevent a pathological document from looping forever. Never let an arbitrary iteration count become your stop logic, and never parse the model's prose for words like "corrected" to decide you're done.
Independent Review Beats Self-Review
Deterministic arithmetic checks catch numeric drift. But for judgment-level correctness — "did this extraction misclassify the vendor?" — a model reviewing its own same-session output is weak. The author retains its original reasoning and tends not to challenge itself.
When you need a semantic second opinion, run an independent, fresh-instance review: a separate request (or subagent) that sees only the document and the extracted output, not the original chain of thought. Fresh eyes catch what self-review rationalizes away.
review = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=512,
system=(
"You are an independent auditor. You did NOT produce this "
"extraction. Compare the JSON against the source document and "
"report any field that contradicts the source."
),
messages=[{"role": "user", "content":
f"SOURCE:\n{invoice_text}\n\nEXTRACTION:\n{extracted_json}"}],
)Keep Case Facts Verbatim
Self-correction depends on numbers staying exact through your pipeline. The danger is progressive summarization: as context is compacted, totals, percentages, and dates go vague — "around 4,000" instead of 3,981.50.
Pull the transactional facts — calculated_total, stated_total, line amounts — into a separate "case facts" block kept verbatim, outside any summary. Trim verbose tool output to only the fields you need, but never let a summarizer touch the numbers your validator must compare.
Track Provenance For Drift
When the two totals disagree, you want to know why, fast. Keep provenance: map each extracted number back to its source — the line of text it came from, and where on the document.
With provenance, a drift alert becomes actionable: "calculated_total summed five line items, but the document shows six." Without it, you only know the numbers disagree, not which one to trust. And render results by content type — financial figures belong in a table where mismatches are easy to scan, not buried in prose.
# Annotate each figure with where it came from
"line_items": [
{"description": "Consulting", "amount": 1200.00,
"source": {"page": 1, "text": "Consulting ... 1,200.00"}}
],
"stated_total": {"value": 3981.50,
"source": {"page": 1, "text": "TOTAL DUE 3,981.50"}}
# On drift, the alert can name the exact conflicting source lines.Quick Check: Designing The Drift Catch
You are extracting invoice totals and must reliably catch cases where the printed total disagrees with the line items. Which design best implements self-correction?
Recap: Catching Drift
Key takeaways for self-correction in extraction pipelines:
- Extract both numbers. Capture
calculated_totalandstated_total— you can't detect drift from a single value. - Validate in code. Use Pydantic-style deterministic checks; keep arithmetic guarantees out of prompts.
- Retry with feedback (original doc + wrong output + exact error) fixes format/structural/arithmetic errors — but NOT genuinely absent data.
- Required = always present. Never require a maybe-absent field, or the model fabricates it.
- Iteration caps are a safety net; the real exit is validation passing — never parse prose for completion.
- Independent fresh-instance review beats same-session self-review for semantic correctness.
- Protect the numbers: keep case facts verbatim outside summaries, and track provenance so a drift alert tells you which value to trust.
Frequently asked questions
Is the “Self-Correction” lesson free?
Yes — the full text of “Self-Correction” is free to read here on the web, and the Claude Architect course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Claude Architect course, upgrade to CoddyKit PRO.
What will I learn in “Self-Correction”?
Extract calculated_total and stated_total to catch drift. You practise Claude Architect with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Claude Architect?
No prior experience is required. Claude Architect on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Self-Correction” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Claude Architect lesson?
Yes. Every Claude Architect lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.