Claude Architect · Lesson

When Retry Helps (and When It Can't)

Great for format errors; useless for missing data.

When Retry Helps (and When It Can't) is a free Claude Architect lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Claude Architect learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

The Retry Reflex

When a model's output fails validation, the tempting move is to just try again. Sometimes that works beautifully. Sometimes it burns tokens and latency for nothing.

As an architect, your job is to know which failure you are looking at before you wire up a retry loop. Retry is a precision tool, not a blanket safety net.

This lesson draws a hard line: retry is excellent for format and structural errors, and useless when the needed information is simply absent from the source.

Two Very Different Failures

Extraction and structured-output pipelines fail in two fundamentally different ways:

Format / structural / arithmetic errors — the answer exists in the source, but the model rendered it wrong: invalid JSON, a missing required field it actually had data for, a total that doesn't add up.
Absent information — the source document simply does not contain the value. There is nothing to extract.

Retry-with-feedback can fix the first class. It cannot conjure data that was never there. Confusing the two is a classic anti-pattern.

What Retry-With-Feedback Actually Sends

A good retry is not "run the same prompt again and hope." It is a corrective retry. You send the model three things:

The original source document
The wrong output it produced
The exact validation error that was raised

This gives the model the specific signal it needs to self-correct. Vague feedback like "that was wrong, try harder" performs far worse than handing it the precise validator message.

def retry_with_feedback(client, source_doc, bad_output, validation_error):
    return client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        messages=[
            {"role": "user", "content": (
                "Extract the invoice fields as JSON.\n\n"
                f"SOURCE DOCUMENT:\n{source_doc}\n\n"
                f"YOUR PREVIOUS OUTPUT (rejected):\n{bad_output}\n\n"
                f"VALIDATION ERROR:\n{validation_error}\n\n"
                "Fix only what the error names. Return corrected JSON."
            )}
        ],
    )

Format Errors: Retry's Sweet Spot

Format and structural problems are exactly where retry shines, because the correct answer is recoverable from the same input:

Malformed JSON or a stray trailing comma
A field present in the doc but missing from the output
Wrong enum casing or an unexpected key
Arithmetic that doesn't reconcile

On the second pass, armed with the validator's message, the model almost always lands the corrected structure. Even better: enforce structure up front with tool_use + a JSON Schema so most syntax errors never happen at all.

Catching Arithmetic With Self-Correction

Arithmetic discrepancies are a format-class error retry can fix — but first you have to detect them. The technique: extract both the value the model computes and the value the document states, then compare.

Pull out calculated_total (sum of line items) AND stated_total (the printed total). If they diverge, you have a concrete, retryable validation error to feed back — not a vague hunch.

from pydantic import BaseModel, model_validator

class Invoice(BaseModel):
    line_items: list[float]
    calculated_total: float
    stated_total: float

    @model_validator(mode="after")
    def totals_match(self):
        if round(self.calculated_total, 2) != round(self.stated_total, 2):
            raise ValueError(
                f"calculated_total {self.calculated_total} != "
                f"stated_total {self.stated_total}"
            )
        return self

Where Retry Hits a Wall

Now the hard limit. If a field is not present in the source, retrying does nothing useful. The model has two bad options on each loop:

Return the same "missing" result — wasted latency and cost.
Fabricate a plausible-looking value to satisfy the validator — far worse, because now you've injected a hallucination into trusted data.

Retry pressure on absent data actively encourages invention. No amount of re-prompting extracts a value that was never written down.

The Schema Trap: Required Fields

This is where a subtle schema design choice bites. Mark a field required ONLY if it is always present. Never require a field that may be absent — the model will fabricate it to satisfy the schema, and your retry loop will keep accepting garbage.

For optional data, make the field optional and let the model report absence honestly. A retry can't rescue a schema that forces invention.

tax_id_tool = {
    "name": "extract_vendor",
    "description": "Extract vendor fields from an invoice.",
    "input_schema": {
        "type": "object",
        "properties": {
            "vendor_name": {"type": "string"},
            # tax_id is OFTEN absent -> optional, never required
            "tax_id": {"type": ["string", "null"]},
        },
        # require ONLY the always-present field
        "required": ["vendor_name"],
    },
}

Make Absence a First-Class Result

The fix for absent data is not retry — it is letting the model say so explicitly. Distinguish a genuine access/format failure (maybe retryable) from a valid empty result (no value exists — stop, don't loop).

Use an enum with an "other"/"not_present" value plus a free-text detail field. That keeps the schema extensible and gives downstream code a clean signal to skip the field instead of triggering a doomed retry.

{
  "properties": {
    "discount_status": {
      "type": "string",
      "enum": ["applied", "none", "not_present", "other"]
    },
    "discount_detail": { "type": "string" }
  },
  "required": ["discount_status"]
}

Route by Error Type, Not by Reflex

Mature pipelines branch on why validation failed. Structured errors enable this; generic ones ("Operation failed") block it.

validation / format / arithmetic mismatch → retry-with-feedback.
transient (timeout, rate limit) → retry locally, the value still exists.
absent data / valid empty result → record "not present" and move on. Do NOT retry.

This mirrors structured MCP errors: an errorCategory and an isRetryable flag let your loop make an intelligent routing decision instead of blindly looping.

def handle(result):
    if result.error_category in ("validation", "arithmetic"):
        return "retry_with_feedback"   # answer is recoverable
    if result.error_category == "transient" and result.is_retryable:
        return "retry_local"           # network/rate-limit blip
    if result.error_category == "absent":
        return "record_not_present"    # NEVER retry absent data
    return "escalate"

Cap the Loop, But Don't Lean on the Cap

Even for genuinely retryable format errors, bound the loop. A retry budget of 2–3 attempts is plenty — if a corrective retry hasn't converged by then, the problem usually isn't format; it's missing data or an over-strict schema.

Treat the cap as a safety net, never the primary stop mechanism. The real terminator is "validation passed." If you find yourself relying on the cap to exit, that's a signal you're retrying something retry can't fix.

def extract(client, doc, validate, max_attempts=3):
    out = first_pass(client, doc)
    for _ in range(max_attempts):
        try:
            return validate(out)          # PRIMARY stop: it's valid
        except ValidationError as e:
            out = retry_with_feedback(client, doc, out, str(e))
    # SAFETY NET only -- not the intended exit path
    raise RuntimeError("unresolved after retries; likely absent data")

When Even a Good Retry Won't Save You

Two more honest limits an architect must respect:

A fresh, independent reviewer beats same-session self-review. An author retains its own reasoning and won't challenge itself — so for a true second opinion, validate with a fresh instance, not another turn of the same context.
Blocking, time-sensitive checks belong inline. The Batch API is 50% cheaper but has no latency SLA and a window up to 24h — great for overnight audits, wrong for a pre-merge or real-time validation gate.

Retry tuning can't compensate for the wrong validation architecture underneath it.

Quick Check: Retry or Not?

A structured-extraction pipeline pulls fields from supplier invoices. The Pydantic validator rejects an output because the tax_id field is empty. On inspection, this particular invoice has no tax ID printed anywhere on it. What is the correct architecture?

Recap: Retry Is a Scalpel, Not a Hammer

Key takeaways:

Retry fixes format, structural, and arithmetic errors — send the original doc + the wrong output + the exact validation error.
Retry can't fix absent data. If it's not in the source, looping only wastes cost and invites fabrication.
Never require a possibly-absent field — make it optional and let the model report "not present."
Route by error type: validation/arithmetic → retry-with-feedback; transient → retry locally; absent → record and stop.
Cap attempts (2–3) as a safety net; the primary stop is "validation passed."
For true second opinions use a fresh instance; keep blocking checks inline, not on the Batch API.

Frequently asked questions

Is the “When Retry Helps (and When It Can't)” lesson free?

Yes — the full text of “When Retry Helps (and When It Can't)” is free to read here on the web, and the Claude Architect course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Claude Architect course, upgrade to CoddyKit PRO.

What will I learn in “When Retry Helps (and When It Can't)”?

Great for format errors; useless for missing data. You practise Claude Architect with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Claude Architect?

No prior experience is required. Claude Architect on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “When Retry Helps (and When It Can't)” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Claude Architect lesson?

Yes. Every Claude Architect lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to Claude Architect