Pemetaan Klaim ke Sumber

In a multi-agent research system, the coordinator aggregates findings from subagents that each pulled facts from different documents. The aggregated answer is only trustworthy if every claim can be traced back to where it came from.

A claim-to-source mapping binds each factual statement to its origin: the URL, the document name, the exact quote, and the publication date. Without that binding, you have an assertion no human reviewer can verify and no downstream agent can audit.

For the Architect exam, provenance lives in Domain 4 (Prompt Engineering & Structured Output) and recurs in Scenario 3 (Multi-Agent Research) and Scenario 6 (Structured Data Extraction).

A source mapping should always carry four anchors so a human or another agent can re-locate and re-verify the evidence:

• URL — where the document lives (or a stable identifier).
• Document name — a human-readable title.
• Quote — the verbatim text that supports the claim, not a paraphrase.
• Publication date — when the source was published or last updated.

The verbatim quote is what lets a reviewer confirm the model did not hallucinate or overstate. The date is what lets you resolve conflicts later, as we'll see.

Don't ask the model to weave citations into prose where they're easy to drop. Instead, force a structured shape with a JSON Schema via a tool. Pairing tool_use with a schema eliminates syntax errors and enforces that required fields are present.

Each claim becomes an object that carries its own source anchors. This is the foundation of a verifiable provenance record.

A subtle but exam-critical rule: mark a field required only if it is always present in the source. If you require a field that may be absent, the model will fabricate a value to satisfy the schema.

A blog post may have no formal publication_date; an internal PDF may have no source_url. So claim, quote, and document_name are required (you always have them), while source_url and publication_date stay optional. An empty date is honest; an invented one is a provenance failure.

To guarantee you get the provenance object back instead of free-form prose, constrain the model with tool_choice. Setting {"type": "tool", "name": "record_claims"} forces that specific tool, so every response arrives as schema-validated JSON.

Using "any" would guarantee some tool is called; forcing the named tool is the tightest guarantee when you only have one extraction tool.

Requiring a verbatim quote (not a paraphrase) is a reliability lever, not just bookkeeping. When the model must copy the exact supporting text, it is far harder to assert something the source never said.

Reinforce this with explicit criteria in the prompt and 2-4 few-shot examples. Few-shot examples are especially effective for output format, edge cases, and reducing hallucination — the model generalizes the pattern rather than merely repeating your samples.

In a hub-and-spoke research system, subagents do not inherit the coordinator's conversation history. Every subagent must return its findings with source mappings attached, because the coordinator has no other way to know where a fact came from.

The coordinator then aggregates these self-contained claim objects. If a subagent returns a bare sentence with no quote or document name, that fact is unverifiable the moment it leaves the subagent's context — treat it as missing provenance, not as a valid finding.

When two sources disagree on a statistic, the wrong move is to silently pick one. The right move is to annotate the conflict and surface both claims with their sources.

Often the publication date resolves the apparent contradiction: a 2021 figure and a 2024 figure aren't contradictory — they're a time series. This is exactly why the date anchor earns its place in every mapping. Preserve both, label them, and let a human (or a date-aware rule) adjudicate.

Self-correction works by extracting two values you can compare. For provenance, have the model emit each claim alongside a flag for whether a supporting quote was actually found, then validate programmatically that every claim carries one.

This catches the silent failure where a confident sentence ships with no evidence behind it. Validation here is structural — perfect for a retry loop.

If validation finds a structural gap (missing quote, malformed date), use retry-with-feedback: send the model the original document, its own wrong output, and the exact validation error. This reliably fixes format and structural mistakes.

Critical boundary: retry does not help when the information is simply absent from the source. If the document genuinely has no publication date, no amount of retrying will conjure one — and you must not let it. Mark the field empty and move on, rather than looping forever.

Once claims are mapped, present them in the format that fits the content. Render financials as tables, news as prose, and technical findings as lists — each with its source anchors visible.

For human oversight, a footnote or trailing column carrying the document name, date, and a link makes verification a glance instead of an investigation. Provenance you can't see is provenance no one will check.

Key takeaways for the exam and for production systems:

• Bind every claim to four anchors: URL, document name, verbatim quote, publication date.
• Enforce the shape with tool_use + JSON Schema and force it via tool_choice.
• Mark a field required only if always present — never require a possibly-absent field, or the model fabricates it.
• Subagents don't inherit history, so each must return self-contained, traceable claims.
• Annotate conflicts; let dates resolve apparent contradictions instead of picking arbitrarily.
• Use retry-with-feedback for structural gaps — but accept that retry can't supply info absent from the source.
• Render by content type (tables/prose/lists) with sources visible for human oversight.