P3.3 deep dive · D2 + D54 decisions5 failure modes

Structured Claim-Source Mapping.

How verified claims get pinned to their source documents in a structured-output schema so the final report cannot fabricate.

Mental modelEvery verified claim emits as a JSON object with claim_id, claim_text, source_url, source_passage, confidence, and notes. The schema is enforced at the verification step. Synthesis renders citations from the schema and cannot invent attributions because the source text is pinned in-record.
Structured Claim-Source Mapping canonical flow.
01 · Summary

What this deep dive covers

Every verified claim emits as a JSON object with claim_id, claim_text, source_url, source_passage, confidence, and notes. The schema is enforced at the verification step. Synthesis renders citations from the schema and cannot invent attributions because the source text is pinned in-record.

3Pattern notes
3Flow notes
4Decisions
5Failure modes
4Exam questions
02 · The pattern

What it is

Structured claim-source mapping is the contract that says every claim in the final report is traceable to a specific passage in a specific source. The verification subagent emits JSON that pairs each claim with its provenance: claim_id, claim_text, source_url, source_passage (the actual quoted text, not just a URL), publication_date, and confidence. Synthesis reads only this schema and renders inline citations from it. No free-form attribution, no model-generated URLs.

The architectural reason is anti-fabrication. A synthesis subagent that writes prose plus citations from memory will eventually produce a citation that looks plausible but doesn't exist. By forcing synthesis to render [1] from a record where source_passage is already pinned, you remove the model's ability to hallucinate the link. If the passage isn't in the schema, there is no citation to render. Either the record is complete or it gets surfaced as a data gap.

The schema also handles the conflicting-source case explicitly. When two sources disagree (45% Pew vs 12% McKinsey), the verification subagent emits a sources_reconciled array with both records pinned, plus a notes field that explains the apparent conflict (different timeframes, different definitions, different populations). Synthesis presents both with attribution; it does not pick a winner. Picking one is misinformation. Preserving both with context is journalism.

03 · How it runs

How it works

The verification subagent receives pooled findings from research subagents and a fact-check rubric. For each candidate claim, it confirms the source is credible and dated, extracts the verbatim source_passage that backs the claim, and assigns a confidence score. The output is JSON only: {verifications: [{claim_id, claim_text, verified, confidence, sources_reconciled: [{stat, source_url, source_passage, publication_date, context}], notes}]}. The schema is enforced via Pydantic in Python or Zod in TypeScript; malformed output fails the verification step and triggers a retry.

Synthesis is read-only. It receives the verified-claims JSON, the coordinator's narrative prompt, and a tool list of [Read] only. It walks the verifications array in order, writes prose that flows logically, and emits inline [1], [2] citations that index into sources_reconciled. The render is mechanical. Synthesis is never asked to invent an attribution; if it tries, the verification record is the only thing the citation can point to.

Data gaps are first-class. When verification finds a claim it cannot confirm (no credible source, conflicting data without enough context, or a research-subagent timeout), it emits {verified: false, notes: 'no credible source within window'}. Synthesis is instructed to acknowledge that gap in prose: Adoption rates among independent musicians remain unverified across our sources. Transparent gaps beat confident fabrication every time. This is the architectural detail that protects the report from looking complete when it isn't.

Structured Claim-Source Mapping sequence diagram.
04 · Configuration decisions

The 4 decisions

01

Two sources disagree (45% Pew vs 12% McKinsey)

Both numbers are correct under their own definitions (any-use vs daily-use). Dropping one is misinformation. Preserving both with context is the journalistic and architectural move.

Right answerPreserve both in sources_reconciled with attribution + notes explaining the difference
02

Synthesis needs a citation for a claim. Where does the URL come from?

Model-generated citations are the canonical fabrication failure. Schema-pinned citations cannot be invented. The model is rendering, not authoring.

Right answerFrom the source_url field in the verification record
03

A claim has no credible source. What happens?

Silent drops produce reports that look complete when they aren't. Acknowledged gaps are honest and let the reader judge confidence.

Right answerEmit {verified: false, notes: 'no credible source'}. Synthesis acknowledges the gap in prose
04

Should the schema include the verbatim source_passage?

Re-fetching introduces a new failure mode (URL went 404, page changed). Pinning the passage at verification time freezes provenance. The record is self-contained.

Right answerYes. The exact quoted text that backs the claim
05 · Right call, wrong call

9 decisions the exam turns into distractors

01 · Two sources disagree (45% Pew vs 12% McKinsey)
Looks right

Pick the higher-confidence source and drop the other

Actually right

Preserve both in sources_reconciled with attribution + notes explaining the difference

DEC-01
02 · Synthesis needs a citation for a claim. Where does the URL come from?
Looks right

Synthesis generates the citation from memory of training data

Actually right

From the source_url field in the verification record

DEC-02
03 · A claim has no credible source. What happens?
Looks right

Drop the claim silently from the final report

Actually right

Emit {verified: false, notes: 'no credible source'}. Synthesis acknowledges the gap in prose

DEC-03
04 · Should the schema include the verbatim source_passage?
Looks right

Just the URL. The passage can be re-fetched at render time

Actually right

Yes. The exact quoted text that backs the claim

DEC-04
05 · Free-form citations
Looks right

Synthesis writes prose with (Pew, 2024) style citations from memory. Half the year tags are wrong; one URL doesn't resolve.

Actually right

Force synthesis to render [1], [2] indexes that resolve through the verified-claims JSON. The model is a renderer, not a citation author.

FAIL-01
06 · Conflict-flattening
Looks right

Verification sees 45% Pew and 12% McKinsey, writes ~30% (averaged). The averaged number doesn't exist anywhere; the report is misinformation.

Actually right

Emit both in sources_reconciled with attribution + notes explaining timeframe/definition/population differences. Synthesis presents both with context.

FAIL-02
07 · Silent unverified-claim drop
Looks right

Verification can't confirm a claim, drops it. Final report reads as if no such question was ever asked.

Actually right

Emit {verified: false, notes: 'unverified'} and instruct synthesis to acknowledge the gap. Transparency beats false completeness.

FAIL-03
08 · Passage-by-URL only
Looks right

Schema stores source_url but not source_passage. A week later the URL returns 404 or the page is rewritten. Report claims become unverifiable.

Actually right

Pin the verbatim source_passage at verification time. The record is self-contained even if the source URL drifts.

FAIL-04
09 · Schema not enforced
Looks right

Verification emits malformed JSON; synthesis parses what it can and improvises the rest. Fabrication slips back in via the missing fields.

Actually right

Validate verification output with Pydantic / Zod. Malformed output fails the step and triggers a retry. Synthesis never sees an incomplete record.

FAIL-05
Structured Claim-Source Mapping failure map.
06 · Practice

5 exam-pattern questions

Work through one question at a time, check the architecture, then move through the set.

Question 1 of 5 · D1Choose the best answer

A web-search subagent times out and returns an empty result list. The coordinator treats this as 'no information available' and moves forward. The final report is incomplete. What is the architectural fix?

07 · FAQ

Frequently asked

Why pin the passage instead of just the URL?

URLs drift. Pages get rewritten, papers retracted, sites moved. Pinning the verbatim source_passage at verification time freezes provenance in the record itself. The record is self-contained even if the source URL later breaks.

How does the schema prevent fabrication if the model is still generating prose?

The model generates prose freely, but citation rendering is mechanical: [1] resolves through the verified-claims JSON to a record with source_url and source_passage. If a record doesn't exist, there's no [N] to render. The model can't invent attributions because it's rendering an array, not authoring citations.

What confidence score threshold should trigger a gap acknowledgement?

Domain-dependent, but a reasonable default is < 0.6 confidence triggers a notes field that synthesis surfaces. Below < 0.3 triggers an explicit gap acknowledgement. Above >= 0.6, present the claim normally. Tune by calibrating against human-graded reports.

Help someone build it

Share this scenario.

One share is one less team repeating the same architecture mistake.