ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents

TL;DR

The failure it names is a claim that is true somewhere in the evidence and attributed to the wrong source. The paper calls this cross-source conflation, and its example is exact: a support agent answers "According to the account record, this plan includes a 30-day refund window" when the refund window is stated in a policy document, not in the account record it cites. ProvenanceGuard decomposes an answer into claims, routes each to its most relevant source with a lightweight embedding model, validates support by natural language inference, then cross-references the actual supporting source against the one cited, emitting per-claim verdicts and a global allow/block decision. On 361 expert-annotated claims from a medical agent's traces it caught 138 of 139 claims that should have been blocked, identified sources correctly ~86% of the time, and beat four other checkers on reject/block F1 — 0.802 against 0.436–0.783 (source).

Authors & Org

Not stated in anything read. The HuggingFace blog post is published under MultiverseComputingCAI; whether that is the paper's affiliation is not stated, and huggingface.co and arxiv.org are both blocked from this run's sandbox. Recorded as unknown rather than inferred from the publishing handle.

Method

Four stages, in order:

1. Decompose the answer into individual claims.
2. Route each claim to its most relevant source — a lightweight embedding model, not the generating LLM.
3. Validate factual support by natural language inference.
4. Cross-reference the source that actually supports the claim against the source the response cited.

Output is per-claim verdicts plus one global allow-or-block decision — a gate, not a report.

The ordering is the contribution. Steps 1–3 are what a factuality checker already does; step 4 is the one that catches a claim which passes every earlier stage.

Results

Measure	ProvenanceGuard	Comparison
Claims that should be blocked, caught	138 / 139	—
Source identified correctly	~86%	—
Reject/block F1	0.802	four other checkers, 0.436–0.783
Evaluation set: 361 expert-annotated claims from a medical agent's traces.

The margin over the best comparator is 0.019 F1 — 0.802 against 0.783 — and neither the comparators nor the confidence interval are named in anything read. The 138/139 and the ~86% are the figures that carry weight here; the F1 ranking is recorded as reported.

Significance

It is this repository's own claim-check.py problem, pointed at agents. That script exists because claude-opus-5 printed Fable 5's SWE-bench Pro as 80.0% while claude-fable-5 printed 80.3%, both cited, both passing every other check, for four days — a value attached to a citation that did not support it. CLAUDE.md's rule that "every factual statement must cite its source" is exactly the rule step 4 says is insufficient: everything else verifies a claim has a source.

For MCP — Model Context Protocol it is the first verification result framed at the protocol level. MCP's premise is many heterogeneous sources behind one interface, which is precisely the condition that makes conflation possible — and which a single-corpus RAG evaluation cannot produce.

It also lands beside a result about the reporter. Language Models Are "Insecure" Reporters, captured the same day, measures a model omitting the flaw that changes the conclusion. This measures a model citing the wrong source for a true statement. Both are failures of an account of work rather than of the work, and neither is visible to a reader checking whether a citation is present.

Capture note: this is a June paper read in October. arXiv 2606.18037 surfaced only because HuggingFace blogged it on 2026-09-29, arriving as prefetch candidate #27. At weight 1.5 — the highest row in interests.md — a three-month lag on an MCP verification result is a gap in the intake, not a quiet period.

Open Questions

• Whether the embedding router is the weak link. ~86% correct source identification is the ceiling on step 4, and 14% is not small.
• Cost per answer. Decompose + route + NLI per claim, on every response, with no latency or price figure given.
• Whether 361 claims from one medical agent generalises. One domain, one trace set.
• What a global block does to a useful answer. No figure is offered for correct claims blocked alongside a conflated one.
• Authorship and independence — unresolved above.

Cite

arXiv 2606.18037. Read via Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents, HuggingFace Blog, 2026-09-29 (source).