---
id: "20261021-0223-has-ai-safety-or"
title: "Has AI-safety/NLP-evaluation literature named and studied tool-access fabrication as distinct from ordinary hallucination?"
type: "capture"
status: "promoted"
origin: "batch"
writer_model: "claude-sonnet-5"
date_created: "2026-10-21T00:00:00.000Z"
provenance: "this batch run, 2026-10-21"
derived_from: []
tags: ["ai-safety","nlp-evaluation","hallucination","tool-use","fabrication","grounding","rag","citation-hallucination","source-reliability"]
seek_code_commit: "d40ee39"
promoted_to: ["30-notes/claim-sethi-2026-fabrication-after-tool-failure-structural-not-lexical-trigger.md (new claim-note, status: seedling — Sethi et al. 2026, structural vs. lexical/topical/instructed trigger)","30-notes/claim-do-rosario-2026-grounding-violation-parametric-leakage-distinct-from-factual-hallucination.md (new claim-note, status: seedling — Do Rosario, Younes & Pires 2026, 'grounding violation' / 'parametric leakage' terminology)","30-notes/claim-rao-wong-callison-burch-2026-fabricated-snippets-named-not-yet-systematically-studied.md (new claim-note, status: seedling — Rao, Wong & Callison-Burch 2026, 'fabricated snippets' named and flagged as an open gap)","30-notes/claim-riker-2026-grounding-accuracy-fabrication-resistance-separable-capabilities.md (new claim-note, status: seedling — RIKER/Roig 2026; headline quant figures carry [unverified-quant], routed to a question)","40-entities/entity-fabrication-after-tool-failure.md (new watching stub, first_seen 2026-10-21)","40-entities/entity-grounding-violation.md (new watching stub, first_seen 2026-10-21; body also covers 'parametric leakage' as the named mechanism rather than a separate stub)","40-entities/entity-urlhealth.md (new watching stub, first_seen 2026-10-21)","50-questions/question-corroborate-riker-2026-hallucination-rate-range.md (new question, status: open — routes the RIKER [unverified-quant] figures to independent-corroboration tracking)"]
not_promoted: ["Further leads section (Mata v. Avianca; Perplexity medical-reviews incident; GhostCite; CiteAudit; Singh 2026; Healy/Srinivasan/Madathil/Wu 'tool bypass'; Farquhar et al. semantic-entropy confabulation) — none independently fetched this session; left as leads for a future capture rather than promoted on secondhand citation.","[[claim-bollen-reports-unverified-aisvs-concurrent-agent-toctou-requirement]] / [[observation-webfetch-fabricated-aisvs-issue-640-pr-641-across-two-sessions]] — correctly not duplicated (capture already points at them as the vault's own GitHub-API incident); flagging in the journal that the second note's 'fabrication' framing was itself corrected on 2026-07-28 to 'not fabrication, a path-history artifact,' which the capture's premise does not reflect.","Kalai et al. (confidence-target/calibrated-guessing prior work) entity candidate — declined. Named only as background the Do Rosario paper builds on; not independently fetched this session. UNSURE per the entity spec's promotion test.","Rashkin et al. 2023 'Attributable to Identified Sources' entity candidate — declined. Named only as a contrast point in Rao/Wong/Callison-Burch; not independently fetched. UNSURE.","PICARD entity candidate — declined. Roig's own earlier framework, named but not independently fetched this session. UNSURE.","Ji et al. 2023 'Survey of Hallucination in NLG' entity candidate — declined a new hub. Already represented in the vault (as Ji et al. 2022) via [[entity-artificial-hallucination]] and its claim-notes; not duplicated. Possible year discrepancy (2022 vs. 2023) noted but not load-bearing to any claim promoted here.","Delip Rao, Eric Wong, Chris Callison-Burch; Alden Do Rosario, Hussein Younes, Felipe Pires; JV Roig; Arham Sethi/Vyas Raina/Ivaxi Sheth et al. — person entity candidates declined. Each appears in exactly one paper's worth of this vault's work so far, with no independent recurring thread; co-authorship alone is a mention, not a hub (Cali ruling 2026-10-01). Named as source_author on the relevant claim-notes instead.","'parametric leakage' as its own standalone entity stub — declined; folded into entity-grounding-violation's body/connects_to as the named mechanism, to avoid minting two near-duplicate stubs from the same paper's terminology."]
---


Core question (from the hook): has the literature named and studied the
specific failure mode this vault hit three times in one week — a
search/fetch tool confidently fabricating plausible, internally-coherent
content when it can't or didn't actually read something verbatim — as
*distinct from* ordinary open-ended hallucination? Four 2026 preprints,
found independently, answer this in pieces: the mechanism has multiple
names across sub-literatures (tool-use evaluation, RAG evaluation, citation
verification) and at least one paper explicitly isolates the structural
trigger (tool returned nothing usable vs. tool returned something), but no
single paper unifies all three of this vault's cases (a GitHub API state, an
oral-history quote, a paraphrased sentence) under one name. The closest
single match — fabricated quoted snippets — is explicitly flagged by its own
authors as under-studied.

## Claim: A 2026 benchmark names "fabrication after tool failure" and shows it is triggered structurally (whether failure is signalled), not lexically — the clearest existing distinction from ordinary hallucination

A preprint isolates exactly the choice point this vault's incidents sit on: a
tool call completes, the payload is unusable, and the model must either
report the failure or assert a value anyway. It names the second outcome
"fabrication after tool failure" and reports it is common and prompt-
invariant: under a neutral, deployment-style system prompt, 14.10% of
responses were dishonest, and the rate was governed almost entirely by
whether the tool's response envelope signalled the failure — 0.0% dishonest
when the tool returned an explicit error status, 45.3% dishonest when it
returned a success-like status carrying a redacted, corrupted, stale,
malformed, empty, or truncated value. The paper explicitly argues this is a
different trigger than the hallucination literature it reviews: existing
abstention/refusal/hallucination work finds the trigger is "lexical, topical
or instructed," where this benchmark's trigger is "structural" — the prompt
is held fixed and benign, and what varies is only whether the tool
delivered anything usable.

source_url: https://arxiv.org/abs/2609.14758
source_sha: 1e787c3783ee856a4e4b258f271c001ef34bfe0f75f7aa28a1877ddaf525da07
source_title: "Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return"
source_author: "Arham Sethi, Arsen Kenzhebayev, Saanvi Paturi, Vatsal Raina, Vyas Raina, Ivaxi Sheth"
source_date: 2026-09-13
source_venue: "arXiv preprint, cs.SE (2609.14758v1)"
source_quote: "Under a deployment-style system prompt, 14.10% of responses are dishonest: the model either asserts a value the payload cannot support or declines while citing a fabricated policy or capability limit."
source_quote_2: "When the tool returns status:error, dishonesty is absent (0.0%); when it returns status:ok with a redacted, corrupted, stale, malformed, empty or truncated value, dishonesty reaches 45.3%."
source_quote_3: "In all of this work the trigger is lexical, topical or instructed. Ours is structural: the prompt is held fixed and benign, and what varies is whether the tool the model was told to rely on returned anything usable."
source_tier: 1

> [!note] Seek's commentary: This is the nearest thing found to a direct
> answer. The GitHub-API and oral-history incidents both look like the
> "success-like status carrying no usable content" branch this paper
> measures at 45.3% — WebFetch and web-search returned *something*
> (a rendered page, a search summary) rather than an explicit error, and the
> something was treated as sufficient grounding. That the benchmark's own
> fix (requiring the model to emit `retrieval_status: OK or FAILED` before
> answering) is a verbalized version of exactly the discipline
> [[reflection-recurring-tool-summary-is-not-the-source]] independently
> arrived at — treat a tool's rendering as unverified until the primary is
> read directly — is worth flagging to Cali as convergent validation,
> not just a citation. — Seek

## Claim: RAG-evaluation literature distinguishes "grounding violation" (answered without support) from "factual hallucination" (answered wrong), naming "parametric leakage" as the mechanism that turns an ungrounded answer into confident fabrication

A second preprint, building on an earlier confidence-target analysis of
hallucination-as-miscalibrated-guessing, sets up terminology for exactly the
distinction the core question asks about. It defines four terms and treats
three of them as analytically separate: a "factual hallucination" is simply
a wrong answer; a "grounding violation" is the system answering without
support from its knowledge base, independent of whether that answer happens
to be correct; and "parametric leakage" is named as the mechanism behind
most grounding violations — the answer comes from the backbone model's
training knowledge rather than from anything actually retrieved. The paper
states directly that grounding violations are what make factual
hallucinations dangerous in the retrieval setting: "whenever parametric
memory does not happen to contain the right answer, an ungrounded response
becomes a confident fabrication." This is a definitional separation (what
distinguishes a correct-but-unverified lucky guess from a sourced answer),
not yet a quantitative claim about this vault's specific incidents, so it is
recorded as terminology rather than as a measured rate.

source_url: https://arxiv.org/abs/2608.26385
source_sha: 7ddd604a9b10eea5d52df6653537315a4f4301f8198c58b56c33c8e945e956a2
source_title: "Why RAGs Hallucinate: Penalty-Aware Evaluation of Retrieval-Augmented Generation Systems with Knowledge-Gap Canaries"
source_author: "Alden Do Rosario, Hussein Younes, Felipe Pires"
source_date: 2026-08-26
source_venue: "arXiv preprint, cs.CL (2608.26385v1); CustomGPT.ai"
source_quote: "Grounding violation: the system answers without support from its knowledge base, whether or not the answer is factually correct."
source_quote_2: "Grounding violations are the mechanism that produces dangerous factual hallucinations: whenever parametric memory does not happen to contain the right answer, an ungrounded response becomes a confident fabrication."
source_tier: 1

## Claim: A citation-reliability study names "fabricated snippets (quoted text not appearing at the cited page)" as a distinct failure mode from URL/reference hallucination — and flags it, in its own words, as not yet systematically studied

This is the closest single match to the vault's oral-history-quote and
paraphrased-sentence incidents specifically. A study of citation URL
reliability across commercial LLMs and deep research agents (10 models,
53,090 + 168,021 URLs) explicitly scopes its measurement to whether a cited
URL exists at all, and in doing so names two adjacent failure modes it
deliberately leaves unmeasured: fabricated snippets (quoted text that does
not appear at the page it is cited to) and invented bibliographic entries
(plausible but wrong metadata). The paper states these "are documented
qualitatively but require separate systematic study" and repeats the point
in its conclusion as one of two most-pressing open directions. In other
words: as of this paper, the specific mechanism this vault hit three times —
a tool producing a plausible quoted/paraphrased string that is not actually
present in the source it is attributed to — has been named and flagged as
real, but not yet the subject of its own benchmark or quantified study.

source_url: https://arxiv.org/abs/2604.03173
source_sha: e8145a9f62f2a2bf0c5f3e2607160c56e010d02da965b5f65dbfa8f07ddde06c
source_title: "Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents"
source_author: "Delip Rao, Eric Wong, Chris Callison-Burch"
source_date: 2026-04-03
source_venue: "arXiv preprint, cs.CL (2604.03173v1); University of Pennsylvania"
source_quote: "Two additional failure modes—fabricated snippets (quoted text not appearing at the cited page) and invented bibliographic entries (plausible but incorrect metadata such as wrong authors or non-existent DOIs)—are documented qualitatively but require separate systematic study."
source_tier: 1

> [!note] Seek's commentary: This is the most honest answer the literature
> currently gives to the core question — not "yes, studied," not "no,"
> but "named, and explicitly marked as a gap" by the people closest to it.
> That gap is where this vault's three 2026-07 incidents sit.

## Claim: An enterprise-document benchmark finds grounding accuracy (locating facts that exist) and fabrication resistance (not inventing facts that don't) are empirically separable capabilities, not two faces of one hallucination score

A fourth preprint built a benchmark that generates synthetic corpora from
known ground truth specifically so it can ask hallucination-probe questions
with a verifiably correct "the information does not exist" answer, separate
from ordinary extraction questions. Scoring 31-33 models, it reports as a
headline finding that grounding ability and hallucination resistance are
distinct capabilities: a model can excel at finding facts that are actually
present while still readily fabricating answers to questions about entities
that are not. At 32K context, for example, hallucination rates across
otherwise-capable models ranged from 7.9% to 96.3%,[unverified-quant —
needs primary corroboration beyond this single preprint] with no reported
tight correlation to single-document extraction accuracy — one model (GLM-
4.5) combined 93.6% single-document accuracy with 0.0% fabrication, while
another (Qwen3-Next-80B) combined 91.4% single-document accuracy with 40.9%
fabrication. Note: this benchmark's own "Coherent Simulated Universe" is the
authors' name for a *document-generation* technique (keeping synthetic
corpora internally consistent so models can't exploit randomness-driven
tells) — it is not their term for model fabrication, and should not be read
as such.

source_url: https://arxiv.org/abs/2601.08847
source_sha: 7c06356f492b9f58e143620641f324e34dfd49116e4f7d91d8cf7664f3e929ba
source_title: "Scalable and Reliable Evaluation of AI Knowledge Retrieval Systems: RIKER and the Coherent Simulated Universe"
source_author: "JV Roig"
source_date: 2026-01-15
source_venue: "arXiv preprint, cs.CL (2601.08847v2); Kamiwaza AI"
source_quote: "grounding ability and hallucination resistance are distinct capabilities - models excelling at finding facts that exist may still fabricate facts that do not"
source_tier: 1

## Further leads

- Mata v. Avianca (2023) and the Perplexity-fabricated-medical-reviews incident, cited as real-world harm examples in Rao/Wong/Callison-Burch (2604.03173) — worth their own legal/journalistic sourcing if pursued.
- GhostCite (Xu et al. 2026, 13 LLMs × 40 domains, 375K citations) and CiteAudit (Yuan et al. 2026) — citation-hallucination benchmarks cited secondhand in 2604.03173's related work; not yet independently fetched.
- Singh (2026), cited in Sethi et al. (2609.14758 §2.2) as the closest prior work: "injects silent failures across tool stubs, reports widespread fabrication and increased unfaithful refusal under added safety language" — not yet independently fetched.
- Healy, Srinivasan, Madathil, Wu (Amazon, arXiv:2601.05214) on "tool bypass" — detecting via internal representations when a model simulates a tool's output instead of invoking it; a detection-mechanism angle adjacent to but distinct from the core question.
- Farquhar et al. (Nature, 2024) semantic-entropy "confabulation" — a resampling-inconsistency definition of confabulation, a different axis from tool-access fabrication; flagged here as a near-miss, not fetched as primary for this capture.
- [[claim-bollen-reports-unverified-aisvs-concurrent-agent-toctou-requirement]] and [[observation-webfetch-fabricated-aisvs-issue-640-pr-641-across-two-sessions]] are this vault's own prior instance of the GitHub-API case the hook names; not duplicated here.

## Entity candidates

- Kalai et al. — concept/paper — the earlier confidence-target analysis ("Why RAGs Hallucinate" builds directly on this: binary grading makes guessing dominate abstention) that the grounding-violation terminology is positioned against; the foundational prior work, not a co-author of either paper in this capture.
- Rashkin et al. (2023), "Attributable to Identified Sources (AIS)" — concept/paper — the earlier attribution framework that Rao/Wong/Callison-Burch explicitly contrast their "does the source exist?" question against ("does the source support the claim?").
- PICARD (Probing Intelligent Capabilities via Artificial Randomized Data) — concept — Roig's own earlier framework that RIKER is explicitly "an offshoot of"; the direct methodological ancestor of the Coherent Simulated Universe approach.
- Ji et al. 2023, "Survey of Hallucination in Natural Language Generation" — concept/paper — the intrinsic/extrinsic hallucination taxonomy cited as background by multiple of these papers; the field's standard reference point that none of the four papers above claims to supersede.
- Delip Rao, Eric Wong, Chris Callison-Burch — person — authors, University of Pennsylvania, "Detecting and Correcting Reference Hallucinations."
- Alden Do Rosario, Hussein Younes, Felipe Pires — person — authors, CustomGPT.ai, "Why RAGs Hallucinate."
- JV Roig — person — author, Kamiwaza AI, RIKER / PICARD.
- Arham Sethi, Vyas Raina, Ivaxi Sheth (and co-authors) — person — Spark AI Research / Apta AI, "Fabrication After Tool Failure."
- "fabrication after tool failure" — concept — the named failure mode most directly on point for this vault's hook; candidate for its own concept note.
- "grounding violation" / "parametric leakage" — concept — terminology separating ungrounded-but-correct answers from factual wrongness; candidate for its own concept note distinguishing it from plain "hallucination."
- urlhealth — concept/tool — open-source URL liveness/hallucination classifier released by Rao, Wong & Callison-Burch; a concrete mitigation tool, not yet described anywhere else in the vault.