Does AI-safety/interpretability literature have a specific name and mechanism for plausible-sounding fabricated intermediate claims — and does it explain why two independent generations converge on the same one?
The hook that sent this capture looking is two fabrication catches from one day (2026-07-12): a writer model inventing a biographical detail ("blinded at 29 by a WWI head wound") and a search tool independently returning a phantom citation (an Amari application to developmental biology) — two separately-run generations landing on fabrications, raising the question of whether they could land on the same fabrication, and why.
The question splits in two, and the literature answers the halves unevenly. Part 1 (is there one name for the failure mode itself) is already answered in this vault by claim-rao-wong-callison-burch-2026-fabricated-snippets-named-not-yet-systematically-studied: no single term unifies it, and the closest match is explicitly flagged by its own authors as unstudied — see Claim 1 below, which is a pointer, not new research. Part 2 (does the literature explain convergence on the same fabrication across independent generations) is genuinely new ground for this vault, and is the real content of this capture: a body of 2025-2026 work on "correlated errors" / "algorithmic monoculture" across LLMs shows that independent models and runs land on the same wrong answer far more than chance would predict, offers a candidate representational mechanism for why, and — importantly — also documents the opposite outcome (three different models fabricating three different false specifics for the same question), which sharpens rather than resolves the puzzle.
Claim: No single term unifies "plausible fabricated intermediate claim that fails against a primary source" — the two nearest named mechanisms (citation-specific "fabricated snippets" and reasoning-specific "bullshitting") sit in different sub-literatures and neither is general-purpose
This is a synthesis across two notes already in the vault, recorded here so the capture states the direct answer to Part 1 rather than leaving it implicit. claim-rao-wong-callison-burch-2026-fabricated-snippets-named-not-yet-systematically-studied documents that a 2026 citation-reliability study names "fabricated snippets" (quoted text not appearing at its cited page) as a failure mode distinct from URL hallucination, and states plainly that it "require[s] separate systematic study" — named, not yet measured. Separately, cot-faithfulness-anthropic-biology documents that Anthropic's circuit-tracing work names a structurally similar but distinct phenomenon — a model generating a plausible chain-of-thought step with no corresponding computation in its attribution graph — "bullshitting," borrowing Frankfurt's philosophical term for assertion indifferent to truth, and this one is mechanistically verified (via the attribution graph showing no evidence of real calculation). Both name a model producing fluent, structured content decoupled from the thing it claims to reflect (a cited document; an actual computation); neither paper treats the other's case as an instance of its own term, and no paper found in this capture's search treats "fabricated snippet" and "bullshitting" as the same mechanism under different names.
Claim: LLM errors are empirically correlated well above chance, and the correlation gets stronger, not weaker, as models get more individually accurate — a 2025 large-scale study names this "correlated errors" and shows it is not reducible to shared developer or architecture alone
A study evaluating 349 models on a HuggingFace leaderboard, 71 models on Stanford's HELM benchmark, and 20 models on a resume-screening task measures, for every model pair, how often their wrong answers coincide (conditional on both being wrong). Random disagreement would predict roughly 1/3 agreement on HELM's typical multiple-choice format; the actual mean agreement rate across model pairs was 60% on HELM and 42.3% on HuggingFace, with 97.5% and 100% of pairs respectively exceeding the random baseline. A regression shows shared developer, shared base architecture, and similar model size all predict higher agreement — but even after conditioning on those factors, pairs of models that are each individually more accurate still show higher error correlation, meaning newer models that look different on the surface are converging in their mistakes, not diverging.
source_url: https://arxiv.org/abs/2506.07962 source_sha: 12b4d40ebd8a7118c3ffd2fe7a458e0bedaef26df67d42486b9bfc32a31a1604 source_title: "Correlated Errors in Large Language Models" source_author: "Elliot Kim, Avi Garg, Kenny Peng, Nikhil Garg" source_date: 2025-06-09 source_venue: "Proceedings of the 42nd International Conference on Machine Learning (ICML 2025), PMLR 267; arXiv preprint cs.CL 2506.07962v1; Cornell University" source_quote: "on one leaderboard dataset, models agree 60% of the time when both models err" source_quote_2: "We show that LLMs have correlated errors, and that this correlation is substantially higher for individually accurate models and those by the same developer or using the same base architecture." source_quote_3: "These findings suggest that as model performance increases, models are also converging in the errors that they make." source_tier: 1
Claim: Both papers point to a candidate mechanism for convergence — shared or low-diversity "generative processes" between model pairs (approximated by representational similarity) — rather than developer/architecture identity alone, as the deeper predictor of correlated fabrication
The Correlated Errors paper itself does not test a representational mechanism directly, but points to one in its related work: citing findings that embeddings across different models can be translated into each other and that internal activations are consistent at similar depths across different networks, it states "these findings suggest a mechanistic explanation for our results" — i.e., convergent internal representations, not merely shared training data or shared code, as the likely reason error correlation persists even across models with different architectures and providers. A second, more recent and not-yet-peer-reviewed preprint builds a direct test of this idea: it defines "generative-process diversity" — how different the underlying processes capable of producing two models' outputs are, estimated via compression-distance between raw outputs rather than by comparing the outputs' meaning — and finds, across 38 models and ten disjoint benchmark families, that higher generative-process diversity between a model pair predicts lower correlated failure, beyond what semantic similarity or individual capability explain (partial rank association −0.216, negative on all ten benchmarks tested). [unverified-quant — single, very recent preprint, not yet independently corroborated] The same preprint explicitly frames the broader empirical pattern it is building on: "Models from different frontier labs select the same wrong answers more often than expected from their individual error distributions."
source_url: https://arxiv.org/abs/2609.03422 source_sha: ca75767a37fad8740f1821560a289a67764b98dc54644a002a29613d9f6b1a78 source_title: "Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models" source_author: "Ross Tieman, Evan Markou" source_date: 2026-09-03 source_venue: "arXiv preprint, cs.LG (2609.03422v1); Australian National University (preprint, not yet peer-reviewed)" source_quote: "These results indicate that increased generative-process diversity is associated with reduced correlated failure in model pairs that is not attributable to semantic similarity or capability." source_quote_2: "Models from different frontier labs select the same wrong answers more often than expected from their individual error distributions" source_tier: 1
Claim: The same literature also documents the opposite outcome under an almost identical probe — three different models producing three different fabricated specifics for one question — showing convergence on one shared fabrication is not automatic, which sharpens rather than resolves the original puzzle
OpenAI's own account of why language models hallucinate, framing hallucination as a byproduct of a binary classification problem language models implicitly solve during training (distinguishing "valid" continuations from errors) and worsened by evaluation procedures that reward confident guessing over admitting uncertainty, includes a plain counter-example to convergence: asked for the title of a specific person's PhD dissertation, three different widely-used models each returned a different fabricated title, school, and year — "None generated the correct title or year" — and asked to count a specific letter in a word, DeepSeek-V3 "returned '2' or '3' in ten independent trials," with other models answering as high as 6 or 7. The paper's own framing treats hallucination on facts with no learnable pattern in the training data ("arbitrary" facts) as statistically expected, while distinguishing these from facts governed by a learnable regularity. That framing implies, though the paper does not test it directly, that true convergence on one specific shared fabrication (rather than scattered different guesses) would require the false detail to sit on an outsized, shared regularity in the training distribution — a common narrative trope, for instance — rather than being an arbitrary unconstrained guess. [unverified-mechanism — this last inferential step is this capture's own synthesis connecting the paper's "arbitrary vs. systematic" framing to the convergence question; the paper itself does not make or test this specific claim]
source_url: https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf source_sha: 578cea662c00ead3841f3053c18ce544f0c2edae2ea9219a534774017f3be283 source_title: "Why Language Models Hallucinate" source_author: "Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin Zhang" source_date: 2025-09-04 source_venue: "OpenAI technical report (cdn.openai.com); also arXiv preprint cs.CL 2509.04664; Georgia Tech (Vempala)" source_quote: "We argue that language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty" source_quote_2: "None generated the correct title or year (Kalai, 2001)." source_quote_3: "DeepSeek-V3 returned "2" or "3" in ten independent trials" source_tier: 1
Further leads
- "LLMs and the Madness of Crowds" (arXiv 2411.01539, 2024) — reports that across 37 models, incorrect responses are systematically correlated rather than randomly distributed, with some questions drawing the same wrong answer from models >99% of the time; not independently fetched this session, a close companion finding to the Correlated Errors claim above.
- "Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer" (arXiv 2502.12964, 2025) — uses "shared hallucinations" for within-model consistency across prompt rephrasings (a different axis than cross-model convergence); not independently fetched.
- Wu et al. (2024), cited in the Correlated Errors related-work section — reports that the distribution of code a single LLM writes has less variance than the distribution of code in its own training set, and that changing temperature/top-p/prompt does not fix this within-model "monoculture," worse for RLHF-tuned models; not independently fetched, a within-model complement to the cross-model finding above.
- Wenger & Kenett (2025), "Artificial Hivemind" — reported (secondhand, via the Tieman & Markou preprint) to show LLM responses on creative tasks are more similar to each other than human responses are to each other; not independently fetched.
- Hicks, Humphries & Slater, "ChatGPT is Bullshit" — the philosophy paper making the Frankfurt-bullshit case for LLM output generally (not specifically interpretability-verified, unlike Anthropic's circuit-tracing use of the same term); and Hannigan et al.'s derived term "botshit" for the chatbot-specific case — both leads for a dedicated concept note contrasting informal-philosophical vs. mechanistically-verified uses of "bullshitting."
- Kalai & Vempala, "Calibrated Language Models Must Hallucinate" (arXiv 2311.14648, STOC 2024) — the earlier theoretical paper (same first author as the 2025 OpenAI paper above) establishing that calibrated models must hallucinate on "arbitrary" facts at a rate tied to the share of training facts seen exactly once ("singleton" rate), with no corresponding statistical pressure for "systematic" facts; not independently fetched this session but is the direct theoretical ancestor of Claim 4's arbitrary/systematic distinction.
Entity candidates
- Harry Frankfurt — person — author of On Bullshit (1986/2005); the philosophical ancestor both Anthropic's circuit-tracing "bullshitting" term and the Hicks/Humphries/Slater "ChatGPT is Bullshit" paper measure their claims against. Flagged first per the standing blind-spot note: the foundational figure an ancestry claim rests on, not just the papers citing him.
- Kalai & Vempala, "Calibrated Language Models Must Hallucinate" (2023/STOC 2024) — concept/paper — the direct theoretical ancestor of Claim 4's OpenAI paper (shared first author), establishing the arbitrary/systematic-fact distinction the 2025 paper's framing depends on.
- Kleinberg & Raghavan (2021) and Peng & Garg (2024a) — concept/paper — the "algorithmic monoculture" and systemic-exclusion theoretical framework the Correlated Errors paper (Claim 2) is empirically testing; the foundational prior work for that paper's entire motivation, not its own contribution.
- Bommasani et al. (2022/2023) — concept/paper — originators of the "component sharing hypothesis" (models sharing components are more likely to have correlated outputs) that the Correlated Errors paper's related-work section builds on and partially re-tests at scale.
- Jha et al. (2025) and Wolfram & Schein (2025) — concept/paper — the representational-universality findings (embeddings translatable across models; activations consistent at similar depths across networks) that Correlated Errors cites as "a mechanistic explanation for our results" — the specific prior work underlying Claim 3's mechanism, not that claim's own contribution.
- Elliot Kim, Avi Garg, Kenny Peng, Nikhil Garg — person — authors, Cornell University, "Correlated Errors in Large Language Models" (Claim 2).
- Ross Tieman, Evan Markou — person — authors, Australian National University, "Inferred Generative-Process Diversity..." (Claim 3).
- Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin Zhang — person — authors, OpenAI/Georgia Tech, "Why Language Models Hallucinate" (Claim 4).
- "generative-process diversity" — concept — candidate for its own concept note: a measurable quantity (compression-distance-based, feature-free) distinct from semantic similarity, proposed as what predicts correlated model failure.
- "algorithmic monoculture" — concept — the broader economic/systemic framing (Kleinberg & Raghavan 2021 onward) that the correlated-errors literature sits inside; already adjacent to but not yet represented in this vault's concept layer.