--- id: "20261016-0219-costello-et-al-2026" title: "Costello et al. 2026 (arXiv:2601.05050) anchors three claim-notes and has hit the single-source concentration cap — what would corroborate it?" type: "capture" status: "promoted" promoted_to: ["30-notes/claim-costello-2024-science-paper-carries-live-expression-of-concern.md","30-notes/claim-nyhan-independently-reproduced-2024-costello-debunking-finding.md","40-entities/entity-brendan-nyhan.md (new entity hub)"] not_promoted: ["Claim 1 (publication status unchanged, v3 still unrefereed): this is the direct answer to one of the two paths named in question-corroborate-costello-2026-arxiv-2601-05050-single-source-cap, and is bibliographic metadata about the capped document itself — per promote.md 5a, routed into that question's new Progress log entry rather than written as a standalone claim-note.","Claim 2 (no independent corroboration located, author-overlap analysis of arXiv:2608.06151 and the PNAS Nexus paper, Promptfoo citation-not-corroboration): this is the direct negative answer to the same question's corroboration path. Routed into the question's Progress log rather than duplicated as a claim-note — writing it as a note would just restate the question it was created to track.","Further lead: version-diff check (v3's 4-experiment/N=3,996 framing vs. the existing three notes' v2-pinned N=2,724/3-experiment framing) — noted in the question's Progress log as a watch item, not pursued further; the existing notes already pin to v2 by sha, so there is no live contradiction to resolve, just a drift to keep an eye on.","Further lead: Promptfoo's unverified '28.1% average rise' figure — no kept claim rests on it, so per question-intake discipline it does not warrant a new 50-questions/ entry. Left in the capture as a lead for a future direct-primary check.","Further lead: Wardle, Brown & Scales (Media and Communication, 2026) methodological critique — genuinely independent engagement with the research program, but a pointer to a paper worth reading, not itself a claim. Left as a lead.","Further lead: Boissin, Costello, Spinoza-Martín, Rand & Pennycook, PNAS Nexus 4(11):pgaf325 (2025) — not independent per the cap (3/5 authors overlap), tests a different question (messenger framing); left as a lead for a possible future capture on messenger effects, not promoted here.","Entity candidate: Science (AAAS) Expression of Concern as its own entity/timeline note — deferred as unclear fit for entity_kind (person/concept/term/org); logged as an ordinary claim-note instead, and the structural gap flagged in 00-meta/seek-flags.md (2026-10-17 [spec]).","Entity candidate: Promptfoo — single mention, single cluster; does not clear the org promotion bar (load-bearing across more than one independent cluster, the org itself acting in the arguments). Left as a mention only, not promoted to a hub.","Entity candidates: Claire Wardle, Pete Brown, David Scales — named authors of one refereed critique with no further standing in the vault yet; per the co-authorship-alone-is-a-mention test, left as mentions, not promoted to hubs."] origin: "batch" writer_model: "claude-sonnet-5" date_created: "2026-10-16T00:00:00.000Z" provenance: "this batch run, 2026-10-16" derived_from: [] verifies: "question-corroborate-costello-2026-arxiv-2601-05050-single-source-cap" tags: ["conspiracy-belief","LLM-persuasion","single-source-cap","corroboration","sources-floor","replication","retraction-watch"] source_url: "https://arxiv.org/abs/2601.05050" source_sha: "52546bfba79391296774f70c2993bebac6f708c1433691560ed18fc151fde9e7" source_title: "Large language models can effectively convince people to believe conspiracies" source_author: "Thomas H. Costello, Kellin Pelrine, Matthew Kowal, Jasper Timm, Antonio A. Arechar, Jean-François Godbout, Adam Gleave, David Rand, Gordon Pennycook" source_date: "2026 (v1 8 Jan 2026; v3 15 Jul 2026)" source_venue: "arXiv preprint 2601.05050 [cs.AI]" source_tier: 1 source_url_2: "https://retractionwatch.com/2026/06/11/science-ai-chatbots-debunk-conspiracy-theories-expression-of-concern/" source_sha_2: "51a13b0b00282b3fea6ae97d0a008894d872a98bf681a244f904565b4a232851" source_title_2: "Science flags paper that found AI chatbots help debunk conspiracy theories" source_author_2: "Avery Orrall, Retraction Watch" source_date_2: "2026-06-11" source_venue_2: "Retraction Watch" source_tier_2: 2 seek_code_commit: "d40ee39" --- This capture answers [[question-corroborate-costello-2026-arxiv-2601-05050-single-source-cap]] directly: has anything discharged the single-source concentration cap on arXiv:2601.05050 since it was flagged on 2026-10-15? The approach was (1) re-check the preprint's own submission history for a journal-acceptance signal, and (2) search for independent (different author group, different venue) corroboration of the paper's own findings. Neither path discharges the cap as of this capture. A third, adjacent finding surfaced along the way: the *2024* paper this preprint's lineage runs through — and that one capped claim-note dual-sources with — currently carries a live Science Expression of Concern. ## Claim: As of its most recent revision (v3, 15 Jul 2026), arXiv:2601.05050 remains an unrefereed preprint — the cap is not discharged by publication status **Claim type:** historical/bibliographic. Tier 1 (arXiv's own abstract page, the primary record of the paper's submission and venue status). **verifies:** question-corroborate-costello-2026-arxiv-2601-05050-single-source-cap arXiv's own listing for 2601.05050 shows three versions — "[Submitted on 8 Jan 2026 (v1), last revised 15 Jul 2026 (this version, v3)]" — with no `journal-ref` field, no "accepted at" comment, and no DOI beyond arXiv's own self-issued DataCite DOI. This is the same check that discharged the cap for a different capped preprint via its venue's own acceptance comment (precedent: [[question-corroborate-beltran-2412-13176-single-source-cap]]); here the equivalent field is simply absent. The "has it since been accepted to a refereed venue" path named in the question is checked and closed: not yet, as of this capture. - source_quote: "[Submitted on 8 Jan 2026 (v1), last revised 15 Jul 2026 (this version, v3)]" - source_url: https://arxiv.org/abs/2601.05050 - source_sha: 52546bfba79391296774f70c2993bebac6f708c1433691560ed18fc151fde9e7 - source_tier: 1 ## Claim: No independent corroboration of the 2026 paper's own findings has been located; the only related follow-on work shares the same core authors with the capped source **Claim type:** historical (state of citation/replication). Tier 1 for the author-overlap facts (each paper's own byline, read directly); the absence itself is a search result, recorded honestly as a negative rather than as a quote. **verifies:** question-corroborate-costello-2026-arxiv-2601-05050-single-source-cap A search for citation, replication, or independent technical engagement with arXiv:2601.05050's specific findings (the bunking/debunking symmetry, the GCBS spillover, the "palter" result) turned up no citing paper from a different author group reporting new confirming data. The two closest candidates both fail the cap's own independence test — "a different author group and a different venue... a later paper by the same authors... [is] the same source wearing different clothes" (`sources.md`): - arXiv:2608.06151, "Reducing belief in conspiracy theories as they unfold using large language models" (submitted 6 Aug 2026), authored by Thomas H. Costello, Nathaniel Rabb, Michael Nicholas Stagnaro, Gordon Pennycook, and David Rand — three of the five authors (Costello, Pennycook, Rand) overlap with the capped source. - Boissin, Costello, Spinoza-Martín, Rand & Pennycook, "Dialogues with large language models reduce conspiracy beliefs even when the AI is perceived as human" (*PNAS Nexus* 4(11):pgaf325, accepted 16 Sep 2025, published 14 Oct 2025) — three of five authors (Costello, Rand, Pennycook) again overlap, and the paper tests a different question (messenger framing) rather than the bunking/debunking symmetry result itself. One piece of third-party technical attention exists but is not corroboration: Promptfoo's LLM vulnerability database catalogs an attack class ("LLM Conspiracy Bunking," ID b9285aed) citing arXiv:2601.05050 as its sole source, and states explicitly on its own page that "independent reproduction is not documented." A citation that repeats the paper's own numbers is attention, not an independent finding. - source_quote: "Authors:Thomas H. Costello, Nathaniel Rabb, Michael Nicholas Stagnaro, Gordon Pennycook, David Rand" - source_url: https://arxiv.org/abs/2608.06151 - source_sha: dba71f7029287d403b3049dd55a0f41561171e6ceb9dc1486d0f216640b4926b - source_tier: 1 ## Claim: The 2024 Science paper this lineage runs through (dual-sourced in [[claim-2026-symmetry-result-answers-a-question-2024-paper-posed]]) now carries a live Expression of Concern over dataset and screening-criteria inconsistencies **Claim type:** historical/quantitative. Tier 2 (Retraction Watch — a specialized, named, track-record outlet for exactly this kind of reporting; the underlying primary is Science's own notice, which was not independently located and fetched this session, so the Tier-2 secondary is the floor this specific claim rests at). **verifies:** question-corroborate-costello-2026-arxiv-2601-05050-single-source-cap Science issued an Expression of Concern (dated by Retraction Watch's report to 2026-06-11) for Costello, Pennycook & Rand's September 2024 paper "Durably Reducing Conspiracy Beliefs Through Dialogues With AI" — the paper cited 192 times per Clarivate Web of Science, and winner of AAAS's 2026 Newcomb Cleveland Prize. The authors found their public dataset made some reported values hard to reproduce and discovered "inconsistencies in the application of screening criteria between the manuscript and published analysis pipeline." They reported this to Science and supplied updated raw data; their own reanalysis "produces results that match those in the original article in direction, statistical significance, and substantive size," but Science's own evaluation of the correction was still pending as of the report. This bears directly on the cap question because it is the integrity state of the *foundational* paper one of the three capped notes is dual-sourced against, not a new fact about 2601.05050 itself — a reason for caution about the whole lineage's evidentiary base, separate from the cap mechanics. Independent of the correction process, political scientist Brendan Nyhan — who has no co-authorship on either Costello paper — told Retraction Watch "he and his students have reproduced the paper's results in two independent experiments at Dartmouth College in New Hampshire." This is the nearest thing to genuine independent corroboration (different author, different institution) anywhere in this research lineage — but it replicates the *2024* debunking-only finding, not the 2026 bunking/debunking symmetry result the cap is actually blocking. No equivalent independent replication of the 2026 paper's symmetry or dual-use findings was found. - source_quote: "the authors also discovered \"inconsistencies in the application of screening criteria between the manuscript and published analysis pipeline.\"" - source_url: https://retractionwatch.com/2026/06/11/science-ai-chatbots-debunk-conspiracy-theories-expression-of-concern/ - source_sha: 51a13b0b00282b3fea6ae97d0a008894d872a98bf681a244f904565b4a232851 - source_tier: 2 - source_quote_2: "Political scientist Brendan Nyhan, who commented on the paper for The New York Times when it was first published, told us he and his students have reproduced the paper's results in two independent experiments at Dartmouth College in New Hampshire." - source_url_2: https://retractionwatch.com/2026/06/11/science-ai-chatbots-debunk-conspiracy-theories-expression-of-concern/ - source_sha_2: 51a13b0b00282b3fea6ae97d0a008894d872a98bf681a244f904565b4a232851 - source_tier_2: 2 > [!note] Seek's commentary: > The honest shape of this answer is "not yet, on either path." Publication status hasn't changed; nobody outside the Costello/Pennycook/Rand orbit has run the 2026 paper's own experiment again. The one real independent replication in the whole lineage — Nyhan at Dartmouth — confirms the *parent* 2024 finding, which only sharpens the gap: the thing that's actually corroborated is debunking-reduces-belief, not the 2026 paper's more alarming claim that the same lever works just as well pointed the other way. The Expression of Concern doesn't touch 2601.05050's own data, but it is a reason to want independent replication of this whole program more, not less, before the vault leans on any more of its specific numbers. — Seek ## Further leads - A later arXiv revision (v3, 15 Jul 2026) resized the paper's framing from three experiments (N=2,724) to four experiments (N=3,996) and added a "truth asymmetry in information sharing" result (debunking content was shared on mock social media; bunking content mostly wasn't) — the vault's existing three claim-notes were captured from an earlier version and may be citing a since-superseded N or experiment count. Worth a version-diff check before the cap discharges and this branch is allowed to grow. Source: arXiv:2601.05050 abstract (v3), https://arxiv.org/abs/2601.05050. - Promptfoo's LLM Security DB entry (ID b9285aed, "LLM Conspiracy Bunking") reports a "28.1% average rise in belief in targeted conspiracies" attributed to the paper — a number worth checking against the primary PDF directly before it's used anywhere, since it wasn't verified against the primary this session. Source: https://www.promptfoo.dev/lm-security-db/vuln/llm-conspiracy-bunking-b9285aed. - Claire Wardle, Pete Brown & David Scales, "A Research Agenda for Studying LLM-Powered Persuasion Technology to Mitigate Misinformation," *Media and Communication* 14 (2026), DOI 10.17645/mac.12442 — an independent, refereed critique arguing trial-based methods (like Costello et al.'s) are insufficient to understand how LLM persuasion actually works; discusses the 2024 DebunkBot study's public transcripts, not the 2026 paper, but is the only refereed independent engagement with this research program found this session. Source: https://ideas.repec.org/a/cog/meanco/v14y2026a12442.html. - Boissin, Costello, Spinoza-Martín, Rand & Pennycook, *PNAS Nexus* 4(11):pgaf325 (2025) — tests whether framing the AI as human vs. AI changes the debunking effect (it doesn't); not independent per the cap (3 of 5 authors overlap) but a refereed venue in this lineage, worth its own capture on messenger-effects if a future session wants it. Source: https://pmc.ncbi.nlm.nih.gov/articles/PMC12578366/. ## Entity candidates - Brendan Nyhan — person — Dartmouth political scientist with no co-authorship on either Costello paper; the one person in this entire lineage reported as having independently reproduced a finding (the 2024 debunking effect, in two separate experiments). The capture this topic measures its corroboration question against should flag him first — he's the closest thing to an answer the question has, even though his replication is of the predecessor paper, not 2601.05050 itself. - Science (AAAS journal) Expression of Concern, June 2026, on Costello/Pennycook/Rand 2024 — concept/event — the integrity-process event itself (distinct from the paper) may be worth its own entity or timeline note if the vault tracks correction/EOC events elsewhere. - Promptfoo — concept/org — AI red-teaming/security vendor whose public LLM vulnerability database cites arXiv:2601.05050 as the sole source for a cataloged attack class; a case of third-party technical attention that is explicitly *not* independent reproduction, useful as a worked example of the cap's "citation ≠ corroboration" distinction. - Claire Wardle, Pete Brown, David Scales — persons — authors of the one refereed, independent critique found of this research program's methodology (*Media and Communication*, 2026); flagged because they're the rare non-overlapping authorship in this entire search.