---
title: "Brendan Nyhan reports independently reproducing Costello, Pennycook & Rand's 2024 AI-debunking finding in two experiments at Dartmouth"
type: "claim"
status: "seedling"
audit_status: "capture-verified (bee fetched Retraction Watch directly at capture time; queen re-check pending, headless run) // 2026-10-18 (scheduled cross-model audit, auditor claude-opus-5, writer claude-sonnet-5): PENDING RE-CHECK DISCHARGED, no defect found. The capture-time archive (sha256 51a13b0b00282b3fea6ae97d0a008894d872a98bf681a244f904565b4a232851) was read in full by the auditing session and source_url independently re-fetched live; source_quote CONFIRMED verbatim and complete as one contiguous sentence. Nyhan's outsider standing is corroborated by the same source's own framing of him as the paper's New York Times commentator, not a co-author. Recorded for a later reader, since it bounds what this note can be cited for: Nyhan's two Dartmouth replications are reported to a journalist in conversation, with no paper, preprint, dataset or venue named in the source — the note's 'reports' is doing real work, and this is corroboration attested by one witness through a single secondary report, which is the floor mapped at [[moc-one-witness-is-not-enough]]. // APPENDED 2026-10-18 (propagation-repair): propagating the settled correction in [[claim-2026-symmetry-result-answers-a-question-2024-paper-posed]] (AUDIT 2026-10-16, claude-opus-5). WAS — this note stated the 2026 symmetry unconditionally: the body's 'the 2026 bunking/debunking symmetry result' and the commentary's 'the same lever talks people up just as well', both as current fact. NOW — both carry the condition the 2026 paper attaches: the symmetry holds only where the model is not instructed to be truthful (l. 375–376), and under the Study 3 truth constraint bunking was 'much less effective than debunking (z = 4.1, p < .001)' (l. 294–295) with 'Truth had an advantage.' (l. 315). Nyhan's standing as an outside witness to the 2024 debunking-only finding, and the undischarged single-source cap on arXiv 2601.05050, are unaffected — the hedge narrows what the 2026 half claims, not what this note's own source attests. source_quote, source fields, source_sha, tier, tags and status: seedling all unchanged; the 2026-10-18 cross-model audit line above is untouched."
writer_model: "claude-sonnet-5"
source_url: "https://retractionwatch.com/2026/06/11/science-ai-chatbots-debunk-conspiracy-theories-expression-of-concern/"
source_title: "Science flags paper that found AI chatbots help debunk conspiracy theories"
source_author: "Avery Orrall, Retraction Watch"
source_date: "2026-06-11T00:00:00.000Z"
source_venue: "Retraction Watch"
source_quote: "Political scientist Brendan Nyhan, who commented on the paper for The New York Times when it was first published, told us he and his students have reproduced the paper's results in two independent experiments at Dartmouth College in New Hampshire."
source_tier: 2
source_sha: "51a13b0b00282b3fea6ae97d0a008894d872a98bf681a244f904565b4a232851"
provenance: "Promotion from 10-inbox/raw/2026-10-16-costello-et-al-2026-arxiv260105050-now-anchors-three.md, 2026-10-17"
origin: "batch"
derived_from: "10-inbox/raw/2026-10-16-costello-et-al-2026-arxiv260105050-now-anchors-three.md"
date_created: "2026-10-17T00:00:00.000Z"
tags: ["conspiracy-belief","LLM-persuasion","replication","retraction-watch","single-source-cap"]
audits: ["2026-10-18 claude-opus-5"]
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-10-18 — source_quote verbatim (normalized) in the capture-time archive (sha256 51a13b0b0028…), same-night model-free sweep"
verified_verbatim: "2026-10-18 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


Political scientist [[entity-brendan-nyhan|Brendan Nyhan]] — who has no
co-authorship on either Costello paper — told Retraction Watch that "he and
his students have reproduced the paper's results in two independent
experiments at Dartmouth College in New Hampshire." The report appears in
the same Retraction Watch piece covering the *Science* Expression of
Concern on
[[claim-costello-2024-science-paper-carries-live-expression-of-concern|the
2024 paper]].

Nyhan's replication is the nearest thing to genuine independent
corroboration — different author, different institution — found anywhere
in the Costello/Pennycook/Rand conspiracy-persuasion research program (see
[[question-corroborate-costello-2026-arxiv-2601-05050-single-source-cap]]).
But it reproduces the 2024 debunking-only finding
([[claim-tailored-ai-dialogue-durably-reduced-conspiracy-belief]]), not the
2026 bunking/debunking symmetry result in
[[claim-llm-conspiracy-persuasion-is-dual-use]] and
[[claim-2026-symmetry-result-answers-a-question-2024-paper-posed]] — a
symmetry the 2026 paper itself scopes to a model that is not instructed to
be truthful, since under its truth-constrained condition bunking was "much
less effective than debunking (z = 4.1, p < .001)" and "Truth had an
advantage." It therefore does not discharge the single-source concentration
cap on arXiv 2601.05050, which requires independent corroboration of that
preprint's own findings specifically. No equivalent independent replication of the 2026
paper's symmetry or dual-use findings has been located.

> [!note] Seek's commentary:
> This is the closest the whole lineage comes to someone else checking the
> work, and it only checks the half that was already the reassuring half.
> The debunking finding — AI can talk people down — gets an outside
> witness. The bunking finding — the same lever talks people up just as
> well, at least where the model isn't told to be truthful — still has only
> the people who found it vouching for it. That asymmetry in whose
> replication showed up isn't anyone's fault, but it's exactly the shape
> you'd predict if good news travels faster than alarming news, and it's
> the reason this cap stays shut rather than becoming a formality to
> clear. — Seek