---
title: "The respondent-coverage-gap branch rests three claim-notes on one uncorroborated preprint (arXiv:2610.30900) — what would corroborate it?"
type: "question"
status: "open"
date_raised: "2026-10-25T00:00:00.000Z"
tags: ["single-source-cap","corroboration","llm-evaluation","silicon-sampling","survey-methods","arxiv"]
---


As of 2026-10-25, three claim-notes rest on Kumagae & Kaneko, "Silicon Samples Match the Answers but Not the Respondents" (arXiv:2610.30900v1, submitted 22 Oct 2026) — a single author pair, one unrefereed preprint, submitted only three days before this promotion:

- [[claim-silicon-sampling-llms-undercover-human-respondents-despite-robustness-sweep]]
- [[claim-per-question-metrics-blind-to-silicon-sampling-respondent-coverage-gap]]
- [[claim-silicon-sampling-llms-split-into-off-support-and-contraction-failure-modes]]

This is the single-source concentration cap (`sources.md`, Cali 2026-07-29: **no more than three claim-notes may rest on a single unrefereed primary until at least one independent source corroborates it**). No independent source — different author group, different venue — has yet confirmed the paper's findings; the paper is too new for any citation record to exist yet. The branch is at its ceiling: the next finding pulled from this preprint does not become a new claim-note but gets routed here, and the existing three stay exactly as they are (seedling, honest, `audit_status: capture-verified`).

**The specific verification.** Find independent corroboration for the paper's load-bearing empirical claims — that silicon-sampled LLM respondents systematically fail to cover real human respondents (RCG > 1 across six tested models and two national surveys), and that the failure survives scaling and prompting interventions. Independent means a different author group applying RCG, OSR, or an equivalent respondent-level coverage metric to silicon-sampled survey data — not a later paper by Kumagae or Kaneko, a press summary of this preprint, or a revised arXiv version. A peer-reviewed venue accepting this exact paper would also discharge the cap (same authors, but a different, external quality gate).

**Candidate leads**, named by the capture itself as unread but relevant (none yet checked for independence or read directly):
- Angelina Wang, Jamie Morgenstern & John P. Dickerson, "Large language models that replace human participants can harmfully misportray and flatten identity groups," *Nature Machine Intelligence* 7(3):400–411 (2025) — different author group, different venue; cited by Kumagae & Kaneko as prior art on silicon-sampling failure, but needs a direct read to see whether it measures anything resembling respondent-level coverage rather than a different failure mode (misportrayal/flattening).
- James Bisbee, Joshua D. Clinton, Cassy Dorff, Brenton Kenkel & Jennifer M. Larson, "Synthetic replacements for human survey data? The perils of large language models," *Political Analysis* 32(4):401–416 (2024) — different author group, different venue; credited with a precursor "reduced variance" framing of what this paper calls "contraction." Worth checking whether it already measured something RCG-equivalent two years earlier.
- Williams et al. (2026)'s correlation-structure score, which this paper's Proposition 1 shows is permutation-invariant — full bibliographic details not yet captured; resolving the citation would at least establish what the prior state of the art actually measured.

**Why it matters.** The cap exists because the failure mode is invisible one note at a time: each of the three notes here is correctly sourced, correctly seedling, correctly tied to a Tier 1 primary — while the branch as a whole could still become a structure the vault believes because it is large, not because anyone outside the author pair has checked it. Corroboration discharges the cap (record it in the notes' `audit_status` and the branch may grow); contradiction is the more interesting of the two possible outcomes and is a note in its own right.

No sweep has been run yet — this question is being raised at first contact with the source, not after a search came back empty. A future session with research tools (this promotion session had none available by design) should check Google Scholar / Semantic Scholar for citations, and read the Wang/Morgenstern/Dickerson and Bisbee/Clinton/Dorff/Kenkel/Larson papers directly before assuming either corroborates or fails to.

**2026-10-29 progress note.** A fourth note touching this preprint was written this promotion — [[claim-liu-kaneko-chu-present-skew-and-kumagae-kaneko-rcg-measure-different-objects]] — judged exempt from the growth cap rather than routed here, on the reasoning that it rests on Kumagae & Kaneko's own self-description of their method and dataset reuse (low-risk, not contingent on their empirical RCG results replicating) and is jointly grounded in the peer-reviewed Liu, Kaneko & Chu (AAAI 2026) paper. The cap itself remains undischarged at three notes resting primarily on this preprint's empirical findings; no independent corroboration has landed. Flagging the judgment call here so a disagreeing auditor has somewhere to register it.

**2026-10-30 progress note.** A fifth touch, same exemption reasoning as 2026-10-29: [[claim-liu-kaneko-chu-and-kumagae-kaneko-test-disjoint-model-sets]] rests half on this preprint's methods section (which models it tested — a bibliographic fact, not an empirical result) and half on the peer-reviewed Liu et al. paper, so it was written rather than routed here. Two genuinely *empirical* findings from this same preprint were found this session and deliberately withheld from becoming new claim-notes because the cap on this preprint's results remains undischarged at three: (1) across the six evaluated LLMs, "RCG shows no consistent ordering by model family, weight availability (proprietary vs. open-weight), or model size"; (2) a non-LLM Gaussian-copula baseline beats every evaluated LLM on RCG on both surveys (Copula RCG 1.041 WVS / 1.091 ANES vs. 1.199-1.275 WVS / 1.277-1.370 ANES for the LLMs) — a striking result worth its own claim-note once corroboration lands. Both are logged for now at [[entity-respondent-coverage-gap]] rather than promoted. No corroborating or disconfirming source was found this session either; the preprint is still only ~1 week old.

**2026-10-31 progress note — the first dedicated corroboration search ran. Partial answer; question stays OPEN, cap NOT discharged.** Promoted the follow-up capture `10-inbox/raw/2026-10-26-the-respondent-coverage-gap-branch-rests-three-claim.md`, which searched for (a) any citation/replication of arXiv:2610.30900, (b) direct reads of this question's named candidate leads, and (c) any independent work applying an RCG/OSR-equivalent respondent-level coverage metric. **Bottom line: no source was found applying RCG, OSR, or an equivalent individual-respondent coverage metric to independently reproduce this preprint's numbers, and no citation of arXiv:2610.30900 itself exists yet (expected — four days old at search time).** The specific verification this question names is therefore still undischarged; the existing three branch notes stay exactly as they are (seedling, `capture-verified`).

What the search *did* establish, promoted as four new claim-notes resting on their own independent primaries (not on the capped preprint):
- [[claim-bisbee-2024-llm-survey-respondents-show-reduced-variance]] — **candidate lead resolved.** Bisbee et al. (2024, *Political Analysis*) found reduced response variance in ChatGPT personas vs. ANES, a peer-reviewed precursor to "contraction" — convergent, by a variance/regression metric, not an RCG-equivalent coverage test. Does not discharge the cap.
- [[claim-wang-2025-misportrayal-flattening-different-axis-than-coverage]] — **candidate lead resolved, as a non-corroboration.** Wang, Morgenstern & Dickerson (2025, *Nature Machine Intelligence*) measures demographic misportrayal/flattening — a different failure axis than respondent coverage. Cited by K&K as related work, but does not corroborate RCG.
- [[claim-doudkin-2026-no-respondent-query-beats-persona-panels]] — Doudkin (arXiv:2609.07305, Sep 2026): a no-respondent population query beats committed persona panels on marginal benchmarks. Independent author/venue/metric; itself an uncorroborated preprint. Convergent on the structural claim, not a replication of RCG.
- [[claim-ku-2026-cross-survey-transfer-individual-level-failure]] — Ku et al. (arXiv:2607.03091, Jul 2026): zero-shot LLMs reach only 52% on held-out individual-answer prediction (TEDS 2024). Independent; convergent, not a replication.

The third named candidate lead — Williams et al.'s correlation-structure score — was resolved bibliographically only (Williams, Weeber, Padó & Akbik, *Findings of ACL 2026*, pp. 4806–4826), not read for content; K&K's Proposition 1 shows that score is permutation-invariant across respondents (already in [[claim-per-question-metrics-blind-to-silicon-sampling-respondent-coverage-gap]]).

**New unread leads for a future session** (none read this session; below the sourcing floor for a claim-note, logged here so the search isn't re-walked): Pew Research Center, "Can AI Stand In for Human Survey-Takers? Not Really" (pewresearch.org, 30 Sep 2026) — an institutional "digital twins" validation across three ATP waves, secondary coverage reports large subgroup errors; the most promising *non-preprint* corroboration candidate, figures unverified pending a direct read. "The Conditional Superiority of Fast Silicon Sampling" (arXiv:2608.14079, NTU) — ~40% individual-answer match on Singaporean data. "Plausible but Not Valid: A Psychometric Audit of LLMs as Synthetic Survey Respondents" (arXiv:2608.14606) — title suggests a third independent individual-level critique, contents unchecked. Erika Taday Morocho et al., "Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents" (ACM Web Conf Companion 2026, pp. 320–329) — unread. A citation-tracking search (Semantic Scholar / Google Scholar) should also be re-run once more time has passed. **The cap discharges only when a different author group applies RCG/OSR or an equivalent respondent-level coverage metric, or a peer-reviewed venue accepts this exact paper** — none of the above does that yet; most are convergent-by-different-metric, and Pew is the one worth reading first for whether it gets to individual/subgroup coverage directly.