Bisbee et al. (2024) found LLM-simulated survey respondents show systematically lower response variance than real respondents, a peer-reviewed precursor to the respondent-coverage-gap paper's 'contraction' failure mode

llm-evaluationsilicon-samplingsurvey-methodssynthetic-datacorroboration

Bisbee, Clinton, Dorff, Kenkel & Larson (Political Analysis 32(4), 2024) prompted ChatGPT with demographic personas to generate feeling-thermometer scores for 11 sociopolitical groups and compared the synthetic output to the 2016–2020 American National Election Study (ANES). Average scores tracked the ANES averages closely, but the synthetic responses showed systematically lower variance than the real survey, and regression coefficients fit on the generated data often differed significantly from ANES-based estimates. The authors conclude that aggregate agreement does not make the output safe for inference.

This anticipates, by roughly two years and a different route, the "contraction" failure mode later named by Kumagae & Kaneko's respondent coverage gap (RCG) paper — generated respondents clustering onto a narrower band of profiles than real respondents occupy (see claim-silicon-sampling-llms-split-into-off-support-and-contraction-failure-modes). The two works are independent: a different author group, a different peer-reviewed venue, and a different metric (variance and regression comparison, not a nearest-respondent distance ratio). It is therefore convergent prior evidence that silicon-sampled populations narrow relative to real ones — not a replication of RCG's specific numbers, and not independent confirmation of the RCG metric itself. The distinction is load-bearing for the open question of whether the RCG preprint is corroborated: a shared conclusion reached by a different measurement is support for the underlying phenomenon, not a second application of the same test.