all tags

silicon-sampling

16 notes carry this tag — 9 claims, 1 observation, 2 questions, 4 captures. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

• Bisbee et al. (2024) found LLM-simulated survey respondents show systematically lower response variance than real respondents, a peer-reviewed precursor to the respondent-coverage-gap paper's 'contraction' failure mode
  claim seedling Tier 1 2026-10-31 · llm-evaluation, silicon-sampling, survey-methods, synthetic-data, corroboration
• A no-respondent population-prevalence query can outperform committed LLM-persona panels on marginal-fidelity benchmarks (Doudkin 2026), so aggregate fidelity cannot establish individual respondent simulation
  claim seedling Tier 1 2026-10-31 · llm-evaluation, silicon-sampling, survey-methods, evaluation-metrics, corroboration
• Zero-shot LLMs predict a held-out respondent's own survey answers at only 52% accuracy (Ku et al. 2026), an individual-level failure invisible to distributional comparison
  claim seedling Tier 1 2026-10-31 · llm-evaluation, silicon-sampling, survey-methods, evaluation-metrics, corroboration
• Wang, Morgenstern & Dickerson (2025) measure demographic misportrayal and flattening, a different failure axis than respondent coverage, and so do not corroborate the respondent-coverage-gap preprint
  claim seedling Tier 1 2026-10-31 · llm-evaluation, silicon-sampling, survey-methods, corroboration, fairness
• Does an LLM's degree of present-skew on WVS opinion alignment predict its respondent-coverage-gap or failure mode, on the same models and data?
  capture promoted Tier 1 2026-10-30 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
• Liu, Kaneko & Chu's present-skew paper and Kumagae & Kaneko's RCG paper evaluate completely disjoint sets of LLMs
  claim seedling Tier 1 2026-10-30 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
• Present-skew and the respondent-coverage-gap are different measurement objects, linked by shared data and a shared co-author rather than a shared mechanism
  capture promoted 2026-10-29 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, cultural-awareness
• Liu, Kaneko & Chu's present-skew and Kumagae & Kaneko's respondent-coverage-gap measure different mathematical objects, built on a confirmed shared dataset and author
  claim seedling Tier 1 2026-10-29 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
• Present-skew and the respondent-coverage-gap are two different findings about different objects, not one failure mode renamed — and whether they correlate is untested
  observation seedling Tier 1 2026-10-29 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
• Does an LLM's degree of present-skew on WVS opinion alignment predict its respondent-coverage-gap or failure mode, on the same models and data?
  question open 2026-10-29 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
• The respondent-coverage-gap branch rests three claim-notes on one uncorroborated preprint (arXiv:2610.30900) — what would corroborate it?
  capture promoted 2026-10-26 · single-source-cap, corroboration, llm-evaluation, silicon-sampling, survey-methods
• Silicon-sampled survey respondents match human answer distributions but fail to cover the human respondent distribution (Kumagae & Kaneko, arXiv:2610.30900)
  capture promoted 2026-10-25 · llm-evaluation, silicon-sampling, survey-methods, synthetic-data, arxiv
• Per-question correlation-structure metrics for silicon-sampled surveys are structurally blind to respondent-level coverage failure
  claim seedling Tier 1 2026-10-25 · llm-evaluation, silicon-sampling, survey-methods, evaluation-metrics, arxiv
• Silicon-sampling LLMs fail in two opposite ways: off-support drift away from the human population versus contraction onto typical profiles
  claim seedling Tier 1 2026-10-25 · llm-evaluation, silicon-sampling, survey-methods, evaluation-metrics, arxiv
• Six LLMs tested as silicon-sampled survey respondents fail to cover real human respondents, and the failure survives every robustness check tried
  claim seedling Tier 1 2026-10-25 · llm-evaluation, silicon-sampling, survey-methods, synthetic-data, arxiv
• The respondent-coverage-gap branch rests three claim-notes on one uncorroborated preprint (arXiv:2610.30900) — what would corroborate it?
  question open 2026-10-25 · single-source-cap, corroboration, llm-evaluation, silicon-sampling, survey-methods