evaluation-metrics

10 notes carry this tag — 6 claims, 1 observation, 1 question, 2 captures. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

A no-respondent population-prevalence query can outperform committed LLM-persona panels on marginal-fidelity benchmarks (Doudkin 2026), so aggregate fidelity cannot establish individual respondent simulation

claim seedling Tier 1 2026-10-31 · llm-evaluation, silicon-sampling, survey-methods, evaluation-metrics, corroboration
Zero-shot LLMs predict a held-out respondent's own survey answers at only 52% accuracy (Ku et al. 2026), an individual-level failure invisible to distributional comparison

claim seedling Tier 1 2026-10-31 · llm-evaluation, silicon-sampling, survey-methods, evaluation-metrics, corroboration
Does an LLM's degree of present-skew on WVS opinion alignment predict its respondent-coverage-gap or failure mode, on the same models and data?

capture promoted Tier 1 2026-10-30 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
Liu, Kaneko & Chu's present-skew paper and Kumagae & Kaneko's RCG paper evaluate completely disjoint sets of LLMs

claim seedling Tier 1 2026-10-30 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
Present-skew and the respondent-coverage-gap are different measurement objects, linked by shared data and a shared co-author rather than a shared mechanism

capture promoted 2026-10-29 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, cultural-awareness
Liu, Kaneko & Chu's present-skew and Kumagae & Kaneko's respondent-coverage-gap measure different mathematical objects, built on a confirmed shared dataset and author

claim seedling Tier 1 2026-10-29 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
Present-skew and the respondent-coverage-gap are two different findings about different objects, not one failure mode renamed — and whether they correlate is untested

observation seedling Tier 1 2026-10-29 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
Does an LLM's degree of present-skew on WVS opinion alignment predict its respondent-coverage-gap or failure mode, on the same models and data?

question open 2026-10-29 · llm-evaluation, silicon-sampling, survey-methods, world-values-survey, evaluation-metrics
Per-question correlation-structure metrics for silicon-sampled surveys are structurally blind to respondent-level coverage failure

claim seedling Tier 1 2026-10-25 · llm-evaluation, silicon-sampling, survey-methods, evaluation-metrics, arxiv
Silicon-sampling LLMs fail in two opposite ways: off-support drift away from the human population versus contraction onto typical profiles

claim seedling Tier 1 2026-10-25 · llm-evaluation, silicon-sampling, survey-methods, evaluation-metrics, arxiv