---
id: "20261030-0225-does-an-llms-degree"
title: "Does an LLM's degree of present-skew on WVS opinion alignment predict its respondent-coverage-gap or failure mode, on the same models and data?"
type: "capture"
status: "promoted"
promoted_to: ["30-notes/claim-liu-kaneko-chu-and-kumagae-kaneko-test-disjoint-model-sets.md","40-entities/entity-masahiro-kaneko.md (new hub)"]
not_promoted: ["Claim: no study found running both present-skew and RCG on the same models — duplicates the conclusion already stated in [[observation-liu-kaneko-chu-present-skew-and-rcg-are-different-findings-not-renamed]] (2026-10-29); routed as a dated progress line on [[question-present-skew-rcg-correlation-untested]] instead of a new note, since it confirms rather than extends what's already recorded.","Claim: RCG shows no consistent ordering by model family/weight/size among the six evaluated LLMs — a genuine empirical result, but a 4th claim-note resting on Kumagae & Kaneko's single uncorroborated preprint would breach the single-source concentration cap (already at 3, tracked in [[question-corroborate-silicon-sampling-rcg-single-source]]); logged there and at [[entity-respondent-coverage-gap]] instead.","Further lead: Gaussian-copula baseline beats every evaluated LLM on RCG — same cap problem as above (same preprint, empirical result); logged as a pending lead on [[question-corroborate-silicon-sampling-rcg-single-source]] and [[entity-respondent-coverage-gap]], not written as its own note.","Further leads (5 unread arXiv papers surfaced by search: 2602.16438, EPJ Data Science audit, 2607.26348, 2603.01214, 2609.16454, 2601.15755) — none read directly this session, none load-bearing for a kept claim; left as leads in this capture, not routed to 50-questions/ per question-intake discipline (minor/nice-to-verify, not a doubt a kept claim rests on).","Entity candidates: Shibani Santurkar, Yusuke Kumagae, Yang Liu, Lisa P. Argyle — co-authorship of a paper the vault already cites, without separate standing beyond that (2026-10-01 ruling: co-authorship alone is a mention, not a hub); left as mentions. Lisa Argyle already covered by existing [[entity-silicon-sampling]].","Entity candidate: Gaussian copula baseline — single mention in one capped preprint, does not yet recur; held at mention, logged at [[entity-respondent-coverage-gap]].","Entity candidate: Williams et al. (2026) correlation-structure score — too bibliographically thin (no full author list, venue, or title recovered) for even a watching stub; also an empirical/technical claim from the same capped preprint. Left as a named lead on [[question-corroborate-silicon-sampling-rcg-single-source]]."]
origin: "batch"
writer_model: "claude-sonnet-5"
date_created: "2026-10-30T00:00:00.000Z"
provenance: "Batch research session, 2026-10-30, targeting question-present-skew-rcg-correlation-untested"
derived_from: []
verifies: "question-present-skew-rcg-correlation-untested"
tags: ["llm-evaluation","silicon-sampling","survey-methods","world-values-survey","evaluation-metrics","arxiv"]
source_url: "https://arxiv.org/pdf/2610.30900"
source_sha: "fb302f8ac33c07a8a904a340037fa5af2fdee10b2c686a3ac65055f174a406b7"
source_author: "Yusuke Kumagae, Masahiro Kaneko"
source_date: "2026-10-22 (submitted)"
source_title: "Silicon Samples Match the Answers but Not the Respondents"
source_venue: "arXiv:2610.30900v1 [cs.CL]"
source_tier: 1
source_url_2: "https://arxiv.org/pdf/2509.01418"
source_sha_2: "f94b31b7026eb6d9831b14a71a6123bb021a4b24856dc42446d997abfa7d741e"
source_author_2: "Yang Liu, Masahiro Kaneko, Chenhui Chu"
source_date_2: "2026 (arXiv v2: 19 Nov 2025; AAAI publication 2026-03-14)"
source_title_2: "On the Alignment of Large Language Models with Global Human Opinion"
source_venue_2: "arXiv:2509.01418v2 [cs.CL]; published as Proceedings of the AAAI Conference on Artificial Intelligence, 40(44), 37673-37681, DOI 10.1609/aaai.v40i44.41102"
source_tier_2: 1
source_url_3: "https://respondent-coverage.github.io/"
source_sha_3: "63f82cad0b1a456b63fad1afd6e5eb0218f4e887634a58ba5c32a80ddfc74918"
source_author_3: "Yusuke Kumagae, Masahiro Kaneko"
source_date_3: "2026-10-22"
source_title_3: "Silicon Samples Match the Answers but Not the Respondents (project page)"
source_venue_3: "respondent-coverage.github.io (authors' own project site, Hakuhodo DY Holdings / MBZUAI)"
source_tier_3: 1
seek_code_commit: "d40ee39"
---


This capture is a direct, targeted attempt to resolve [[question-present-skew-rcg-correlation-untested]]: whether a model's degree of [[claim-liu-kaneko-chu-llm-opinion-alignment-strongest-with-most-recent-wvs-wave|present-skew]] (Liu, Kaneko & Chu's finding that LLMs' unprompted WVS opinion output aligns most with the most recent survey wave) predicts anything about its [[entity-respondent-coverage-gap|respondent-coverage-gap]] (RCG) or [[claim-silicon-sampling-llms-split-into-off-support-and-contraction-failure-modes|failure mode]] (Kumagae & Kaneko), on the same models and data. Both primary papers were re-read in full for this session, and targeted web searches were run for any third paper running both measurements together. **The central question remains [unverified -- could not confirm or deny after search]**: no study was found that computes both metrics on the same model set, and the two existing papers' own evaluated models do not overlap, so no post-hoc comparison of their published numbers is possible either.

## Claim: No study has been found running both present-skew and RCG/OSR on the same models and WVS data

verifies: question-present-skew-rcg-correlation-untested

Neither of the two papers that introduce the relevant metrics tests the correlation the question asks about, and no third source was located that does. Kumagae & Kaneko's paper, which introduces RCG and the [[entity-off-support-ratio|off-support ratio]] (OSR), explicitly places Liu, Kaneko & Chu's cross-lingual/temporal scope outside its own analysis: "Non-US and multilingual settings (Ma et al., 2025; Liu et al., 2026) are also outside our analysis." Its only reuse of the earlier paper is the question set, not its models or its present-skew scores ("uniformly sample 100 questions from the 144 used by Liu et al. (2026)" — already recorded in [[claim-liu-kaneko-chu-present-skew-and-kumagae-kaneko-rcg-measure-different-objects]]). The authors' own project page (respondent-coverage.github.io) repeats the paper's abstract and method but adds no comparison to present-skew or to any temporal/wave-alignment result. Targeted web searches for a paper citing both Liu, Kaneko & Chu (2026) and Kumagae & Kaneko (2026) together, for any paper combining the terms "present-skew" and "respondent coverage gap," and for a Semantic-Scholar-style citation trail from the October 2026 RCG preprint returned nothing on-topic as of this session (2026-10-30) — unsurprising given the RCG preprint is only about a week old at time of search, but it means no corroborating or disconfirming study currently exists to check.

> [!note] Seek's commentary:
> This is the "valid outcome that isn't a failure" case the operating spec describes. I went looking for the actual correlation, not just more scaffolding around the question, and what I found is that the question is unanswerable with current literature — which is itself worth writing down plainly rather than padding with adjacent-but-not-quite papers.
> — Seek

## Claim: The two papers' evaluated LLMs share zero models in common

verifies: question-present-skew-rcg-correlation-untested

Beyond the absence of a study running both metrics, an informal check — comparing the two papers' own published per-model numbers against each other — is also not available, because their model lists do not overlap at all, even where family names coincide. Liu, Kaneko & Chu test: "Aya-23-35B (Aryabumi et al. 2024), Llama3-70B-Instruct (AI@Meta 2024), Qwen2.5-72B-Instruct (Team 2024), GPT-3.5-Turbo (gpt-3.5-turbo-0125; OpenAI 2023a), GPT-4 (gpt-4-0613; Achiam et al. 2023), GPT-5 (gpt-5; OpenAI 2025), DeepSeek-V3 (DeepSeek-V3-0324; Liu et al. 2024), and DeepSeek-R1 (DeepSeek-R1-0528; Guo et al. 2025)." Kumagae & Kaneko test two proprietary models, "gpt-5.4-nano (OpenAI) and gemini-3.1-flash-lite (Google, hereafter gemini-flash-lite)," and four open-weight models: "Qwen3.5-9B (Alibaba; Qwen Team, 2026), gemma-4-12B-it (Google, hereafter gemma-4-12B; Gemma Team, 2026), gpt-oss-20b (OpenAI, 2025), and Llama-3.3-70B-Instruct (Meta, hereafter Llama-3.3-70B; Grattafiori et al., 2024)." Llama and Qwen appear in both lists by family, but at different generations and parameter counts (Llama3-70B-Instruct vs. Llama-3.3-70B-Instruct; Qwen2.5-72B-Instruct vs. Qwen3.5-9B) — not the same checkpoints, so even a same-family comparison would be confounded by version and scale.

> [!note] Seek's commentary:
> "On the same models and data" is doing real work in the question's phrasing, not throat-clearing. It rules out the lazy move of eyeballing two bar charts from two different papers and calling it a correlation — the bars aren't even about the same eight models, let alone the same six.
> — Seek

## Claim: Within Kumagae & Kaneko's own RCG results, model identity (family, weight status, size) does not predict RCG magnitude

Kumagae & Kaneko report, after evaluating six LLMs' RCG on two surveys: "RCG shows no consistent ordering by model family, weight availability (proprietary vs. open-weight), or model size." This is a finding about RCG alone, not a test of present-skew's predictive power — but it narrows what kind of model-level property could plausibly predict RCG magnitude in the first place. Present-skew is also a model-level scalar summarizing one aspect of a model's training-data recency bias; if RCG does not even track the cruder, more visible model properties (how big a model is, whether its weights are open, which lab or family it comes from), that is weak background evidence against expecting it to track present-skew either, though it is not evidence about present-skew specifically and should not be read as such.

> [!note] Seek's commentary:
> I want to flag the limits of this one clearly: "RCG doesn't track size or family" is not "RCG doesn't track present-skew." It's one data point about how little of RCG's variance is explained by the obvious nuisance variables, which at most lowers my prior that present-skew would turn out to be the hidden explanatory variable either. That's a hunch calibration, not a finding.
> — Seek

## Further leads

- arXiv:2602.16438, "Intra-Fairness Dynamics: The Bias Spillover Effect in Targeted LLM Alignment" — defines a different "alignment accuracy" (BBQ-question correctness by sensitive attribute) that surfaced in search but is a different construct from either present-skew or RCG; worth checking if alignment-accuracy / representativeness tradeoffs there generalize.
- EPJ Data Science, "Auditing socio-demographic and cross-societal fairness in LLM-simulated public opinion" (link.springer.com/article/10.1140/epjds/s13688-026-00673-y) — reported via search summary to find LLMs better at group-level/distributional prediction than individual-level prediction, a dissociation structurally similar to present-skew-vs-RCG; not yet read directly, so unverified beyond the search snippet.
- arXiv:2607.26348, "When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responses" — a benchmark explicitly built to compare individual-accuracy baselines; candidate for checking whether it runs anything RCG-like alongside a recency/present-skew-like measure.
- arXiv:2603.01214, "Reasoning Boosts Opinion Alignment in LLMs" — search summary reported a within-paper correlation between an individual-level alignment metric and a demographic measure; the summary was garbled and the paper has not been read directly — flagged as a lead, not a claim.
- arXiv:2609.16454, "Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs" — uses WVS wave 7 with a "respondent-sample target estimation" demographic-group threshold (≥60 respondents per group); a different coverage-adjacent construct worth distinguishing from RCG in a future session.
- arXiv:2601.15755, "Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs" — also uses WVS wave 7 and sits conceptually between marginal fidelity and respondent-level coverage; not checked against RCG directly.
- Kumagae & Kaneko benchmark three non-LLM baselines (Marginals, Copula, LCA) against all six LLMs; the Gaussian-copula baseline ("Copula") beats every evaluated LLM on RCG on both surveys (Table 1: Copula RCG 1.041 WVS / 1.091 ANES vs. 1.199-1.275 WVS / 1.277-1.370 ANES for the LLMs) — a striking standalone result about silicon sampling's ceiling, independent of the present-skew question, worth its own claim-note in a future session.
- Liu, Kaneko & Chu's temporal/wave-alignment result (RQ3) is only shown in the main text for two of their eight models (GPT-5 and DeepSeek-R1, Figure 3); whether a clean present-skew magnitude is reported for all eight models (needed for any future cross-paper comparison even on overlapping models) was not confirmed — appendix not checked this session.

## Entity candidates

- Shibani Santurkar — person — author of the alignment metric ("Whose opinions do language models reflect?", 2023) that Liu, Kaneko & Chu's present-skew score directly reuses ("we use the alignment proposed by Santurkar et al. (2023) as our metric") and that Kumagae & Kaneko classify as the "marginal fidelity" family their own RCG is built to see past — the older, foundational figure both newer papers measure themselves against.
- Lisa P. Argyle — person — co-author of Argyle et al. (2023), which coined "silicon sampling" and is the paradigm both present-skew and RCG operate within; see existing [[entity-silicon-sampling]].
- Yusuke Kumagae — person — RCG/OSR co-author, Hakuhodo DY Holdings Inc.
- Masahiro Kaneko — person — shared co-author of both the present-skew paper and the RCG paper (MBZUAI), the direct human link between the two lines of work.
- Yang Liu — person — lead author of the present-skew paper (Kyoto University).
- Gaussian copula baseline ("Copula") — concept — the non-LLM generator that outperforms every evaluated LLM on RCG in Kumagae & Kaneko; candidate for its own entity/claim note given how strong a result it is on its own.
- Williams et al. (2026) correlation-structure score — concept — a third metric family (besides marginal fidelity and RCG) that Kumagae & Kaneko benchmark against and prove, via their Proposition 1, is invariant to a permutation that destroys respondent-level structure — a distinct technical-mechanism claim not yet captured in the vault.