In the Chameleon test (Harry & Feng 2026), fixed personas put ~91% of behavioral variance between-person where real people show ~1%, and no arm — including the authors' own fix — recovers person-specific signal
Using the "Chameleon" dataset (1,667 Reddit users, three posts each in different subreddits, scored on 26 trait dimensions including the Big Five), Harry & Feng measure the share of behavioral variance that lies between different people versus within one person across situations. In the real data the between-person share is about 1% (mean intraclass correlation 0.013): almost all variation is a single person behaving differently from one context to the next. Two fixed-persona arms invert this — a stated trait alone (Arm A) and a stated trait plus a neutral context mention (Arm B) place the between-person share at 91% and 80%. The authors' own proposed alternative, a "person-in-context" design (Arm C) in which a situation activates one of a set of identity states rather than reading one fixed profile, comes closer on average (ICC 0.285) but still overshoots the real value on four of five traits by 24–65×, matching closely only on extraversion.
The caveat is load-bearing and the authors state it plainly: person-specific signal — the residual correlation the accompanying proof says only a person-in-context design can achieve in principle — is near zero for every arm, including their own fix (mean correlation 0.026, 0.025, 0.004 for Arms A, B, C). The remedy narrows the between/within inversion without yet closing the person-specific gap.
This is the empirical backbone under the paper's formal result, and a variance-decomposition cousin of the silicon-sampling cluster's individual-level findings: reduced response variance in ChatGPT personas vs. ANES (contraction), the respondent coverage gap's off-support/contraction split, and demographic flattening as a distinct axis. It rests on a single unrefereed preprint — see question-corroborate-harry-feng-personas-not-people-single-source.
Source
“The real between-person share is about 1% (mean ICC 0.013), and Arms A and B place it at 91% and 80%.”
claude-opus-4-8 · Promotion from 10-inbox/raw/2026-11-04-outside-arxiv261105292-personas-are-not-people-identity-as.md, 2026-11-21 · raw markdown