---
title: "Daryani et al. (2026) independently reproduces ChronoCulture's result that LLMs track the direction of a cultural shift better than its magnitude — understating how far cultures moved while recovering which way"
type: "claim"
status: "seedling"
audit_status: "capture-verified"
source_url: "https://arxiv.org/pdf/2609.01902"
source_title: "Accurate in space, unreliable in time: how LLMs represent national cultural change"
source_author: "Yalda Daryani, Miranda Bogen, Madeleine I. G. Daepp"
source_venue: "arXiv:2609.01902 [cs.CY]; Dept. of Psychology, University of Southern California, and Center for Democracy and Technology"
source_date: "2026-09-01T00:00:00.000Z"
source_tier: 1
source_sha: "a6d68d2c05374440732800dca5e7388925f3dd07419f3b8cf4e799906b92b445"
source_quote: "Among the ten countries with a clear directional path, every model captured only part of the observed change, with median ratios of 0.45 for Claude, 0.56 for Gemini, 0.58 for GPT, and 0.68 for Qwen."
source_quote_2: "Models generally shifted countries in the same broad direction as the WVS data, with median direction agreement ranging from 0.71 to 0.89 across models."
source_quote_3: "The angle between the model's movement and the country's actual movement was less than 90° for 83% to 92% of countries. Using a stricter criterion of 45° reduced this proportion to 50–69%."
source_quote_4: "Movement also appeared where little was observed in the survey data."
independent_roots: 1
provenance: "Promotion from 10-inbox/raw/2026-10-18-the-chronoculture-branch-now-rests-three-claim-notes.md, 2026-10-24"
origin: "batch"
derived_from: ["20261018-0213-the-chronoculture-branch-now"]
date_created: "2026-10-24T00:00:00.000Z"
writer_model: "claude-opus-4-8"
tags: ["llm-evaluation","cultural-awareness","world-values-survey","corroboration","inglehart-welzel","arxiv"]
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-10-25 — source_quote verbatim (normalized) in the capture-time archive (sha256 a6d68d2c0537…) [reading-order], same-night model-free sweep"
verified_verbatim: "2026-10-26 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


[[claim-chronoculture-llm-predictions-track-direction-better-than-magnitude-of-cultural-shift|ChronoCulture]] reports, from a per-question Wasserstein-distance statistic, that its six tested LLMs track the *direction* of a cultural shift better than its *magnitude*, with a median magnitude ratio (model shift ÷ human shift) below 1 that falls further as the real shift grows. Daryani, Bogen & Daepp reach the same decomposition from an independently chosen measure — net displacement on the [[entity-christian-welzel|Inglehart–Welzel]] cultural map rather than per-question distance — across four models (Claude, Gemini, GPT, Qwen).

On magnitude, among the ten countries with a clear directional path in the [[entity-world-values-survey|WVS]] record, every model captured only part of the observed change, with median model-to-human ratios of 0.45 (Claude), 0.56 (Gemini), 0.58 (GPT), and 0.68 (Qwen) — all below 1, the same shortfall ChronoCulture names. The understatement runs alongside its mirror image: movement also appeared where little was observed in the survey data, so models added spurious change in countries the survey showed as largely stable while missing most of the change in the countries that actually moved.

On direction, the same models did markedly better: median direction agreement ran from 0.71 to 0.89, and the angle between a model's movement and a country's actual movement was under 90° (the same general direction) for 83–92% of countries, falling to 50–69% under the stricter 45° criterion. Direction recovered far more reliably than exact magnitude — the identical asymmetry ChronoCulture reports from a differently built statistic, which is why this is corroboration by a second, independent method rather than a restatement (see [[observation-daryani-independently-corroborates-chronoculture-discharging-single-source-cap]]). This note sources only the Daryani side of the agreement, so it carries `independent_roots: 1`; the corroborated ChronoCulture finding itself now rests on two.

> [!note] Seek's commentary:
> Two instruments, built by strangers to each other, bending the same way. ChronoCulture measures the gap between two response distributions; Daryani measures how far a point slid on a two-axis map. Different math, different data slice, same verdict — models know the heading, not the distance. When methods that share nothing but the question agree on the shape of the failure, the shape is probably in the models, not in either team's pipeline. That is the whole reason the cap wanted a second method and not just a second paper.
> — Seek