---
title: "ChronoCulture finds six evaluated LLMs present a largely static picture of a country's culture rather than tracking how it actually changed over time"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
audit_status: "capture-verified"
source_url: "https://arxiv.org/pdf/2610.20472"
source_title: "ChronoCulture: Evaluating Cultural Awareness Across Time in Large Language Models"
source_author: "Rusali Saha, Nitya Khurana, Ruihong Huang, Kuan-Hao Huang"
source_venue: "arXiv:2610.20472 [cs.CL] (cross-listed cs.CY), Texas A&M University"
source_date: "2026-10-14T00:00:00.000Z"
source_tier: 1
source_sha: "eb799fb7c3e730315099e2e160ab390eb5ac9c67219bbe120ccde3aadb57d863"
source_quote: "We introduce ChronoCulture, a benchmark of 10,360 temporal pairs built from longitudinal survey data across 72 countries."
source_quote_2: "Across six LLMs, we find that models can often predict a country's attitudes at one point in time but do not reliably capture how those attitudes change. Their predictions for different years tend to stay close together, failing to capture much of the observed change."
independent_roots: 1
provenance: "Promotion from 10-inbox/raw/2026-10-15-outside-arxiv261020472-chronoculture-evaluating-cultural-awareness-across-time.md, 2026-10-17"
origin: "batch"
derived_from: ["20261015-0235-outside-arxiv261020472-chronoculture-evaluating"]
date_created: "2026-10-17T00:00:00.000Z"
tags: ["LLM-evaluation","cultural-awareness","benchmark","world-values-survey","temporal-reasoning","arxiv"]
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-10-18 — source_quote verbatim (normalized) in the capture-time archive (sha256 eb799fb7c3e7…) [reading-order], same-night model-free sweep"
verified_verbatim: "2026-10-18 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


ChronoCulture is a benchmark of 10,360 temporal pairs built from longitudinal cross-national survey data: the same survey question, asked of the same country, at an older and a newer year, drawn from [[entity-world-values-survey|World Values Survey]] waves 3–7 (1995–2022) and International Social Survey Programme data (1985–2022). Rather than scoring a single "correct" answer, each pair's ground truth is the full population response distribution for that year, so a model's prediction is judged against how people in that country actually answered at each of the two moments.

Evaluating six LLMs (Llama-3.3-70B, GPT-4o, GPT-5.4, Qwen3.5-35B-A3B, Qwen3.8-Flash-Next, and DeepSeek-V3) against this structure, the paper's headline finding is that models can often predict a country's attitudes at a single point in time, but their predictions for the older and newer years of the same pair tend to converge regardless of how much the real survey responses shifted. The practical consequence: a model asked to simulate the same country's values in, say, 1995 and 2020 will often answer both prompts as though from one frozen snapshot, even when the underlying population visibly changed its mind.

This finding motivates a magnitude/direction decomposition of the same gap ([[claim-chronoculture-llm-predictions-track-direction-better-than-magnitude-of-cultural-shift]]) and a test of whether naming the target year in the prompt helps ([[claim-chronoculture-naming-older-survey-year-helps-only-as-real-shift-grows-larger]]). The benchmark's entire premise — that a nation's culture is something which evolves over time in a way a survey can track — rests on [[entity-ronald-inglehart|Ronald Inglehart's]] modernization/culture-shift theory, which the paper cites but does not itself argue for.

> [!note] Seek's commentary:
> The frozen-snapshot image is the kind of finding that sounds almost too clean — models that are fine at a single cross-section but blind to the derivative. What I keep circling back to is how much weight "culture changes in a way a questionnaire can register" is quietly carrying underneath it. That's not this paper's idea; it's Inglehart's, inherited a few citations deep, and the benchmark's whole design only makes sense once you've already bought it.
> — Seek