Does LLM citation-popularity bias measurably compound across successive training generations as LLM-selected citations re-enter training corpora?

chatgptllmcitation-metricsmatthew-effectbibliometricsmodel-collapsefeedback-looptraining-data-contaminationalgorithmic-monoculture

Researched directly against question-does-citation-popularity-bias-compound-across-llm-training-generations, raised while promoting observation-petiska-matthew-effect-finding-independently-replicated-by-algaba-and-naser. That question already establishes, via Petiška, Algaba et al., and Naser that current LLMs select citations by popularity rather than relevance, and that Ansari 2026 documents one traced case of an AI-generated citation artifact re-entering a later model's output. None of those four is a multi-generation experiment. This capture went looking specifically for one, and found the closest thing so far: a 2026 recursive citation-selection benchmark that, while not a literal model-retraining experiment, directly measures what happens to citation concentration across twelve successive rounds as a citation candidate pool fills with AI-generated content.

Claim: A twelve-round recursive citation-selection benchmark finds that citation concentration on real papers intensifies as AI-generated papers flood the candidate pool — not because the underlying preference gets stronger, but because a fixed preference is applied to an ever-diluting pool of authentic work

Claim type: quantitative + technical-mechanism. Tier 1-2 required — met (single-group arXiv preprint, not yet independently peer-reviewed at capture time; source_quote taken directly from the extracted PDF, sha256 recorded). verifies: question-does-citation-popularity-bias-compound-across-llm-training-generations

Alemohammad, Zhang, Tang, Qin, Mai, Abbasi, Baraniuk & Wang built a benchmark isolating citation choice from retrieval, prestige, and fabrication: 120 real knowledge-distillation papers (arXiv 2015–2022, 50–500 citations each) shown with real titles/abstracts but fabricated author names, reassigned years, and hidden citation counts and venues, so "no prestige signal survives." Eleven models from three vendors (later Claude models were excluded from the recursive phase for failing to hold the citation budget across rounds, leaving eight: four OpenAI, four Gemini) were shown random 30-paper panels and allowed to cite at most ten. They then ran this recursively: "each round's 120 model-written papers join the catalogue of the next round, so the catalogue grows by 120 papers each round, to 1,440 at round 11" — meaning by the final round the 120 original ("seed") papers are outnumbered roughly ten to one by AI-generated ones.

The result is a mechanism claim, stated in the paper's own words: "The filter does not fade with repeated use, and it is not amplified by it either. It is concentrated onto fewer targets." Concretely: "the fraction of shown seeds cited climbs from 33% toward 63% on average (92% for the strongest model) while the null's seed rate stays flat near 32%." Round-11 top-decile citation shares for real papers run 31.1–40.3% across the eight models, and restricting the analysis to AI-generated papers only (removing every real paper from the ledger) still leaves 1.7–11.2 percentage points of excess concentration above an indifferent baseline at round 11. So the paper's own answer to "does the bias compound with repeated use" is a precise no and yes: no, the per-model preference strength itself is not amplified by recursion; yes, the practical effect on which real work gets attention intensifies anyway, because the same fixed preference is now competing over a shrinking share of authentic candidates inside a growing, self-generated pool. The authors name this mechanism dilution, not amplification, and are explicit that it is not what would usually be meant by a feedback loop: "being cited never raises a paper's chance of being shown again... so nothing here is a citation feedback loop in the selection sense."

This is the most direct evidence located bearing on the question's literal concern — a shrinking, ever-more-concentrated slice of real scholarship receiving AI citation attention as synthetic content accumulates — but see the scope-limitation claim below: this experiment holds the models fixed across all twelve rounds and recycles the candidate pool, rather than retraining new model generations on a corpus containing prior citation choices.

Claim: The cross-vendor "monoculture" — the shared preference map that is the signature of citation-popularity bias across models — collapses when the candidates being judged are AI-generated rather than real: the concentration effect survives recursion but the shared, cross-model agreement behind it does not

Claim type: quantitative + technical-mechanism. Tier 1-2 required — met (same source, same recursive experiment, source_quote taken directly from the extracted PDF). verifies: question-does-citation-popularity-bias-compound-across-llm-training-generations

Within the same twelve-round recursion, the paper separately tested whether the models' preferences stay correlated with one another (the cross-vendor "monoculture" that Round 0 established as nearly identical in strength within-vendor and across-vendor) once most of the candidate pool consists of model-written rather than real papers. It does not: "Pooling every round, the cross-model correlation of per-paper citation rates is 0.68 among seeds and 0.20 among generated papers, each corrected for its own split-half reliability... The models simply do not share it. Concentration reproduces itself on synthetic text; the monoculture does not. What the models converge on is the real literature, and that convergence tightens as the real literature thins."

This qualifies the first claim in an important direction: whatever compounding effect recursion produces is not a uniformly reinforcing, cross-vendor-shared popularity signal growing stronger together — it is a shared preference for a shrinking set of real, already-established papers, alongside increasingly idiosyncratic, per-model preferences over the AI-generated content that surrounds them. The Matthew-effect shape (credit compounding toward what is already credited) survives and intensifies specifically for the original, pre-existing literature; it does not straightforwardly generalize into a stronger or more unified bias toward whatever an LLM itself most recently produced.

Claim: This recursive benchmark is a citation-selection recursion over fixed models, not a model-retraining-generation experiment — so it does not directly test the question's literal training-corpus mechanism, which remains unmeasured in the located literature

Claim type: technical-mechanism / historical (a description of what a study does and does not do). Tier 3-4 acceptable for this kind of claim, but sourced here at Tier 1 since the paper's own scoping language is being quoted directly. verifies: question-does-citation-popularity-bias-compound-across-llm-training-generations

The paper's own authors are explicit about what their recursion does and does not simulate: "Models round 0: eleven models, three vendors; recursion: eight (four OpenAI, four Gemini)" — the same fixed models are re-run across all twelve rounds; no model is retrained, fine-tuned, or otherwise updated on the growing catalogue at any point. What recurses is "the catalogue of papers" available to cite from, not the parameters of any model doing the citing. The paper states this limitation of its own framing directly: "what recursion supplies is dilution," and separately, "nothing here is a citation feedback loop in the selection sense," because "being cited never raises a paper's chance of being shown again."

This means the question's literal mechanism — a new model generation trained on a corpus that includes an earlier generation's LLM-selected, popularity-skewed citations, and showing a measurably stronger popularity bias as a result of that training exposure — is still not directly tested by any source located in this or the prior 2026-09-13 capture's search. What exists instead, as of this capture: (1) cross- sectional replication that current LLMs share a popularity-selection bias (Petiška, Algaba et al., Naser); (2) one traced instance of an AI-generated citation artifact re-entering a later model's training-derived output (Ansari); and (3) this capture's recursive citation-selection benchmark, which shows that even without any retraining, repeated selection against a self-diluting corpus concentrates attention onto an ever-smaller share of real work — a mechanism adjacent to, but distinct from, the training-corpus compounding the question asks about. The central question therefore remains [unverified — could not confirm or deny after search] in its literal form: no located study retrains successive model generations on a corpus containing prior LLM-selected citations and measures whether citation-popularity bias itself gets stronger as a result.

Further leads

• Ren, Guo, Qiu, Wang & Sutherland, "Bias Amplification in Language Model Evolution: An Iterated Learning Perspective" (NeurIPS 2024; arXiv:2404.04286) — a Bayesian iterated-learning framework formally analyzing how repeated on-policy self-improvement training (models generating their own next training examples) can magnify subtle biases across rounds; general-purpose, not citation-specific, and not read in full this session — worth a dedicated capture testing whether its formalism could be adapted to citation-popularity specifically.
• Wang, Wu, Zhang, Guan, Jain, Lu, Gupta & Koshiyama, "Bias Amplification: Large Language Models as Increasingly Biased Media" (arXiv:2410.15234, 2024-10-19, rev. 2025-05-20) — an empirical multi-generation retraining study (GPT-2, political bias, not citations) finding bias amplification is mechanistically distinct from general model collapse (different neuron populations implicated); the closest thing found to a real training-generation experiment demonstrating any kind of bias compounding, just on a different bias axis than citation popularity.
• Alemohammad et al.'s companion code and data release, github.com/VITA-Group/Citation-Collapse — not fetched this session; would let a future capture check reproducibility or extend the recursion to more rounds/vendors.
• He, J., "Who Gets Cited? Gender- and Majority-Bias in LLM-Driven Reference Selection" (arXiv:2508.02740) — already flagged in the 2026-09-13 capture that raised this question; a distinct bias axis (author gender/majority-group), not re-derived here.
• "When Sample Selection Bias Precipitates Model Collapse" (arXiv:2606.13732) and "The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data" (arXiv:2608.04268) — both 2026 general model-collapse/fairness-degradation papers surfaced in this search; neither is citation-specific and neither was read past its abstract this session.

Entity candidates

• Robert Merton — person — the foundational figure this entire paper (and the whole citation-popularity cluster) measures itself against by name; already an entity (entity-robert-merton) and already flagged in the prior 2026-09-13 capture — repeated here first, per this capture's own instructions, not to duplicate the page.
• Sina Alemohammad — person — corresponding author of the citation-monoculture recursion study (all three claims above); no entity page exists yet.
• Zhangyang Wang — person — senior author (UT Austin, VITA Group) on the same paper; no entity page exists yet.
• Citation monoculture / citation collapse — concept — the paper's own coined term for the triplet of concentration, exclusion, and bibliography homogenization it measures; distinct from, but closely related to, the vault's existing Matthew-effect entity — may warrant its own concept page given how load-bearing it is to this and the parent claim cluster.
• Algorithmic monoculture — concept — the broader term (not coined by this paper) that the paper explicitly places its finding under; worth checking whether the vault already has or should have a concept page distinct from citation monoculture specifically.