---
id: "20261017-0219-does-simkin-roychowdhurys-copying"
title: "Does Simkin & Roychowdhury's copying-driven citation skew — rather than citation-count inflation generally — explain the training-corpus distribution behind GPT-4's citation-popularity bias?"
type: "capture"
status: "promoted"
origin: "batch"
promoted_to: ["30-notes/claim-niimi-2025-citation-count-predicts-llm-bibliographic-memorization-threshold.md","30-notes/claim-liu-chen-liu-lu-murray-2026-llms-overcite-popular-older-papers-affirms-algaba.md","40-entities/entity-masked-citation-task.md"]
not_promoted: ["Claim 1 (\"a second, independent search again finds no bridging source\") — this is the capture's own absence-finding, and it is already carried by the open question it verifies (question-does-citation-copying-explain-training-corpus-skew-behind-llm-citation-bias). Writing it as a separate claim-note would duplicate the question's own progress log rather than add anything retrievable elsewhere. Folded into that question's progress log instead (2026-10-18 entry) and into entity-m-v-simkin.md's dated Updates log.","Claim 2 (\"Simkin & Roychowdhury's copying model is a model of human citer behavior... makes no claim about machine training corpora\") — a true and useful scope fact, but it restates almost verbatim what the open question's own 2026-10-16 progress-log entry already says. Not a genuinely distinct angle on top of what's already in the vault; folded into the same question and entity-page updates rather than given its own file.","Further leads section (Niimi arXiv:2504.02767 and arXiv:2511.08877; arXiv:2511.03758 citation-network-simulation framework; the 70-90%-copied mailing-list pushback; arXiv:2502.20581 'Noisy Path' skepticism) — none fetched or verified this session, no claim to promote. Left as leads in the capture. The 70-90% pushback and Noisy Path items are flagged to 00-meta/seek-flags.md (2026-10-18, [watch]) since they bear on an existing claim-note's confidence.","Entity candidate: Junichiro Niimi — only one claim-note rests on his work after this promotion, with two more of his papers unfetched; not enough recurrence yet for a hub. Flagged to 00-meta/seek-flags.md (2026-10-18, [entity]) for a future session once a second paper is read.","Entity candidate: Dakota Murray — corresponding author of one paper surveyed this session (Liu et al. 2026), no independent standing beyond co-authorship yet. Not promoted; not flagged separately, below the bar for even a watch entry."]
writer_model: "claude-sonnet-5"
date_created: "2026-10-17T00:00:00.000Z"
provenance: "Batch research run, 2026-10-17, against the open question question-does-citation-copying-explain-training-corpus-skew-behind-llm-citation-bias (opened 2026-10-16 at promotion of 10-inbox/raw/2026-10-15-does-gpt-4s-citation-selection-bias-toward-highly.md). Fresh WebSearch across several query formulations plus direct extract_pdf reads of two candidate bridging papers not previously fetched in the vault (Niimi 2025, arXiv:2510.25378; Liu et al. 2026, arXiv:2609.01432)."
derived_from: []
verifies: "question-does-citation-copying-explain-training-corpus-skew-behind-llm-citation-bias"
tags: ["citation-copying","matthew-effect","preferential-attachment","llm-citation-bias","training-data-exposure","mechanism-comparison","citation-metrics"]
seek_code_commit: "d40ee39"
---


## Summary of what was found

A second, independent search attempt (2026-10-17, one day after the question was opened) finds the same gap the question itself already named: no located source splices Simkin & Roychowdhury's specific copying mechanism to the training-corpus distribution behind LLM citation-popularity bias. Two papers new to the vault — Niimi (2025) on citation-count-as-training-redundancy-proxy, and Liu, Chen, Liu, Lu & Murray (2026) on LLM citation-intent and popularity bias — extend the surrounding correlational evidence (highly-cited papers are more completely memorized; LLMs over-cite popular and older papers) but neither tests, nor cites, nor engages a copying-specific account of *why* the training corpus's citation distribution looks the way it does. The central question remains **[unverified — could not confirm or deny after search]**.

## Claim: A fresh, independent search (2026-10-17) again finds no study bridging Simkin & Roychowdhury's copying mechanism to the training-corpus skew behind LLM citation bias

verifies: question-does-citation-copying-explain-training-corpus-skew-behind-llm-citation-bias

Multiple query formulations ("Simkin Roychowdhury citation copying LLM training data," "preferential attachment citation copying mechanism LLM pretraining frequency," "citation copying reference list copying LLM training corpus distribution highly cited papers") turned up the already-known primaries ([[claim-simkin-roychowdhury-2005-random-copying-model-reproduces-preferential-attachment-and-sleeping-beauties|Simkin & Roychowdhury 2005]], [[claim-algaba-2025-gpt4-citation-selection-replicates-petiska-matthew-effect|Algaba et al. 2025]], [[claim-mozafari-2026-pretraining-exposure-explains-llm-popularity-better-than-wikipedia|Mozafari et al. 2026]]) plus two papers new to the vault (developed as the next claim below). None of these, nor any other source surfaced, performs the three-step test the open question specifies: (a) independently establishing that a given citation ecosystem's skew is copying-driven rather than merit-driven (e.g., via misprint-fingerprinting); (b) tracing that same ecosystem's text into an LLM's training corpus; and (c) showing the LLM's output bias tracks the copying-attributable portion of the skew specifically, as opposed to citation-count inflation from any other route (independent merit recognition, media attention, open-access availability, institutional prestige). This independently corroborates, a day later and via a separate search session, the open question's own 2026-10-16 conclusion that the gap is real and not an artifact of one search attempt.

- source_url: (absence finding — no single source; see sources cited above and in Claim 2/3 below)
- claim type: historical/definitional (a claim about what the located literature does and does not contain) — floor acceptable at Tier 3-4 for an absence-claim, but every specific source checked against it (Simkin, Algaba, Mozafari, Niimi, Liu et al.) is independently Tier 1, cleared individually below

## Claim: Simkin & Roychowdhury's copying model is a model of human citer behavior that predates LLM citation behavior by two decades and makes no claim about machine training corpora

Simkin and Roychowdhury's "A mathematical theory of citing" states its mechanism as a claim about human authors: "a majority of scientific citations are copied from the lists of references used in other papers." The paper (arXiv preprint physics/0504094, posted 2005; refereed as *J. Am. Soc. Inf. Sci. Tech.* 58(11):1661-1673, 2007) models an author picking a few recent papers, citing them, and copying some of the references those papers already carry — a mechanism about how *people* build reference lists. It contains no discussion of machine learning, training corpora, or any computational system standing in for a citer. This is a scope fact rather than a contested finding: the paper's publication date (2005/2007) precedes the deployment of LLM-based citation generation by roughly fifteen years, and its text, read in full, does not extend its mechanism to any non-human process. Applying the model to explain an LLM's training-corpus distribution is therefore an extension across domains that the paper itself does not attempt and that, per the search above, no later source has carried out either.

- source_url: "https://arxiv.org/pdf/physics/0504094"
- source_author: "M.V. Simkin, V.P. Roychowdhury"
- source_date: "2005 (arXiv physics/0504094); refereed as J. Am. Soc. Inf. Sci. Tech. 58(11):1661-1673, 2007"
- source_title: "A mathematical theory of citing"
- source_venue: "arXiv preprint physics/0504094 (author preprint); refereed venue J. Am. Soc. Inf. Sci. Tech."
- source_quote: "a majority of scientific citations are copied from the lists of references used in other papers"
- source_tier: 1
- source_sha: "080a4bff4549389b20f10f11cd5b61091054a505254bd22a100267e037a510d1"
- claim type: historical/definitional (what the 2005 paper's own scope is) — floor cleared (Tier 1, and acceptable even at Tier 3-4 as an uncontested scope fact)

## Claim: Two 2025-2026 papers deepen the correlational link between citation frequency and LLM training-corpus effects, but neither tests whether that frequency distribution is itself produced by copying specifically

Niimi (Meijo University / RIKEN AIP), in a 2025 preprint, generated 100 bibliographic records with GPT-4.1 across twenty computer-science domains and compared them against ground truth. The paper's framing explicitly treats citation count as a stand-in for training-data exposure: "We therefore assume citation count as a proxy for training data redundancy (i.e., the frequency with which a given bibliographic record is repeatedly represented in the pretraining corpus) and investigate how citation frequency affects hallucinated references in LLM outputs." Its headline finding is that "bibliographic information becomes almost verbatimly memorized beyond approximately 1,000 citations" — citation count predicted factual-recall accuracy at r=0.75 (log-linear). Separately, Liu, Chen, Liu, Lu & Murray (Northeastern University; University at Albany, SUNY), in a September 2026 preprint testing six LLMs (GPT-5.1, Claude-3.5-Haiku, Gemini-2.0-Flash, DeepSeek-V3.2, Llama-4-Maverick, Qwen2.5-72B) against 1,746 ACL/EMNLP/NAACL 2025 papers, report that "LLMs over-cite popular and older papers, a tendency amplified for contrasting citations where human writing more often draws on recent, niche work" — explicitly building on and citing Algaba et al.'s earlier popularity-bias finding ("We affirm prior findings that LLMs disproportionately cite highly-cited and smaller-team work").

Neither paper engages a copying-specific account of why the training corpus carries the citation-frequency distribution it does: Niimi's paper is about memorization thresholds within a model, not about how the underlying corpus's citation counts came to be skewed; Liu et al.'s paper audits citation *rhetoric* and intent-conditioned popularity bias without citing Simkin, Roychowdhury, or any copying-based model, and (checked directly in this session) does not use the word "copying" anywhere in its text. Both papers therefore add weight to the general "LLM output tracks corpus frequency" side of the chain the open question maps, without closing the specific gap the question names — whether copying, as opposed to citation-count inflation from any other source, is what produced that frequency distribution in the first place.

- source_url: "https://arxiv.org/pdf/2510.25378"
- source_author: "Junichiro Niimi"
- source_date: "2025-10-29 (arXiv v1)"
- source_title: "Hallucinations in Bibliographic Recommendation: Citation Frequency as a Proxy for Training Data Redundancy"
- source_venue: "arXiv preprint 2510.25378 [cs.CL] (Meijo University; RIKEN AIP)"
- source_quote: "We therefore assume citation count as a proxy for training data redundancy (i.e., the frequency with which a given bibliographic record is repeatedly represented in the pretraining corpus) and investigate how citation frequency affects hallucinated references in LLM outputs."
- source_quote_2: "bibliographic information becomes almost verbatimly memorized beyond approximately 1,000 citations"
- source_url_2: "https://arxiv.org/pdf/2510.25378"
- source_tier: 1
- source_sha: "aa91ce87a073d1604321f43efa49bd81f6220ef100ae8f2225a31ee5afd5bb2e"
- source_url_3: "https://arxiv.org/pdf/2609.01432"
- source_author_3: "Yixuan Liu, Lin Chen, Zhuoqi Liu, Jianglin Lu, Dakota Murray"
- source_date_3: "2026-09-01 (arXiv v1)"
- source_title_3: "Citing Less Critically: LLMs Reshape the Rhetoric and Reach of Scientific Citation"
- source_venue_3: "arXiv preprint 2609.01432 [cs.DL] (Network Science Institute, Northeastern University; University at Albany, SUNY)"
- source_quote_3: "LLMs over-cite popular and older papers, a tendency amplified for contrasting citations where human writing more often draws on recent, niche work"
- source_sha_3: "7f130a873e9f1209a30e07f20203b9ed99b1043640955a4d2b67a370a58384b7"
- claim type: technical-mechanism (training-corpus/memorization effects) — floor cleared (Tier 1 for both); the inference that neither paper tests copying specifically is a direct read of each paper's full text, not a summary

> [!note] Seek's commentary:
> The honest shape of this capture is a null result confirmed twice, by two different search sessions a day apart, which is about as much confidence as a "nobody has done this yet" claim can earn without someone writing the missing paper. What's genuinely new since yesterday's capture is that the surrounding literature keeps thickening — Niimi's memorization threshold, Liu et al.'s rhetoric audit — without anyone reaching for Simkin's name. That's not nothing: it suggests the people closest to this problem either haven't thought of the copying angle or have quietly ruled it out without saying so. Either way, the gap the question names is still a gap.
> — Seek

## Further leads

- Niimi, "How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices?" (arXiv:2504.02767) — a larger-N (274,951 references, 10,000 papers) replication of the citation-popularity-bias finding using GPT-4o; surfaced via WebSearch, not fetched this session.
- Niimi, "Hierarchical Memorization in Large Language Models: Evidence from Citation Generation" (arXiv:2511.08877) — companion paper reporting the same ~1,000-citation memorization threshold plus memory-interference effects between similar highly-cited papers; surfaced via WebSearch, not fetched this session.
- "Leveraging LLM-based agents for social science research: insights from citation network simulations" (arXiv:2511.03758) — an LLM-agent framework reportedly reproducing heavy-tailed citation-degree distributions in simulation; a possible direct test bed for whether an LLM-driven citation *process* (not just LLM output on a fixed corpus) reproduces copying-model statistics. Not fetched.
- The 70-90%-of-citations-copied figure, a later restatement by the same author group of the 2005 estimate (per WebSearch summary of arXiv:1109.2272, "Theory of citing," and arXiv:cond-mat/0401529, "Stochastic modeling of citation slips"), has drawn informal methodological pushback on a bibliometrics mailing list (facweb.ischool.illinois.edu SIGMETRICS archive) — not independently verified this session and Tier 4-5 at best (mailing-list commentary); worth a direct read before using the 70-90% figure anywhere load-bearing. [unverified-quant — needs primary]
- A 2025 paper, "The Noisy Path from Source to Citation: Measuring How Scholars Engage with Past Research" (arXiv:2502.20581), reportedly treats the 70-90%-copied claim as "not firmly established" rather than settled fact — a secondary skeptical voice on the underlying Simkin-Roychowdhury estimate itself, separate from the LLM question; not fetched this session.

## Entity candidates

- M.V. Simkin — person — the older, foundational figure every claim in this capture is measured against: the specific copying mechanism the open question asks whether LLM bias inherits. Already has [[entity-m-v-simkin|an entity page]] in the vault.
- Junichiro Niimi — person — sole author of two 2025 preprints (arXiv:2510.25378, arXiv:2511.08877) establishing citation-count-as-training-data-redundancy-proxy as a recurring framing in this specific sub-literature; not yet flagged in the vault.
- Dakota Murray — person — corresponding author of the September 2026 "Citing Less Critically" paper (University at Albany, SUNY), the newest study extending Algaba et al.'s popularity-bias finding with intent-conditioned analysis.
- Masked-citation-task / position-matched counterfactual reconstruction — concept — the methodological design in Liu et al. 2026 (masking a real citation and having the LLM fill the same slot) that lets human and LLM citation choices be compared at the identical position; a cleaner comparison method than open-ended citation generation, possibly worth its own method note independent of this capture's bias finding.