Niimi 2025 finds LLM bibliographic-record recall correlates with citation count (r=0.75, log-linear), becoming near-verbatim above roughly 1,000 citations

llm-citation-biastraining-data-exposurememorizationcitation-metricsmatthew-effectmechanism-comparison

Niimi (Meijo University; RIKEN AIP), in a 2025 preprint, generated 100 bibliographic records with GPT-4.1 across twenty computer-science domains and compared them against ground truth, treating citation count as a stand-in for how often a record repeats in the pretraining corpus: "We therefore assume citation count as a proxy for training data redundancy (i.e., the frequency with which a given bibliographic record is repeatedly represented in the pretraining corpus) and investigate how citation frequency affects hallucinated references in LLM outputs." The paper's headline finding is a log-linear correlation of r=0.75 between citation count and factual-recall accuracy, with a sharp practical threshold: "bibliographic information becomes almost verbatimly memorized beyond approximately 1,000 citations." Below that count, hallucination in reproduced bibliographic details rises roughly in proportion to how rarely the record appears.

This joins Mozafari et al.'s direct exposure-count confirmation and Naser's cross-vendor popularity audit as further evidence that LLM output tracks training-corpus frequency. Like those two, it does not test or engage Simkin & Roychowdhury's copying mechanism as a candidate explanation for why the underlying citation-frequency distribution looks the way it does — it measures a memorization effect downstream of that distribution, not its cause. See question-does-citation-copying-explain-training-corpus-skew-behind-llm-citation-bias.