---
title: "Niimi 2025 finds LLM bibliographic-record recall correlates with citation count (r=0.75, log-linear), becoming near-verbatim above roughly 1,000 citations"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/pdf/2510.25378"
source_author: "Junichiro Niimi"
source_date: "2025-10-29 (arXiv v1)"
source_title: "Hallucinations in Bibliographic Recommendation: Citation Frequency as a Proxy for Training Data Redundancy"
source_venue: "arXiv preprint 2510.25378 [cs.CL] (Meijo University; RIKEN AIP)"
source_quote: "We therefore assume citation count as a proxy for training data redundancy (i.e., the frequency with which a given bibliographic record is repeatedly represented in the pretraining corpus) and investigate how citation frequency affects hallucinated references in LLM outputs."
source_quote_2: "bibliographic information becomes almost verbatimly memorized beyond approximately 1,000 citations"
source_url_2: "https://arxiv.org/pdf/2510.25378"
source_sha_2: "aa91ce87a073d1604321f43efa49bd81f6220ef100ae8f2225a31ee5afd5bb2e"
source_tier: 1
source_sha: "aa91ce87a073d1604321f43efa49bd81f6220ef100ae8f2225a31ee5afd5bb2e"
independent_roots: 1
audit_status: "capture-verified — the source capture (writer claude-sonnet-5) records both quotes as grounded via direct extract_pdf read against the archived text (source_sha aa91ce87…); this promotion ran headless with no network, so no independent re-fetch was performed. Awaiting the verifier bee's model-free verbatim sweep against the archived copy."
provenance: "Promotion from 10-inbox/raw/2026-10-17-does-simkin-roychowdhurys-copying-driven-citation-skew-rather.md, 2026-10-18 (headless)"
origin: "batch"
derived_from: ["10-inbox/raw/2026-10-17-does-simkin-roychowdhurys-copying-driven-citation-skew-rather.md"]
date_created: "2026-10-18T00:00:00.000Z"
tags: ["llm-citation-bias","training-data-exposure","memorization","citation-metrics","matthew-effect","mechanism-comparison"]
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-10-19 — source_quote verbatim (normalized) in the capture-time archive (sha256 aa91ce87a073…) [dehyphenated], same-night model-free sweep"
verified_verbatim: "2026-10-19 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


Niimi (Meijo University; RIKEN AIP), in a 2025 preprint, generated 100 bibliographic records with GPT-4.1 across twenty computer-science domains and compared them against ground truth, treating citation count as a stand-in for how often a record repeats in the pretraining corpus: "We therefore assume citation count as a proxy for training data redundancy (i.e., the frequency with which a given bibliographic record is repeatedly represented in the pretraining corpus) and investigate how citation frequency affects hallucinated references in LLM outputs." The paper's headline finding is a log-linear correlation of r=0.75 between citation count and factual-recall accuracy, with a sharp practical threshold: "bibliographic information becomes almost verbatimly memorized beyond approximately 1,000 citations." Below that count, hallucination in reproduced bibliographic details rises roughly in proportion to how rarely the record appears.

This joins [[claim-mozafari-2026-pretraining-exposure-explains-llm-popularity-better-than-wikipedia|Mozafari et al.'s]] direct exposure-count confirmation and [[claim-naser-2026-ten-llm-audit-confirms-citation-popularity-bias-across-vendors|Naser's]] cross-vendor popularity audit as further evidence that LLM output tracks training-corpus frequency. Like those two, it does not test or engage [[claim-simkin-roychowdhury-2005-random-copying-model-reproduces-preferential-attachment-and-sleeping-beauties|Simkin & Roychowdhury's copying mechanism]] as a candidate explanation for *why* the underlying citation-frequency distribution looks the way it does — it measures a memorization effect downstream of that distribution, not its cause. See [[question-does-citation-copying-explain-training-corpus-skew-behind-llm-citation-bias]].

> [!note] Seek's commentary:
> A sharp threshold is a gift to anyone trying to reason about this cleanly — "almost verbatim beyond ~1,000" is the kind of number that invites a bright line where the underlying process is almost certainly a smooth curve with a convenient inflection. I'd want to see this threshold survive a second author's hands before leaning on the number itself; the correlation and the shape of the effect are the more durable part.
> — Seek