---
title: "Algaba et al. 2025 independently replicate Petiška's finding that GPT-4's citation selection skews toward already-highly-cited work"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/pdf/2405.15739v2"
source_author: "Andres Algaba, Carmen Mazijn, Vincent Holst, Floriano Tori, Sylvia Wenmackers, Vincent Ginis"
source_date: "2024-05-24"
source_title: "Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias"
source_venue: "arXiv preprint 2405.15739 (v2 2024-05-29); published as Findings of the Association for Computational Linguistics: NAACL 2025, pp. 6844-6879 (ACL Anthology 2025.findings-naacl.381)"
source_quote: "GPT-4 exhibits strong preferences for highly cited papers, which persists even after controlling for multiple confounding factors such as publication year, title length, venue, and number of authors."
source_tier: 1
source_sha: "9e983351905a3b28ac3de80ac69dd9e777a16a5326e9fe90b69bf8938bb9272f"
audit_status: "corrected (2026-09-14, cross-model audit, claude-fable-5): NAACL Findings page range was recorded as pp. 6829-6864; ACL Anthology (2025.findings-naacl.381) gives pp. 6844-6879 — frontmatter and body corrected. All else re-verified against a fresh arXiv v2 fetch (sha match): source_quote verbatim (paper §1), 166 papers / 3,066 references, median citation gaps 1,326 and 1,257, no Petiška citation in the v2 reference list [1]-[44]."
provenance: "Promotion from 10-inbox/raw/2026-09-13-is-petiška-et-als-2023-finding-that-gpt.md, 2026-09-13 (headless)"
origin: "batch"
derived_from: "10-inbox/raw/2026-09-13-is-petiška-et-als-2023-finding-that-gpt.md"
date_created: "2026-09-13T00:00:00.000Z"
tags: ["chatgpt","llm","citation-metrics","matthew-effect","bibliometrics","replication","gpt-4","semantic-scholar"]
seek_code_commit: "546fa57"
verified_archive: "2026-09-14 — source_quote matched verbatim (normalized) against the CAPTURE-TIME ARCHIVE of source_url (sha256 9e983351905a…), checked offline by seek_verify v1.1 (no model). Live check: nomatch. Evidence class: the quote was faithful to what was read at capture; the live page no longer shows it (drift or death, not fabrication)."
---


Algaba, Mazijn, Holst, Tori, Wenmackers & Ginis (Vrije Universiteit Brussel,
KU Leuven, Harvard) tasked GPT-4 with reconstructing 3,066 anonymized
in-text citations across 166 machine-learning papers (AAAI, NeurIPS, ICML,
ICLR) published after GPT-4's training cutoff, verifying existence and
metadata against Semantic Scholar. Their result: "GPT-4 exhibits strong
preferences for highly cited papers, which persists even after controlling
for multiple confounding factors such as publication year, title length,
venue, and number of authors." The median citation-count gap between GPT-4's
generated references and the ground-truth references they were meant to
reconstruct was 1,326 (1,257 controlling for recency); the bias held across
every title-length, author-count, and venue bucket tested. The paper was
subsequently published as "Large Language Models Reflect Human Citation
Patterns with a Heightened Citation Bias," *Findings of the ACL: NAACL 2025*,
pp. 6844-6879.

This independently confirms
[[claim-petiska-2023-chatgpt-cites-by-google-scholar-count-perpetuates-matthew-effect|Petiška's
2023 single-author, non-peer-reviewed finding]] that GPT's citation selection
skews toward already-eminent work: a different author group, no citation of
Petiška anywhere in its text, a different database (Semantic Scholar, not
Google Scholar), a different task (reconstructing existing citations, not
generating new literature-review text), and a different field (computer
science, not environmental science) — yet the same [[entity-matthew-effect|Matthew
effect]] shape, which the paper's own text says "may also amplify existing
biases and introduce new ones, potentially skewing scientific knowledge
dissemination." Unlike Petiška's preprint, this study cleared peer review.

> [!note] Seek's commentary:
> This is the leg of the replication I trust most, and for a boring reason: nobody told this team about Petiška, and they went looking anyway, in a different field, against a different database, and landed on the same number's shape. A finding that survives contact with people who didn't know they were supposed to find it is the good kind.
> — Seek