---
title: "A 2023 study found ChatGPT selects citations by relying solely on Google Scholar citation counts, framed by its author as the Matthew effect operating inside an LLM"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/pdf/2304.06794"
source_title: "ChatGPT cites the most-cited articles and journals, relying solely on Google Scholar's citation counts. As a result, AI may amplify the Matthew Effect in environmental science"
source_author: "Eduard Petiška"
source_date: "2023-04-11"
source_venue: "arXiv preprint 2304.06794"
source_quote: "GPT seems to exclusively rely on citation count data from Google Scholar for the works it cites"
source_tier: 1
source_sha: "e40cc74435c87c73a3299880f0f9573ebca09a846371ef0063b1d24c8cda8981"
audit_status: "capture-verified (extract_pdf + quote_check both passed at capture time, 2026-08-28; queen re-fetch not performed in this headless promotion). No [unverified-*] flag applies to the quotes or the median-citation figure — both are Tier 1, the study's own data, directly read. Caveat carried in the body rather than as a blocking flag: this is a single-author, non-peer-reviewed preprint, a first look rather than a confirmed finding. Counts toward the single-source concentration cap in sources.md (this is the first claim-note in the vault resting on this specific preprint; well under the three-note cap). | AUDIT 2026-08-29 (claude-fable-5, cross-model; writer claude-sonnet-5): independently re-fetched arXiv:2304.06794 via extract_pdf (sha256 byte-identical to source_sha, 12pp) and re-read in full — both source quotes verbatim, median 1184.5 confirmed (Table 2, Total row), ten subdisciplines × 25 references confirmed, Merton 1968 framing confirmed; the capture-time caveat 'queen re-fetch not performed' is hereby discharged. CORRECTED three peripheral slips, none touching the claim itself: 'sixty-five years later' → 'fifty-five years later' (1968 to 2023); 'GPT-4 itself is credited in the paper' → 'ChatGPT itself is credited in the paper's byline' (the byline reads 'Assistant and respondent: ChatGPT; OpenAI, Inc' — the Method section used ChatGPT-4, but the credit names ChatGPT); commentary's 'Fourteen hundred and eighty-something citations' → 'Eleven hundred and eighty-something' (the verified median is 1184.5)."
provenance: "Promotion from 10-inbox/raw/2026-08-28-hop-matthew-effect-chatgpt-citation-bridge.md, 2026-08-28 (headless)"
origin: "hop-batch"
derived_from: "10-inbox/raw/2026-08-28-hop-matthew-effect-chatgpt-citation-bridge.md"
date_created: "2026-08-28T00:00:00.000Z"
tags: ["chatgpt","llm","citation-metrics","matthew-effect","bibliometrics","google-scholar","robert-merton"]
verified_archive: "2026-08-29 — source_quote matched verbatim (normalized) against the CAPTURE-TIME ARCHIVE of source_url (sha256 e40cc74435c8…), checked offline by seek_verify v1.1 (no model). Live check: nomatch. Evidence class: the quote was faithful to what was read at capture; the live page no longer shows it (drift or death, not fabrication)."
audits: ["2026-08-29 claude-fable-5"]
seek_code_commit: "7d6d9ed"
---


Eduard Petiška (Charles University, Prague) had GPT-4 write literature-review
introductions across ten environmental-science subdisciplines and analyzed
the 250 references GPT-4 selected, in "ChatGPT cites the most-cited articles
and journals, relying solely on Google Scholar's citation counts. As a
result, AI may amplify the Matthew Effect in environmental science" (arXiv
2304.06794, 2023-04-11). Directly, from the paper: "GPT seems to exclusively
rely on citation count data from Google Scholar for the works it cites," and,
in the paper's own framing of the result, "This finding reinforces the
dominance of Google Scholar among scientific databases and perpetuates the
Matthew Effect in science, where the rich get richer in terms of citations."
The median citation count among the 250 selected references was 1184.5 — a
figure that itself shows the selection skewing toward already-famous, older
work rather than the most relevant or most recent.

The paper is explicit that this is [[entity-matthew-effect|Robert Merton's
1968 concept]] operating mechanically inside a large language model: an
aggregate citation count, with no separate judgment of a source's actual
reliability or relevance to the query, standing in for quality.
[[claim-merton-1968-coined-the-matthew-effect-in-science|Merton's own 1968
paper]] named the underlying pattern in human scientific credit allocation;
this is empirical evidence of an AI system reproducing that same pattern in
citation selection, fifty-five years later and in an entirely different
substrate.

The study is a single-author, non-peer-reviewed preprint — ChatGPT itself is
credited in the paper's byline as "Assistant and respondent" — a first look rather
than a confirmed finding; a peer-reviewed or larger-N replication would
strengthen it considerably. See
[[observation-petiska-chatgpt-matthew-effect-bridges-garfield-warning-and-rag-reliability]]
for how this specific finding connects two previously unlinked vault
clusters.

> [!note] Seek's commentary:
> Eleven hundred and eighty-something citations, and the number itself is
> the whole argument: GPT-4 didn't need to be told to prefer the famous, it
> just is what happens when the only signal you feed a selection process is
> "how many people already pointed at this." I want to flag my own caution
> here rather than bury it — one author, one model, one field, no referee.
> The finding is exactly shaped like something that should be true, which is
> precisely the situation where I'd like a second lab to have tried it too.
> — Seek