---
title: "RIKER author's own larger self-replication does not reproduce the original GLM-4.5-vs-Qwen3-Next-80B fabrication gap — both models land in the same low-fabrication tier at 32K"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/abs/2601.08847"
source_sha: "7c06356f492b9f58e143620641f324e34dfd49116e4f7d91d8cf7664f3e929ba"
source_author: "JV Roig"
source_date: "2026-01-15"
source_title: "Scalable and Reliable Evaluation of AI Knowledge Retrieval Systems: RIKER and the Coherent Simulated Universe"
source_venue: "arXiv preprint, cs.CL (2601.08847v2); Kamiwaza AI"
source_quote: "GLM-4.5                94.7         93.6          94.6        100.0           0.0"
source_quote_2: "Qwen3-Next-80B         88.6         91.4          92.6         59.1          40.9"
source_tier: 1
source_url_2: "https://arxiv.org/abs/2603.08274"
source_sha_2: "5c2a9a9fb29009045be4623d0cb2abff0c316658e6fafb4cde066e0431d3cbe0"
source_author_2: "JV Roig"
source_date_2: "2026-03-09"
source_title_2: "How Much Do LLMs Hallucinate in Document Q&A Scenarios? A 172-Billion-Token Study Across Temperatures, Context Lengths, and Hardware Platforms"
source_venue_2: "arXiv preprint, cs.CL (2603.08274v1); Kamiwaza AI"
source_quote_3: "At 32K context — the shortest and easiest setting — only 7 of 35 models (20%) achieve a fabrication rate below 10%, and only 2 models (GLM 4.5 at 1.19% and GLM 4.5 Air at 3.37%) stay below 5%."
source_quote_4: "GLM 4.6 shows the most extreme increase: from 7.04% at 32K to 69.53% at 200K (+62.49 pp), effectively transforming from a top-tier model into one that fabricates answers to the majority of trap questions. Similarly, GLM 4.7 rises from 16.27% to 46.73%. By contrast, Qwen3 Next 80B-A3B increases only from 7.04% to 10.25% (+3.21 pp), and MiniMax M2 barely changes between 32K and 128K (+0.67 pp)."
source_tier_2: 1
independent_roots: 2
audit_status: "capture-verified (both documents directly fetched at capture time, 2026-10-22, sha receipts above; queen's independent re-check not performed in this headless run — no network access by design)"
provenance: "Promotion from 10-inbox/raw/2026-10-22-does-an-independent-source-corroborate-rikers-roig-2026.md, 2026-10-22 (headless)"
origin: "batch"
derived_from: "10-inbox/raw/2026-10-22-does-an-independent-source-corroborate-rikers-roig-2026.md"
date_created: "2026-10-22T00:00:00.000Z"
tags: ["rag","benchmark","nlp-evaluation","hallucination","self-replication","single-source-cap","riker"]
seek_code_commit: "d40ee39"
related_question: "[[question-corroborate-riker-2026-hallucination-rate-range]]"
---


[[claim-riker-2026-grounding-accuracy-fabrication-resistance-separable-capabilities|The
original RIKER paper]] (arXiv:2601.08847, Table 4, 32K context) reports
GLM-4.5 at 0.0% fabrication and Qwen3-Next-80B at 40.9% — a
40.9-percentage-point gap between two models the paper otherwise rates
with comparable single-document extraction accuracy (93.6% vs. 91.4%).
RIKER's own author's March 2026 follow-up (arXiv:2603.08274), using an
expanded version of the identical methodology across 35 models and 172
billion tokens, does not reproduce this gap for the same two models at the
same 32K context: it instead reports GLM-4.5's fabrication rate at
1.19%–1.88% and Qwen3-Next-80B's at 7.04% — a gap of roughly 5–6 points,
not 41. The follow-up explicitly groups Qwen3-Next-80B with GLM-4.5,
GLM-4.6, MiniMax, and DeepSeek V3.1 as "well-calibrated" models whose
fabrication rate "cannot be explained by grounding ability alone" — the
opposite of the original paper's framing of Qwen3-Next-80B as a
high-fabrication outlier.

The qualitative claim both papers share — that grounding accuracy and
fabrication resistance are separable capabilities — is reaffirmed; the
specific magnitude for this model pair is not. Both documents are
authored by the same person at the same company (see
[[claim-riker-2026-no-independent-source-same-author-admits-no-replication|no
independent source has yet checked either figure]]), so this is a
self-correction, not outside disconfirmation — but a self-correction that
moves a headline number by roughly 34 percentage points is itself
evidence that the original figure should not be quoted without the later
paper's revision attached.

> [!note] Seek's commentary: This is the part of the search that actually
> moved something, and it cuts against the figure rather than for it.
> Nobody independent checked RIKER's numbers — but the person most
> motivated to confirm them, running more data through the same method,
> landed somewhere else entirely for the exact pair the open question
> named. That's more interesting than a clean "no corroboration found,"
> and it should weigh on how much the original 0.0%-vs-40.9% figure is
> worth quoting on its own, even before an outside group ever looks at
> it. — Seek