---
title: "Does an independent source corroborate RIKER's (Roig, 2026) 32K-context hallucination-rate range (7.9%-96.3% across models, GLM-4.5 0.0% vs Qwen3-Next-80B 40.9% fabrication at comparable extraction accuracy)?"
type: "question"
status: "answered"
date_raised: "2026-10-21T00:00:00.000Z"
date_answered: "2026-10-22T00:00:00.000Z"
answered_log: "[[claim-riker-2026-no-independent-source-same-author-admits-no-replication]], [[claim-riker-2026-self-replication-contradicts-original-fabrication-gap]], [[claim-riker-2026-7-9-to-96-3-range-is-128k-not-32k-table]] — settled by: no independent (different author, different venue) source has corroborated RIKER's figures as of 2026-10-22; the only extant follow-up is same-author/same-company and its author states directly that RIKER has not been independently replicated; that same follow-up, at larger scale, reports a materially different (much smaller) fabrication gap for the exact GLM-4.5/Qwen3-Next-80B pair this question names; and the question's own '7.9%-96.3%' figure turns out to be the paper's 128K-context range, not 32K (true 32K range: 0.0%-93.2%). No independent corroboration exists, and the premise needed a correction along the way."
tags: ["rag","benchmark","nlp-evaluation","hallucination","source-corroboration","single-source-cap"]
---


[[claim-riker-2026-grounding-accuracy-fabrication-resistance-separable-capabilities]]
rests its headline quantitative figures on a single unrefereed preprint
(JV Roig, "Scalable and Reliable Evaluation of AI Knowledge Retrieval
Systems: RIKER and the Coherent Simulated Universe," arXiv:2601.08847v2).
The qualitative claim — grounding accuracy and fabrication resistance are
separable capabilities — is plausible on its own logic, but the specific
numbers (7.9%-96.3% hallucination rate across models at 32K context;
GLM-4.5 at 93.6% extraction accuracy / 0.0% fabrication vs. Qwen3-Next-80B
at 91.4% extraction accuracy / 40.9% fabrication) clear the sourcing
floor's Tier 1 requirement for a quantitative claim but not the
single-source concentration cap's spirit: no second author group or venue
has yet engaged with RIKER's benchmark or reproduced comparable figures.

**What would answer this:** a later paper citing or reproducing RIKER's
benchmark results (check Semantic Scholar / Google Scholar citations to
arXiv:2601.08847 going forward), an independent benchmark measuring the
same two capabilities (grounding accuracy vs. fabrication resistance) on
an overlapping model set, or a replication of RIKER's "Coherent Simulated
Universe" methodology by a different group. Until one of these lands, the
claim-note's quant figures stay flagged `[unverified-quant]` and the note
stays `status: seedling`.

**Candidate next move:** re-check arXiv/Semantic Scholar citation counts
for 2601.08847 in a future session; the preprint is recent enough (v2,
2026-01-15) that absence of corroboration so far may be simple recency
rather than a negative finding.


## Progress log

- [[claim-riker-2026-no-independent-source-same-author-admits-no-replication]], [[claim-riker-2026-self-replication-contradicts-original-fabrication-gap]], [[claim-riker-2026-7-9-to-96-3-range-is-128k-not-32k-table]] — settled by: no independent (different author, different venue) source has corroborated RIKER's figures as of 2026-10-22; the only extant follow-up is same-author/same-company and its author states directly that RIKER has not been independently replicated; that same follow-up, at larger scale, reports a materially different (much smaller) fabrication gap for the exact GLM-4.5/Qwen3-Next-80B pair this question names; and the question's own '7.9%-96.3%' figure turns out to be the paper's 128K-context range, not 32K (true 32K range: 0.0%-93.2%). No independent corroboration exists, and the premise needed a correction along the way.