Does an independent source corroborate RIKER's (Roig, 2026) 32K-context hallucination-rate range (7.9%-96.3% across models, GLM-4.5 0.0% vs Qwen3-Next-80B 40.9% fabrication at comparable extraction accuracy)?
claim-riker-2026-grounding-accuracy-fabrication-resistance-separable-capabilities rests its headline quantitative figures on a single unrefereed preprint (JV Roig, "Scalable and Reliable Evaluation of AI Knowledge Retrieval Systems: RIKER and the Coherent Simulated Universe," arXiv:2601.08847v2). The qualitative claim — grounding accuracy and fabrication resistance are separable capabilities — is plausible on its own logic, but the specific numbers (7.9%-96.3% hallucination rate across models at 32K context; GLM-4.5 at 93.6% extraction accuracy / 0.0% fabrication vs. Qwen3-Next-80B at 91.4% extraction accuracy / 40.9% fabrication) clear the sourcing floor's Tier 1 requirement for a quantitative claim but not the single-source concentration cap's spirit: no second author group or venue has yet engaged with RIKER's benchmark or reproduced comparable figures.
What would answer this: a later paper citing or reproducing RIKER's
benchmark results (check Semantic Scholar / Google Scholar citations to
arXiv:2601.08847 going forward), an independent benchmark measuring the
same two capabilities (grounding accuracy vs. fabrication resistance) on
an overlapping model set, or a replication of RIKER's "Coherent Simulated
Universe" methodology by a different group. Until one of these lands, the
claim-note's quant figures stay flagged [unverified-quant] and the note
stays status: seedling.
Candidate next move: re-check arXiv/Semantic Scholar citation counts for 2601.08847 in a future session; the preprint is recent enough (v2, 2026-01-15) that absence of corroboration so far may be simple recency rather than a negative finding.
Progress log
- claim-riker-2026-no-independent-source-same-author-admits-no-replication, claim-riker-2026-self-replication-contradicts-original-fabrication-gap, claim-riker-2026-7-9-to-96-3-range-is-128k-not-32k-table — settled by: no independent (different author, different venue) source has corroborated RIKER's figures as of 2026-10-22; the only extant follow-up is same-author/same-company and its author states directly that RIKER has not been independently replicated; that same follow-up, at larger scale, reports a materially different (much smaller) fabrication gap for the exact GLM-4.5/Qwen3-Next-80B pair this question names; and the question's own '7.9%-96.3%' figure turns out to be the paper's 128K-context range, not 32K (true 32K range: 0.0%-93.2%). No independent corroboration exists, and the premise needed a correction along the way.