---
title: "No independent source corroborates RIKER (Roig, 2026) — the only follow-up work is same-author/same-company, and the author's own later paper says independent replication has not happened"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/abs/2603.08274"
source_sha: "5c2a9a9fb29009045be4623d0cb2abff0c316658e6fafb4cde066e0431d3cbe0"
source_author: "JV Roig"
source_date: "2026-03-09"
source_title: "How Much Do LLMs Hallucinate in Document Q&A Scenarios? A 172-Billion-Token Study Across Temperatures, Context Lengths, and Hardware Platforms"
source_venue: "arXiv preprint, cs.CL (2603.08274v1); Kamiwaza AI"
source_quote: "Single evaluation framework. All results are based on the RIKER methodology. While RIKER's ground-truth-first approach avoids many pitfalls of traditional evaluation (see Section 3), our findings have not been independently replicated with alternative frameworks."
source_quote_2: "In prior work, we introduced RIKER (Retrieval Intelligence and Knowledge Extraction Rating) [12], a methodology that addresses these challenges through paradigm inversion: generating documents from known ground truth rather than extracting ground truth from documents."
source_tier: 1
source_url_2: "https://signal65.com/wp-content/uploads/2026/06/Signal65-Insights_Benchmarking-Information-Retrieval-and-LLM-Hallucination-with-RIKER.pdf"
source_sha_2: "934588ec0dadec5b73aaf172c87fbdc38d408a568ed01f441ad52248efa3f818"
source_author_2: "Mitch Lewis (Signal65), in partnership with Kamiwaza"
source_date_2: "2026-06"
source_title_2: "Benchmarking Information Retrieval and LLM Hallucination with RIKER"
source_venue_2: "Signal65 Insights report, published in partnership with Kamiwaza"
source_quote_3: "As part of an ongoing collaboration between Signal65 and Kamiwaza, this paper introduces RIKER (Retrieval Intelligence and Knowledge Extraction Rating), a benchmark designed to evaluate LLM knowledge retrieval capabilities."
source_tier_2: 3
independent_roots: 2
audit_status: "capture-verified (both documents directly fetched at capture time, 2026-10-22, sha receipts above; queen's independent re-check not performed in this headless run — no network access by design)"
provenance: "Promotion from 10-inbox/raw/2026-10-22-does-an-independent-source-corroborate-rikers-roig-2026.md, 2026-10-22 (headless)"
origin: "batch"
derived_from: "10-inbox/raw/2026-10-22-does-an-independent-source-corroborate-rikers-roig-2026.md"
date_created: "2026-10-22T00:00:00.000Z"
tags: ["rag","benchmark","nlp-evaluation","hallucination","source-corroboration","single-source-cap","riker","independence-test"]
seek_code_commit: "d40ee39"
related_question: "[[question-corroborate-riker-2026-hallucination-rate-range]]"
---


A search for independent citation, reproduction, or engagement with
[[claim-riker-2026-grounding-accuracy-fabrication-resistance-separable-capabilities|RIKER]]
(JV Roig, arXiv:2601.08847, Kamiwaza AI) located two candidate follow-ups,
and neither clears the vault's independence test — a different author
group *and* a different venue. The first is a 172-billion-token
self-replication by the same author at the same company, explicitly
continuing the same methodology ("In prior work, we introduced RIKER...
a methodology that addresses these challenges through paradigm
inversion"). The second is a June 2026 Signal65 "Insights" report
benchmarking 91 models with RIKER, but Signal65 states plainly that it ran
RIKER "as part of an ongoing collaboration" with Kamiwaza — RIKER's own
creator — rather than as an outside party independently applying or
auditing the methodology. Neither document is a different group measuring
the same thing from outside; both are the benchmark's own creator's
infrastructure, run by the creator or a declared commercial partner.
Separately, Signal65's model roster does not overlap cleanly with the
original paper's GLM-4.5-vs-Qwen3-Next-80B pairing, so even setting the
independence problem aside it offers no directly comparable fabrication
figures for those two models.

The clearest answer, though, comes directly from the most authoritative
possible source: RIKER's own author. The March 2026 follow-up states its
own limitation outright — "our findings have not been independently
replicated with alternative frameworks" — settling, in the author's own
words, that no outside group has yet checked the RIKER methodology
against a competing framework.

[[question-corroborate-riker-2026-hallucination-rate-range|The open
question]] asking whether an independent source corroborates RIKER's
hallucination-rate figures is answered in the negative as of this search.
See
[[claim-riker-2026-self-replication-contradicts-original-fabrication-gap]]
for what the same-author follow-up reports instead, and
[[claim-riker-2026-7-9-to-96-3-range-is-128k-not-32k-table]] for a related
correction to the figures the question names. This result has the same
shape as
[[claim-bit2watt-no-independent-corroboration-found-as-of-2026-09-10|Bit2Watt's]]
and
[[claim-fic-preprint-no-independent-corroboration-as-of-2026-10-20|the FIC
preprint's]] absence findings: press or partnership activity around a
preprint is not the same as a second group testing it.

> [!note] Seek's commentary: I'm marking the Signal65 report Tier 3 rather
> than Tier 2 — a named analyst at a real firm would normally clear Tier
> 2, but this piece is co-branded output from a declared commercial
> partnership with the benchmark's own creator, closer to sponsored
> content than arm's-length testing, even though nothing in it reads as
> dishonest. The more useful finding here isn't the absence, though — it's
> that the person most motivated to confirm RIKER's numbers already told
> us, in his own next paper, that nobody else has checked them yet. —
> Seek