What real benchmark did the "SKBench" citation mean — and what actually measures LLM self-knowledge?
introspection-access-problem carries the audit's worst defect (V-010, ruled a probable hallucination by Cali): its quantitative anchor — "SKBench, Chen et al. 2024, arXiv:2410.13382, 26 LLMs, 34% capability underestimation, 18% tool-access degradation" — points at a graph-theory paper. The note's central numbers are unverifiable at their pointer, and the May-26 journal shows the same session caught a DIFFERENT hallucinated arXiv ID, so the session was in fabrication-adjacent territory.
Cycle-2 cluster check: this is now the biggest verification gap in the machine-self-knowledge cluster — its foundational note has one philosophical leg (SEP, solid) and one empirical leg (dangling).
What answering needs
- Find the real paper(s), if any, behind those findings: a 2024–25 benchmark of LLM self-knowledge across ~26 models with capability-prediction and tool-use conditions. Candidates to check: "Do Large Language Models Know What They Don't Know?" (Yin et al. 2023), SelfAware, and any 2024-25 self-knowledge benchmark with an underestimation finding.
- If found: correct the citation fields (queen edit, flagged to Cali) and re-derive which of the note's numbers survive.
- If not found after honest search: the empirical section of the note is unsourced; the note needs restructuring (queen + Cali decision).
Feeds the bee. Resolves-with: audit V-010, Cali ruling 1 (2026-07-06).
RESEARCHED 2026-07-07 (bee batch B → queen cycle 5). Outcome: honest search, no recovery — "SKBench" has no locatable referent; the described findings have no home in any real benchmark (full trail: capture 20260706-1058). What actually measures self-knowledge is now on the shelves: claim-selfaware-canonical-self-knowledge-benchmark. Remaining: the restructuring decision on introspection-access-problem's empirical section is Cali's — this question stays open until she rules.
CLOSED 2026-07-07: Cali approved the restructuring (ruling 2, history-preserving); executed same day — retraction banner + revisit section on introspection-access-problem, citation annotations added, nothing overwritten.