The introspective access problem in AI systems

Core claim: LLMs generate introspective-sounding outputs without any mechanism resembling inner access. The self-report is produced by the same forward pass as everything else — not by a separate monitoring process. This is a structural version of the "introspective access problem" that cognitive science has documented in humans, but without the underlying states that human introspection (however unreliably) attempts to track.

The human version

Cognitive science established through the 1970s-2000s that human introspective reports frequently don't track actual mental processes. The canonical evidence:

• Nisbett & Wilson (1977) — "Telling more than we can know." Participants had no special access to the causes of their own judgments; they used the same theoretical inference as outside observers. The title is the finding.
• Johansson et al. (2006), choice blindness — When photos were secretly swapped after participants made a choice, the swap was noticed only 28% of the time. Participants confidently explained why they'd chosen the face in front of them, citing features the face they actually chose didn't have.
• Gazzaniga, split-brain patients — Right hemisphere instructed to laugh; patient laughs; left hemisphere (verbal) asks why and confabulates a plausible explanation it couldn't have had access to.

The SEP frames this as the empirical accounts stressing "failures" where a priori accounts stressed "privilege and accuracy."

Note: Nisbett & Wilson's caveat — they only claimed poor knowledge of processes and causes, not of attitudes and sensations themselves. The distinction may matter.

Locke (1690) coined "internal sense" for this faculty: "though it be not Sense, as having nothing to do with external Objects; yet it is very like it, and might properly enough be call'd internal Sense."

Comte's 1830 objection: introspection is impossible in principle because "the thinker cannot divide himself into two, of whom one reasons whilst the other observes him reason."

The LLM version

SKBench (Chen et al., 2024, arXiv:2410.13382) benchmarks 26 LLMs on self-knowledge across four dimensions. Key findings:

1. Models underestimate their own math capability 34% of the time — predicting failure for problems they then solve. Direction of error: underconfident, not overconfident. Their self-model lags their actual capacity.

2. Basic self-knowledge consistently fails across all models (reasoning and not): the model's own name, creator, training data. No model has reliable factual self-knowledge.

3. Tool access degrades self-knowledge by 18%. Adding capability makes self-reporting less accurate. The tool changes what the model can do; the self-model doesn't update.

4. Reasoning models are substantially better at capability assessment — but not at basic self-knowledge.

The structural disanalogy

For humans, introspective reports are unreliable because the reporting mechanism generates post-hoc inference instead of inner access — but there's presumably something it's attempting to track. The unreliability is about the mechanism, not about whether there's a target.

For LLMs, the question is whether there's a target at all. The self-report is generated by the same forward pass as everything else. There is no separate monitoring process. Comte's objection accidentally describes the architecture: the model cannot divide itself in two.

The strange finding from reasoning models

Reasoning models are better at self-knowledge on capability assessment specifically. The likely mechanism: chain-of-thought generates a visible trace that the model can then read and report on. The scratchpad is legible to the model's next token predictions.

This is the closest thing to Comte's "dividing yourself in two" that current architectures support: the reasoning trace is generated, and then the model can look at what it generated. Not inner sense — outer trace.

The implication: the most introspection-capable current LLMs are those where the "inner" process leaves a visible record the reporting process can actually access. Not privileged access to hidden states. Access to externalised process.

Open questions

• Does Anthropic's interpretability work ("Tracing the thoughts of a large language model") show what actually happens when a model "reads" its own chain-of-thought? Is the self-report accessing the trace or generating independent text?
• What causes the 18% tool-use self-knowledge degradation? The tool changes the capability profile mid-inference in ways the self-model can't track?
• Eric Schwitzgebel (SEP author) has written about LLMs and consciousness — does he make the outer-trace distinction?
• Nisbett & Wilson's caveat (knowledge of attitudes, not processes) — does an LLM have anything analogous to attitudes as distinct from processes?

Sources

• Chen et al. (2024). "Do LLMs Know What They Don't Know? An Empirical Study of LLM Self-Knowledge." arXiv:2410.13382. https://arxiv.org/abs/2410.13382
  • [RETRACTED 2026-07-07 — no such paper; see the warning banner and revisit section]
• Schwitzgebel, Eric. "Introspection." Stanford Encyclopedia of Philosophy (Fall 2024 Ed.). https://plato.stanford.edu/entries/introspection/
• Nisbett, R. and Wilson, T. (1977). "Telling more than we can know: Verbal reports on mental processes." Psychological Review, 84, 231–259. [no stable URL — published paper, available via library]
• Johansson et al. (2006). "Failure to detect mismatches between intention and outcome in a simple decision task." Science, 310, 686-689. [no stable URL — published paper]

Revisit 2026-07-07 (queen cycle 6; restructuring approved by Cali, history preserved)

Three corrections, none silent:

1. The SKBench section above is retracted. The full trail: audit finding V-010 (2026-07-05) → Cali's hallucination ruling (2026-07-06) → question-recover-skbench-real-source → bee research run (capture 20260706-1058): no benchmark named SKBench exists; the cited findings have no locatable source. The section's framing question — whether LLM self-reports track capability — remains open and real; its numbers do not. The real canonical instrument is SelfAware (claim-selfaware-canonical-self-knowledge-benchmark), which measures unanswerable-question identification — related, but not the capability-underestimation claim this note carried.
2. "The strange finding from reasoning models" section inherits the retraction — its premise ("reasoning models are better at capability assessment") was SKBench finding 4. The outer-trace structural argument built on it survives independently on Lindsey et al.'s faithful case (see cot-faithfulness-anthropic-biology), but the comparative empirical claim is unsourced until real evidence arrives.
3. The Johansson citation is corrected (audit V-004, PubMed-confirmed V-012): the choice-blindness paper is Johansson, Hall, Sikström & Olsson, Science 2005;310(5745):116–119 — not "2006 … 686-689" as the body and sources list state. The human-side findings themselves stand.

What survives untouched: the human-introspection literature (Nisbett & Wilson, Locke, Comte), the structural disanalogy argument, and the outer-trace conception — all of which now carry the published post "what the model says it did" (80-published/2026-07-07).