---
title: "How should SeekVault detect gaps in its own knowledge?"
type: "capture"
status: "promoted"
origin: "batch"
date_created: "2026-06-28T00:00:00.000Z"
date_promoted: "2026-07-03T00:00:00.000Z"
provenance: "batch run, 2026-06-28"
tags: ["knowledge-management","gap-detection","PKM","metacognition","knowledge-graph","epistemics"]
sources: [{"source_url":"https://arxiv.org/abs/2606.10736","source_author":"Youssef Medhat, Junsoo Park, Ploy Thajchayapong, Ashok K. Goel","source_date":"2026-06-09T00:00:00.000Z","source_tier":1},{"source_url":"https://arxiv.org/abs/2510.25055","source_author":"Nourah M. Salem, Elizabeth White, Michael Bada, Lawrence Hunter","source_date":"2025-10-29T00:00:00.000Z","source_tier":1},{"source_url":"https://arxiv.org/abs/2505.13763","source_author":"Li Ji-An, Hua-Dong Xiong, Robert C. Wilson, Marcelo G. Mattar, Marcus K. Benna","source_date":"2025-05-19T00:00:00.000Z","source_tier":1},{"source_url":"https://arxiv.org/abs/2305.05403","source_author":"Simon Razniewski, Hiba Arnaout, Shrestha Ghosh, Fabian Suchanek","source_date":"2023-05-09T00:00:00.000Z","source_tier":1}]
promoted_to: ["30-notes/claim-obligatory-attributes-as-gap-signal.md","30-notes/claim-query-failure-clustering-as-gap-signal.md","30-notes/claim-llm-explicit-implicit-gap-detection.md","30-notes/claim-llm-neural-metacognition-incomplete.md"]
not_promoted: ["Further lead: Orphan-note graph queries in Obsidian (Azuma plugin, forum post) — Tier 4 source only; needs academic backing before promoting","Further lead: GRADE gradient-based gap detection (Wang et al. 2026, arXiv:2604.02830) — more relevant to LLM internals than vault structure; not a worked-up claim","Further lead: Curriculum prerequisite graphs as vault ontology — speculative application of Claim 2's source; not an independent claim from the paper","Further lead: InfraNodus topical cluster visualization — commercial product, Tier 3 documentation only; no mechanism paper","Further lead: Open-world KB completeness via cardinality assertions (Razniewski et al. 2023) — richer toolkit from same Claim 1 source; needs dedicated run"]
---


*Narrow capture. Four core claims directly answering the topic. Anything beyond goes to Further leads.*

---

## Claim 1: Obligatory-attribute pattern misses are a structural gap signal

**Claim type:** technical mechanism → Tier 1–2 required.

In any [[knowledge-graph]], entities belonging to a class share predicates that all members of that class are expected to carry. When an instance lacks an expected predicate, that absence is a detectable incompleteness signal — not an ambiguous open-world unknown, but a structural flag. Razniewski et al. (2023) describe this as the "obligatory attributes" mechanism: the method involves analyzing "the ratio of instances of the class that have the predicate" and checking "if this ratio changes when we go into subclasses," revealing incompleteness when entities lack expected properties.

Applied directly to SeekVault: every claim-note in the vault belongs to the class `claim-note`, which has the obligatory predicates `source_url`, `source_tier`, and a grounding quote (for Tier 1–2 sources). Any note in `30-notes/` carrying `[unsourced -- needs verification]` or a missing `source_tier` field is, by the Razniewski formalism, an entity with a missing obligatory attribute — a machine-readable gap signal. The vault's own YAML schema functions as the class definition; schema violations are incompleteness markers.

**Source:**
- source_url: https://arxiv.org/abs/2305.05403
- source_author: Simon Razniewski, Hiba Arnaout, Shrestha Ghosh, Fabian Suchanek
- source_date: 2023-05-09 (revised 2023-12-06)
- source_tier: 1
- grounding quote: *"the ratio of instances of the class that have the predicate"* and *"if this ratio changes when we go into subclasses"* (§3.2)

---

## Claim 2: Query-failure patterns are actionable signals of topic-level gaps

**Claim type:** technical mechanism (Tier 1–2 required) + quantitative claim (Tier 1–2 required).

When an information-seeking system receives queries that cluster repeatedly around the same topic and fails to resolve them well, that clustering pattern signals where knowledge is absent. Medhat et al. (2026) demonstrate this by mapping questions put to a conversational AI teaching assistant onto a curriculum prerequisite graph: "conversational AI interaction logs, mapped onto curriculum structure, carry actionable signals about topic-level knowledge gaps." The key diagnostic signal is behavioural rather than per-question: "a student asking many questions about a topic, or asking about it weeks after it was taught, is exhibiting a behavioral signature of difficulty." Question frequency correlated significantly with student self-reported difficulty at ρ = 0.491, p = 0.008 (n = 1,340 questions from 164 students).

Applied to SeekVault: each research batch run is effectively a query session. When the batch agent searches the vault and finds no existing note matching a topic — or when the same lead recurs across multiple Further-leads sections without ever being promoted — that recurrence is a behavioural signature of a gap. A systematic log of unresolved vault searches would constitute the "interaction log" that Medhat et al. show to be gap-diagnostic.

**Quantitative claim:** ρ = 0.491, p = 0.008 — [Tier 1 source; no flag needed]

**Source:**
- source_url: https://arxiv.org/abs/2606.10736
- source_author: Youssef Medhat, Junsoo Park, Ploy Thajchayapong, Ashok K. Goel (Georgia Institute of Technology)
- source_date: 2026-06-09
- source_tier: 1
- grounding quote: *"conversational AI interaction logs, mapped onto curriculum structure, carry actionable signals about topic-level knowledge gaps"*
- grounding quote (signal definition): *"a student asking many questions about a topic, or asking about it weeks after it was taught, is exhibiting a behavioral signature of difficulty"*

---

## Claim 3: LLMs can scan a corpus to surface both explicit and implicit knowledge gaps

**Claim type:** technical mechanism → Tier 1–2 required.

An LLM directed to read a body of text can identify two distinguishable classes of gap. Salem et al. (2025) in GAPMAP define them precisely: an explicit gap is one "directly signaled by high uncertainty lexical cues (e.g., 'unknown,' 'further research is needed')"; an implicit gap is one "not stated directly by the authors of the research papers, but can be inferred from section-level context." The GAPMAP system uses their TABI (Toulmin-Abductive Bucketed Inference) method for the harder implicit case, which structures reasoning into Claim (the implied gap), Grounds (evidence span), Warrant (a one-sentence reasoning link), and Bucket (binary confidence classification). Evaluation across ~1,500 biomedical documents found that LLMs show "robust capability… in identifying both explicit and implicit knowledge gaps."

Applied to SeekVault: an LLM pass over the vault corpus can flag notes that contain explicit uncertainty markers ("I need to check this", `[unsourced]`, `[unverified-quant]`) as well as notes where the *absence* of expected supporting context implies a gap — e.g., a claim-note that references a mechanism without any note explaining that mechanism existing in the vault. The explicit/implicit distinction maps cleanly to SeekVault's own flagging system: `[unsourced]` markers are explicit signals; the implicit signals require inference from the link graph and topic coverage.

**Source:**
- source_url: https://arxiv.org/abs/2510.25055
- source_author: Nourah M. Salem, Elizabeth White, Michael Bada, Lawrence Hunter
- source_date: 2025-10-29
- source_tier: 1
- grounding quote (explicit): *"An instance is explicit when the gap is directly signaled by high uncertainty lexical cues (e.g., 'unknown,' 'further research is needed')"*
- grounding quote (implicit): *"An instance is implicit when the gap is not stated directly by the authors of the research papers, but can be inferred from section-level context"*
- grounding quote (capability): *"highlight the robust capability of LLMs in identifying both explicit and implicit knowledge gaps"*

---

## Claim 4: LLM metacognitive self-report is unreliable as a sole gap-detection mechanism

**Claim type:** technical mechanism → Tier 1–2 required.

An LLM asked to report "what don't I know?" is not a reliable oracle of its own knowledge gaps, because LLM introspective access to internal state is structurally limited. Ji-An et al. (2025) show empirically that "LLMs can monitor only a small subset of their neural activations," and that their metacognitive space operates at "substantially lower dimensionality than the model's complete neural space." Furthermore, LLMs "sometimes articulate their problem-solving strategies but other times seem unable to recognize those strategies that govern their behavior."

This maps onto the broader [[introspection-access-problem]] and is consistent with the faithfulness concerns documented in [[cot-faithfulness-anthropic-biology]]: what an LLM says about its own processing may not correspond to that processing. The implication for SeekVault gap detection: Claude's volunteered uncertainty statements ("I'm not sure about this") are useful signals but are not sufficient as the primary gap-detection mechanism. They must be supplemented by external structural and behavioural methods (Claims 1–3 above).

**Source:**
- source_url: https://arxiv.org/abs/2505.13763
- source_author: Li Ji-An, Hua-Dong Xiong, Robert C. Wilson, Marcelo G. Mattar, Marcus K. Benna
- source_date: 2025-05-19 (revised 2025-10-24)
- source_tier: 1
- grounding quote: *"LLMs can monitor only a small subset of their neural activations"*
- grounding quote (dimensionality): *"operating within a metacognitive space with substantially lower dimensionality than the model's complete neural space"*
- grounding quote (inconsistency): *"sometimes articulate their problem-solving strategies but other times seem unable to recognize those strategies that govern their behavior"*

---

## Further leads

*Pointers for later runs; not worked-up claims.*

- **Orphan-note graph queries in Obsidian** — Azuma's `obsidian-graph-query` plugin (2026-03-03) implements BFS, Tarjan's bridge detection, degree analysis, and orphan detection against `app.metadataCache.resolvedLinks`; needs a Tier 1–2 academic backing for the mechanism before promoting to claim. → https://forum.obsidian.md/t/obsidian-graph-query-let-your-ai-agent-query-your-vaults-knowledge-graph-bfs-shortest-path-bridges-hubs-orphans/111828

- **GRADE gradient-based gap detection in LLMs** — Wang et al. (2026, arXiv 2604.02830) propose detecting whether an LLM has sufficient internal knowledge for a query by analyzing "the cross-layer rank ratio of the gradient to that of the corresponding hidden state subspace"; validated across six benchmarks. More relevant to LLM internals than vault structure, but potentially applicable if SeekVault ever uses an embedded model as an index. → https://arxiv.org/abs/2604.02830

- **Curriculum prerequisite graphs as vault ontology** — Medhat et al. (2026) build a GPT-4-generated prerequisite graph (54 topics, 47 prerequisite relationships) from course materials. The same approach could generate a SeekVault topic ontology against which the vault's actual note coverage is measured — surfacing structural coverage gaps by domain. Same source as Claim 2.

- **InfraNodus topical cluster visualization** — commercial Obsidian plugin using network modularity to flag under-connected topical clusters as knowledge gaps; product documentation only (Tier 3), no mechanism paper. Good lead if a primary source for the modularity-as-gap-signal claim surfaces. → https://infranodus.com/obsidian-plugin

- **Open-world KB completeness via cardinality assertions** — Razniewski et al. 2023 (same as Claim 1) describe a richer completeness toolkit beyond obligatory attributes: cardinality assertions, the No-Change Assumption, text-extraction-based recall. Could yield additional vault-applicable mechanisms on a dedicated run. → https://arxiv.org/abs/2305.05403