--- title: "Completeness mechanisms in Razniewski et al. 2023 (arXiv:2305.05403) and applicability to SeekVault gap-detection" type: "capture" status: "promoted" date_promoted: "2026-07-07T00:00:00.000Z" promoted_to: ["30-notes/claim-kb-completeness-toolkit-cardinality-nca-recall.md"] not_promoted: ["69% F1 LM-KBC figure — [unverified-quant] honored, named in the note as excluded","Per-mechanism vault-implementation commentary — carried in the note's analogue lines, full detail stays in capture"] origin: "batch" date_created: "2026-07-01T00:00:00.000Z" provenance: "batch run 2026-07-01, researched via ar5iv HTML render (https://arxiv.org/html/2305.05403) and arXiv abstract page" tags: ["knowledge-bases","completeness","gap-detection","cardinality","recall","LLM-limitations","SeekVault"] source_url: "https://arxiv.org/abs/2305.05403" source_author: "Simon Razniewski, Hiba Arnaout, Shrestha Ghosh, Fabian Suchanek" source_date: "2023-05-09 (submitted); revised 2023-12-06" source_tier: "Tier 1 (arXiv cs.AI primary survey paper)" harvested_from: "2026-06-28-how-should-seekvault-detect-gaps-in-its-own-knowledge" filename_note: "Canonical prompt filename exceeded OS 255-byte limit; this shortened slug is the actual file." --- **Paper:** "Completeness, Recall, and Negation in Open-World Knowledge Bases: A Survey" **Source:** https://arxiv.org/abs/2305.05403 — Tier 1 (arXiv cs.AI, 42-page survey, submitted 2023-05-09, revised 2023-12-06) **HTML render verified:** https://arxiv.org/html/2305.05403 resolves and is readable. --- ## Claim: Cardinality assertions are the formal primitive for KB completeness measurement **Claim type:** specific technical mechanism → Tier 1–2 required → **clears floor** (Tier 1). Section 2.4 of the survey defines the core object: > "A cardinality assertion is an assertion of the form |σ(Kⁱ)|=n, where σ is a selection condition, and n is a natural number. It specifies the number of tuples in Kⁱ that satisfy a certain property." Recall for any selection condition σ is then the ratio |σ(K)| / n — matching tuples actually present divided by the asserted total. Sources of the expected cardinality n include: (a) KB predicates that encode counts directly (e.g. `numberOfChildren`, `numberOfAwards`), (b) text extraction that mentions a specific count, and (c) inference from upstream completeness assertions. The mechanism is additive: once n is known for a subject-predicate pair, any divergence of the KB's actual count from n immediately quantifies the gap. **SeekVault applicability (direct):** If SeekVault maintains an expected-cardinality table per entity class or topic (e.g., "a person note should have claim-notes for birth, affiliation, and primary work"), recall = |actual claim-notes present| / n gives a numeric gap signal computable on every sync. No model call is required — only a structured lookup against the vault index. --- ## Claim: The No-Change Assumption (NCA) operationalises temporal convergence as a lightweight completeness signal **Claim type:** specific technical mechanism → Tier 1–2 required → **clears floor** (Tier 1). Section 3.2 ("Weak signals") states: > "The *No-Change Assumption* says that if the number of objects has not changed over recent editions of the KB, then it has 'converged' to the true number, and no more objects are missing." The NCA is classified among "weak signals" for predictive recall assessment — weaker than cardinality assertions derived from external text, but requiring no external reference: it only needs the KB's own version history. It operates at the statement level (per predicate, per subject) rather than globally, so it can declare local convergence while other parts of the KB are still growing. **SeekVault applicability (direct):** SeekVault can track, across successive ingestion runs or daily snapshots, the claim-note count for each topic node. A topic whose count has not grown over N consecutive runs satisfies the NCA and can be flagged *locally convergent* — a weak but zero-cost completeness signal. Topics still growing remain in an open attention state. Implementation requires only a diff of note-counts between snapshots; no model inference is involved. --- ## Claim: Text-extraction-based recall compares what a source document explicitly mentions against what the KB contains **Claim type:** specific technical mechanism → Tier 1–2 required → **clears floor** (Tier 1). Section 3.2 describes the extraction approach: > "If we use a simple open information extraction approach, we can extract the objects that the sentence mentions for the predicate, and compare them to the objects that the complete KB contains. If the sentence mentions all objects, we consider it *complete*." The method exploits Gricean *implicatures*: when a text enumerates a closed list, it implicates no further members. A sentence mentioning n objects for a predicate while the KB holds only k yields a text-grounded recall estimate k/n for that subject-predicate pair. Section 3.2 also notes the complementary direction: "Textual information can also be used to spot incomplete areas of the KB" by observing distributional patterns (e.g., "most winners hold academic degrees, and so flag entities without an alma mater as likely incomplete"). **SeekVault applicability (direct):** During document ingestion, an extraction pass can enumerate all entities, relationships, and claims explicitly mentioned in the source; those not yet present as claim-notes surface as candidate gaps. The comparison is also usable in reverse — vault entries that never appear in any ingested source may indicate orphaned or fabricated claims worth auditing. --- ## Claim: LLMs can extract cardinality information from reference text but exhibit principled limitations on self-aggregation; best 2023 benchmark performance was 69% F1 **Claim type:** specific technical mechanism + quantitative figure → Tier 1–2 required → **clears floor for mechanism** (Tier 1); **quantitative figure (69%)** is cited within the Tier 1 paper from the LM-KBC 2023 shared task — marked `[unverified-quant — needs primary]` pending direct confirmation from LM-KBC 2023 proceedings. Section 4.2 ("Automatic extraction of cardinalities") addresses LLMs: > "The answer to [whether LLMs can extract cardinality assertions from text] appears a confident yes." But for self-aggregation (producing counts from parametric memory without a reference text): > "once the LLMs would need to aggregate themselves, they fall back to common numbers in the relation of interest." The survey additionally observes: > "It appears that Transformer-based architectures exhibit principled limitations, that make correctly solving count tasks over text difficult." On the LM-KBC 2023 benchmark, "even the best-performing system, that relied on GPT-4, achieved only 69% F1-score" on cardinality prediction tasks. `[unverified-quant — needs primary: confirm at LM-KBC 2023 shared task proceedings]` **SeekVault applicability (constrained):** This finding determines how [[claim-ai-inference-means-running-a-model]] should be used within SeekVault's gap-detection pipeline. LLM calls are appropriate for *extracting* cardinality statements that appear explicitly in ingested text (mechanism one above), but should not be used to *generate* expected cardinalities from parametric memory — the count produced will regress toward base-rate values rather than entity-specific truth. Structured cardinality tables and text-grounded extraction are the safer primitives; LLM calls serve as an extraction layer over reference text, not as an oracle. --- ## Further leads - **Partial Completeness Assumption (PCA):** "If a subject has at least one object for a given predicate, then there are no other objects in the real world beyond those that are in the KB" (§2.3) — a weaker, predicate-level closed-world default that could seed a SeekVault heuristic with no external data required. Tier 1: arXiv:2305.05403. - **Obligatory attributes:** "An obligatory attribute of a given class is a predicate that all instances of the class must have in the real world" (§3.2) — auto-detectable from predicate distribution across entity subclasses; could drive SeekVault "missing field" alerts per note type (e.g., every paper note must have an author and date). Tier 1: arXiv:2305.05403. - **Mark & Recapture estimation (Table 2, §3.3):** statistical technique for estimating total entity population size from overlapping incomplete samples — potentially applicable if SeekVault has multiple ingestion sources with partial overlap (e.g., two reading lists that each partially cover a domain). Tier 1: arXiv:2305.05403. - **LM-KBC 2023 shared task:** primary proceedings needed to verify the 69% F1 figure cited in §4.2. Search: "LM-KBC 2023 shared task knowledge base construction proceedings." - **Suchanek & Weikum prior YAGO completeness work:** Fabian Suchanek (co-author) has earlier formal treatments of cardinality assertions in the YAGO KB lineage; these may give deeper formal grounding for the cardinality assertion framework above.