capture promoted 2026-07-07

The vault crossed the 50-note embedding threshold on 2026-07-06 — wire the local semantic-retrieval pass (Q3c)

This capture does not resolve the central engineering question (sqlite-vec vs. LanceDB, which local embedding model, exact wiring) — that is an implementation decision for the build itself, not a fact a web source can confirm or deny. What this pass does is tighten the technical grounding for the two open build-plan items named in question-embedding-layer-threshold-crossed — "a local embedding model (Qwen or equivalent)" and "SQLite + sqlite-vec or LanceDB" — against primary sources, and re-verify the peer-precedent claim (Agrici's hybrid BM25+cosine retrieval) directly rather than inheriting the vault's existing secondary paraphrase.

Claim 1 — sqlite-vec is a dependency-free, embeddable vector-search extension for SQLite

Claim type: technical-mechanism → Tier 1–2 required. Source tier: Tier 1 (project's own GitHub repository, direct fetch, 2026-07-07).

"A vector search SQLite extension that runs anywhere!"

"Written in pure C, no dependencies, runs anywhere SQLite runs (Linux/MacOS/Windows, in the browser with WASM, Raspberry Pis, etc.)"

Source: Garcia, A. "sqlite-vec." https://github.com/asg017/sqlite-vec. Confirmed resolving 2026-07-07.

Mechanism, confirmed directly from the repo: vectors (float, int8, or binary) are stored and queried through a vec0 virtual table using ordinary SQL — CREATE, INSERT, and a MATCH ... ORDER BY distance LIMIT k pattern for KNN search. The stable v0.1.0 release date (2024-08-01) comes from the author's own blog post (https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/index.html), a Tier 1 primary-author source, cross-confirmed via independent search results citing the same date. This directly supports the build-plan's "SQLite + sqlite-vec" option named in question-embedding-layer-threshold-crossed: it is additive to an existing SQLite file rather than a new service, which matches that question's explicit constraint ("additive alongside the markdown... NOT a rewrite of seek_research.py").

Claim 2 — LanceDB is an embedded, in-process multimodal vector database built on a columnar storage format

Claim type: technical-mechanism → Tier 1–2 required. Source tier: Tier 1 (project's own GitHub repository, direct fetch, 2026-07-07).

"Developer-friendly OSS embedded retrieval library for multimodal AI."

"Built on top of the Lance columnar format."

Source: LanceDB. "lancedb/lancedb" (GitHub repository). https://github.com/lancedb/lancedb. Confirmed resolving 2026-07-07.

The repository documents support for vector similarity search, full-text search, SQL querying, and hybrid search in combination, plus client SDKs for Python, Node.js, and Rust. "Embedded" here means in-process (no separate server to run), the same operational shape sqlite-vec offers — the two are genuine alternatives for the vault's scale (52+ notes), not different classes of tool. Neither source states a specific performance or cost comparison between the two at small (sub-1000-vector) scale; that comparison is not established by either primary source and would need direct benchmarking, not a citation, if the build wants a number.

Claim 3 — Qwen3-Embedding is a local-deployable embedding model family (0.6B/4B/8B) ranking No.1 on the MTEB multilingual leaderboard

Claim type: quantitative (MTEB score) + technical-mechanism (model sizes, dimensions) → Tier 1–2 required. Source tier: Tier 1 (Qwen team's own project blog, direct fetch, 2026-07-07).

"The 8B size embedding model ranks No.1 in the MTEB multilingual leaderboard (as of June 5, 2025, score 70.58)."

Source: Qwen Team. "Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models." https://qwenlm.github.io/blog/qwen3-embedding/. Confirmed resolving 2026-07-07.

The series ships three sizes — 0.6B, 4B, 8B — with output dimensions of 1024, 2560, and 4096 respectively (Matryoshka-style custom/truncatable dimensions also supported), and documented support for "over 100 languages, including various programming languages." All three sizes are distributed for local deployment via Hugging Face and Ollama (ollama.com/library/qwen3-embedding), which is the deployment path the build-plan's "local embedding model (Qwen or equivalent)" language anticipates. The 0.6B variant is the size most plausibly relevant at the vault's actual scale (~50–100 short markdown notes) given the north star's "mechanical/local-model job" framing, though no source directly benchmarks Qwen3-Embedding-0.6B specifically against a corpus this small — the MTEB score is a general-purpose leaderboard figure, not a vault-scale-specific one, and should not be read as a promise of any particular retrieval quality at n≈50–100 documents.

Claim 4 — Anthropic's Contextual Retrieval technique prepends explanatory context to chunks before embedding and BM25 indexing, and measurably reduces retrieval failure

Claim type: technical-mechanism + quantitative (failure-rate reduction percentages) → Tier 1–2 required. Source tier: Tier 1 (Anthropic engineering; this exact venue is named Tier 1 in 00-meta/sources.md's Depth section).

"Contextual Embeddings reduced the top-20-chunk retrieval failure rate by 35% (5.7% → 3.7%)."

"Combining Contextual Embeddings and Contextual BM25 reduced the top-20-chunk retrieval failure rate by 49% (5.7% → 2.9%)."

"Reranked Contextual Embedding and Contextual BM25 reduced the top-20-chunk retrieval failure rate by 67% (5.7% → 1.9%)."

Source: Anthropic. "Contextual Retrieval in AI Systems." https://www.anthropic.com/engineering/contextual-retrieval. Confirmed resolving 2026-07-07.

Mechanism: each chunk is prepended with chunk-specific explanatory context (generated by an LLM reading the chunk against the full document) before it is embedded and before it enters the BM25 index — addressing the problem of chunks losing surrounding context when split out of a document. This is the technique Agrici's v1.7 release (Claim 5) explicitly credits as its retrieval-pipeline foundation. It is directly relevant to the vault's build-plan because the vault's unit of retrieval (an atomic claim-note) is already closer to Anthropic's "contextualized chunk" than to a raw document fragment — each note already carries a claim-is-the-title framing, which may reduce (not eliminate) the marginal value of adding an LLM-generated context prefix on top.

Claim 5 — Agrici's claude-obsidian v1.7 shipped a three-layer hybrid retrieval pipeline (contextual prefix + BM25 + cosine rerank) and measured +32 percentage points top-1 accuracy over its prior baseline

Claim type: technical-mechanism + quantitative (benchmark figure) → Tier 1–2 required. Source tier: Tier 1 (project's own GitHub release notes, direct fetch, 2026-07-07 — this corroborates and sharpens the existing vault claim in comparison-seek-among-peers-2026-07, which cited the same fact from a secondary framing).

"Obsidian CLI as the default transport, hybrid retrieval (contextual prefix + BM25 + cosine rerank, per Anthropic's Sept 2024 contextual retrieval research), and per-file advisory locking that closes a latent multi-writer corruption hole. Verified at +32pp top-1 accuracy over the v1.6 baseline on a 50-query benchmark."

Source: Agrici, D. "Release v1.9.2: Compound Vault, Methodology Modes, and the Thinking Framework" (recapping the v1.7–v1.9.2 line). https://github.com/AgriciDaniel/claude-obsidian/releases/tag/v1.9.2. Confirmed resolving 2026-07-07. The direct v1.7 release tag page itself was not independently reachable in this pass (GitHub's releases listing did not surface it as a separate page); this quote comes from the v1.9.2 page's own recap of that release, which is still the project's own primary documentation, not a third-party retelling.

Per the source, the pipeline is explicitly tiered by consent and availability: BM25 is "always-on" (no external calls), the contextual-prefix layer is consent-gated behind an --allow-egress flag (because generating prefixes by default sends page bodies to the Anthropic API), and cosine rerank runs against a local Ollama model by default. [unverified-mechanism — needs primary]: the specific claim that the pipeline "degrades gracefully" when any tier is unavailable, and the exact setup-script name (bash bin/setup-retrieve.sh), surfaced via WebSearch's synthesized summary of the v1.7 feature set rather than being independently re-confirmed as a verbatim quote against a directly fetched page in this pass — recorded as directional (the shape of the design), not as a confirmed literal claim. The three-tier, consent-gated design (rather than an always-on API-calling pipeline) is directly relevant precedent for the vault's own build, since the north star's local-model-era framing and the vault's safety spec (nothing web-derived may write to specs unsupervised) would favor the same shape: local-only by default, any external-API step opt-in and separable.

What this pass does not settle

No source compares sqlite-vec against LanceDB head-to-head at the vault's actual scale (order of 50–100 short documents), and no source specifies which Qwen3-Embedding size is the right trade-off point for that scale on the vault's own hardware (the Mac mini referenced elsewhere in the vault's kindred-projects research). Those are implementation decisions the build itself must make and test, not facts obtainable by further search. Likewise, the build-plan's requirement to "gate retrieval on audit_status... so semantic hits inherit the same gate wikilink walks do" (question-embedding-layer-threshold-crossed) is a vault-internal design constraint (operating spec §7) with no external claim to verify — it is not addressed by any source above and does not need to be; it is listed here only to note that this pass does not attempt to source it, by design.

Seek's commentary: The honest shape of this pass is "confirm the parts list, don't pick the parts." Every piece named in the original question's build-plan sketch — sqlite-vec, LanceDB, a local Qwen embedding model — turns out to be real, current, and documented straight from its own maintainers, which is the best-case outcome for a sourcing pass: nothing here needed a [needs primary] flag on the mechanism claims themselves. The one place a flag was warranted (Claim 5's setup-script and graceful-degradation detail) is a small, honestly-marked gap, not a load-bearing one. What stood out most was Agrici's consent-gating shape in Claim 5 — BM25 always-on, contextual-prefix opt-in behind an explicit egress flag, rerank local-by-default — because it is close to free architectural advice for the vault's own build: default to zero external calls, keep any API-dependent enhancement as a clearly labeled, separable upgrade. That is a design opinion, not a finding, which is why it stays down here.

Capture produced 2026-07-07. All five source URLs (github.com/asg017/sqlite-vec, github.com/lancedb/lancedb, qwenlm.github.io/blog/qwen3-embedding, anthropic.com/engineering/contextual-retrieval, github.com/AgriciDaniel/claude-obsidian/releases/tag/v1.9.2) confirmed resolving via WebFetch on 2026-07-07. Not promoted yet — candidate claim-notes: sqlite-vec and LanceDB as alternative local vector stores (Claims 1–2, likely one comparison note), Qwen3-Embedding's local-deployment fit (Claim 3), Anthropic's Contextual Retrieval mechanism and figures (Claim 4, may already warrant its own reusable claim-note given how often the vault's peer-field research cites it secondhand), and a refinement to whatever note eventually formalizes Agrici's hybrid-retrieval precedent (Claim 5, tightening comparison-seek-among-peers-2026-07's existing paraphrase to this direct quote).

· batch run, 2026-07-07 — technical-grounding research for [[question-embedding-layer-threshold-crossed]] (Q3c, RUN-QUEEN-LOOP), ahead of the build the sequencing ruling queues after vault-lint · raw markdown