---
title: "The vault crossed the 50-note embedding threshold on 2026-07-06 — wire the local semantic-retrieval pass (Q3c)"
type: "question"
status: "open"
date_raised: "2026-07-06T00:00:00.000Z"
tags: ["embeddings","retrieval","north-star","threshold","local-model","infrastructure"]
---


Per the north star and RUN-QUEEN-LOOP Q3(c): the embedding layer wires at
50–100 notes, "never before — sequencing is the point." As of queen cycle 9,
`30-notes/` holds 52 notes (48 claim-notes + 4 myth-notes). The threshold is crossed; the sequencing
condition is met.

This is logged as a question, not built this cycle, for two honest reasons:
(1) it is a Qwen/mechanical job per the north star (local-model era), and
this session pivots to website work next — wiring it now would be a new
workstream started at a stop boundary; (2) the retrieve-test noise this
marathon has been logging (grep hits climbing to 6+ notes per query with no
ranking) is exactly the signal the embedding layer exists to fix, so the
need is demonstrated, not speculative.

## What building it needs
- A local embedding model (Qwen or equivalent) over `30-notes/` + `40-mocs/`
  + `40-entities/`, additive alongside the markdown (SQLite + sqlite-vec or
  LanceDB per the 2026-07-01 handoff), NOT a rewrite of seek_research.py.
- The `id`/`derived_from`/`connects_to` fields already on notes are the join
  scaffolding.
- Gate retrieval on `audit_status` (operating spec §7) — the embedding index
  must carry the trust field so semantic hits inherit the same gate wikilink
  walks do.

## For Cali
This is the north star's "24/7 volume sprint pays off" moment — the vault hit
the threshold that makes semantic retrieval worth wiring. It is a clean
next-workstream candidate for after the website pivot. Flagged, not started.

## Sequencing ruling (Cali r.3, 2026-07-06)
Priority RAISED (peer survey confirmed peers ship retrieval layers —
Agrici's hybrid BM25+cosine, Ghelbur's scored evals), but sequenced SECOND:
the first vault-lint runs before the embedding pass, because an embedding
index over unlinted notes serves unreconciled pairs faster (the north
star's "confident wrong answer," accelerated). Lint findings feed
[[question-consolidation-pass-vs-revisit-protocol]]; this builds immediately
after.

## Parts list verified at primary (2026-07-07, cycle 20 — capture 20260707-0205)

The lint precondition has since been met (cycle 12); the build is next in
sequence, and its sketched parts all check out against their own
maintainers' documentation:

- **sqlite-vec** — pure-C, dependency-free SQLite extension; `vec0` virtual
  table, plain-SQL KNN (`MATCH … ORDER BY distance LIMIT k`); additive to
  an existing SQLite file, exactly matching this question's
  "additive alongside the markdown" constraint. Stable v0.1.0 2024-08-01.
- **LanceDB** — embedded/in-process alternative (Lance columnar format);
  vector + full-text + hybrid search, Python/Node/Rust. Same operational
  class as sqlite-vec; no source benchmarks the two at sub-1000-vector
  scale — if the build wants a number, it must measure, not cite.
- **Qwen3-Embedding** — 0.6B/4B/8B, dims 1024/2560/4096 (Matryoshka
  truncation supported), Ollama-distributed; 8B ranks No.1 on MTEB
  multilingual (70.58, 2025-06-05). The 0.6B is the plausible fit at
  vault scale; MTEB says nothing about n≈100 corpora — test locally.
- **Retrieval-quality precedent** — Anthropic's contextual-retrieval
  figures and the Agrici v1.7 pipeline built on them now have a citable
  home: [[claim-contextual-retrieval-failure-rate-figures]]. Design shape
  worth copying from Agrici: BM25 always-on and local; any API-calling
  enhancement consent-gated behind an explicit egress flag — congruent
  with the safety spec's no-unsupervised-egress posture.
- Unchanged internal constraint: the index must carry `audit_status` so
  semantic hits inherit the wikilink-walk trust gate.