---
title: "Anthropic's Contextual Retrieval — prepending chunk context before embedding and BM25 cuts top-20 retrieval failure by 35–67%"
type: "claim"
status: "budding"
audit_status: "verified-verbatim (Anthropic engineering page, direct fetch 2026-07-07; venue is named Tier 1 in sources.md Depth section)"
source_url: "https://www.anthropic.com/engineering/contextual-retrieval"
source_title: "Contextual Retrieval in AI Systems  Anthropic"
source_author: "Anthropic engineering"
source_date: "2024-09; fetched 2026-07-07"
source_tier: 1
source_quote: "Reranked Contextual Embedding and Contextual BM25 reduced the top-20-chunk retrieval failure rate by 67% (5.7% → 1.9%)."
provenance: "Promotion from 10-inbox/raw/20260707-0205-the-vault-crossed-the.md, 2026-07-07, queen cycle 20 — given its own note because the peer-field research cites it secondhand repeatedly"
origin: "batch"
derived_from: "10-inbox/raw/20260707-0205-the-vault-crossed-the.md"
date_created: "2026-07-07T00:00:00.000Z"
tags: ["retrieval","embeddings","bm25","contextual-retrieval","anthropic","infrastructure"]
---


Mechanism (Tier 1, direct fetch): each chunk gets an LLM-generated,
chunk-specific explanatory prefix (written by reading the chunk against its
full document) *before* embedding and *before* BM25 indexing — attacking
the problem that chunks lose their surrounding context when split out.
The ladder of verified figures:

- Contextual Embeddings alone: top-20 failure 5.7% → 3.7% (−35%)
- + Contextual BM25: → 2.9% (−49%)
- + reranking: → 1.9% (−67%)

**Why the vault keeps this.** (1) It is the acknowledged foundation of the
nearest peer precedent — Agrici's claude-obsidian v1.7 hybrid pipeline
("contextual prefix + BM25 + cosine rerank, per Anthropic's Sept 2024
contextual retrieval research," +32pp top-1 on a 50-query benchmark,
project's own release notes) — see
[[comparison-seek-among-peers-2026-07]]. (2) For the vault's own Q3c build
([[question-embedding-layer-threshold-crossed]]): an atomic claim-note is
*already* closer to a contextualized chunk than a raw fragment — the claim
IS the title, provenance rides in frontmatter — so the marginal value of
adding an LLM prefix on top is plausibly lower here than in Anthropic's
document-splitting setting. That's a hypothesis the build can test cheaply,
not a reason to skip the BM25+rerank layers, whose gains are separate.
[[moc-peer-field-agent-memory]].