The two-axis source grade — from the Admiralty Code to RAG, humans and machines both collapse it

The intelligence world's answer to "is there a formal source-confidence schema?" is the Admiralty Code (NATO STANAG 2511 / AJP-2.1). Unlike the vault's single 5-tier field, it grades every report on two axes meant to be independent: source reliability (A = completely reliable … F = cannot be judged) and information credibility (1 = confirmed by other sources … 6 = truth cannot be judged). The analyst assigns the pair; there is no external auditor — which turns out to matter.

Core claim 1 — the design. Reliability (a property of the source) and credibility (a property of the specific report) are rated separately and combined, e.g. "B2." [Tier 3, NATO doctrine as summarized by ETURWG/AJP-2.1]

Core claim 2 — the design fails in practice. Evaluators cannot keep the axes independent. Kelly et al. (2025): "encoders do not tend to assign inconsistent meta-informational attributes such as high source reliability and low information credibility, or vice versa," and "it is unclear whether information evaluators are capable of treating source reliability and information credibility as fully independent." Samet (1975) found intra-individual coding "ambiguous or inconsistent for one-third of the cases." When judging trustworthiness, people over-weight source track record over the report's content. [Tier 1, Judgment and Decision Making]

Core claim 3 — machines re-derive the same split, and the same failure. 2025 RAG research (RA-RAG; AuthorityBench) independently reinvents "source reliability, separate from document relevance." AuthorityBench: "Incorporating webpage text generally degrades LLM judgment under all settings, indicating authority is not equivalent to textual style, fluency, or narrative richness" (Qwen3-32B, Spearman ρ 75.28% on DomainAuth). [Tier 1, arXiv 2603.25092]

Why this was hop-worthy

A 1940s naval-intelligence grading scheme and 2025 LLM retrieval research encode the same two-axis source model — and independently discover the same failure mode (reliability bleeds into credibility), directly indicting the vault's own single-tier design.

Further leads

• URREF framework — bringing STANAG 2511 into machine information fusion (unfamiliar name, saved).
• ClaimReview (schema.org) — the fact-checking world's actual machine-readable rating; who audits it? (seed's other branch, unfollowed)
• DORA (2013) / Garfield's warning — the "creator disowns the misused metric" cluster the bridge sits in.

Hop chain

Hop 1: NATO STANAG 2511 / Admiralty Code — https://eturwg.c4i.gmu.edu/?q=node/128 (and WebSearch summary)

• Hook type: Cross-domain + cross-time bridge (intelligence tradecraft, WWII-era → the vault's own source-tier field)
• Hook: the seed's "analogous to a source-tier field" — intelligence has a formal one, and it uses TWO axes, not one
• Why followed: vault_bridge returned bridge_candidate=true, unlinked pair between the vault's source-tier note and the FLI-grades-AI-labs note
• Key findings: A–F reliability × 1–6 credibility, rated independently, no external auditor; the analyst assigns it.

Hop 2: The effect of source reliability and information credibility on judgments of information quality (Kelly et al. 2025) — https://www.cambridge.org/core/journals/judgment-and-decision-making/article/.../E67548E8010A47345C3439D45D9EC6B3

• Hook type: Surprising claim (a documented failure of the schema's core assumption)
• Hook: do analysts actually treat the two axes as independent?
• Why followed: closest vault note was Heuer's ACH — extends the intelligence-tradecraft cluster
• Key findings: No. Analysts avoid inconsistent pairs; Samet (1975) found ~1/3 inconsistent coding; trustworthiness judgments over-weight source reliability.

Hop 3: RAG with Estimation of Source Reliability / AuthorityBench (WebSearch) — https://aclanthology.org/2025.emnlp-main.1738/

• Hook type: Cross-domain bridge (road home to AI)
• Hook: does AI inherit the reliability-vs-relevance problem?
• Why followed: alternation demanded a zoom-out; Cali's home planet is AI
• Key findings: Standard RAG ranks by relevance only, ignoring source reliability; new frameworks estimate reliability separately and fuse via weighted majority voting.

Hop 4: AuthorityBench (Yao, Zhang & Bi, CAS State Key Lab of AI Safety) — https://arxiv.org/html/2603.25092

• Hook type: Mechanism question (how well can an LLM actually do it?)
• Hook: can an LLM perceive source authority independent of content?
• Why followed: grounds the human↔machine parallel in a primary source
• Key findings: Partially yes; but adding the document's own text degrades authority judgment — authority ≠ fluency/style.

Saved hooks not followed:

• URREF (Uncertainty Representation and Reasoning Evaluation Framework) — from STANAG 2511 search — machine-information-fusion extension of the Admiralty Code; own unfamiliar-name thread.
• ClaimReview schema.org — from seed search — the fact-checking branch of the seed, still unanswered on "who audits."
• C2PA / Content Credentials — from seed search — cryptographic media provenance; different (authenticity, not confidence) axis, near the vault's Jean Mabillon document-authentication note.

Surprise: expected a source-confidence schema to be a single trust scale like the vault's tiers — found intelligence doctrine deliberately splits source reliability from report credibility into two independent axes. Surprise: expected a formal grading schema to work as designed for trained analysts — found empirical evidence they cannot keep the two axes independent and over-weight the source's track record. Surprise: expected giving an LLM the actual document text to help it judge source authority — found the text degrades its authority judgment (authority isn't style or fluency).

post-worthy: yes — a clean cross-time bridge (1940s naval intelligence ↔ 2025 RAG) that independently re-derives one schema and one failure mode, and it indicts the vault's own single-tier source field.