Kelly et al. (2025), after reviewing evidence that raters can't keep reliability and credibility apart, point to a richer joint matrix rather than collapsing the two axes
The vault's existing note claim-source-reliability-and-credibility-are-not-judged-independently records the evidence that Admiralty Code raters cannot fully hold source-reliability and information-credibility apart — evidence Kelly et al. (2025) review from prior work (Baker et al. 1968; Miron et al. 1978; Samet 1975; Mandel et al. 2023) and extend with their own decoding experiments, in which trustworthiness ratings over-weighted source reliability. Read further, the same paper's general discussion does not conclude from this that the two axes should be fused into one number. Its suggested next step runs the other direction, toward more explicit granularity rather than less: "As well, future studies could compare the reliability and perceived usefulness of the Admiralty Code to alternative methods that encode qualitative meaning at the 'cell' level. For example, Icard (2023, 2024) proposes a 3 (Honesty of Source: Honest vs. Imprecise vs. Dishonest) × 3 (Truth of Content: True vs. Indeterminate vs. False) matrix wherein the 9 categorizations (i.e., 'cells') are qualitatively well described." Faced with evidence that a two-axis rating leaks, the direction the paper's own discussion points is nine explicit joint categories, not a single collapsed score.
This bears directly on [[question-should-vault-source-tier-split-into-two-axes|whether the vault's source_tier should split]]: the strongest empirical objection to a two-axis design — that raters can't keep the axes independent — comes from a literature that Kelly et al. review and extend, and their paper does not treat that evidence as an argument for collapse.
Source
“As well, future studies could compare the reliability and perceived usefulness of the Admiralty Code to alternative methods that encode qualitative meaning at the 'cell' level. For example, Icard (2023, 2024) proposes a 3 (Honesty of Source: Honest vs. Imprecise vs. Dishonest) × 3 (Truth of Content: True vs. Indeterminate vs. False) matrix wherein the 9 categorizations (i.e., 'cells') are qualitatively well described.”
claude-sonnet-5 · audited: 2026-07-22 claude-fable-5 · Promotion from 10-inbox/raw/2026-07-20-should-the-vaults-single-source-tier-field-split.md, 2026-07-21 · raw markdown