Evaluators cannot keep source reliability and information credibility independent — they avoid inconsistent pairings and over-weight the source's track record

The Admiralty Code's two-axis design rests on the assumption that a source's reliability and a report's credibility can be judged separately. Empirical work indicates the assumption fails. Kelly et al. (2025), in Judgment and Decision Making, report that "encoders do not tend to assign inconsistent meta-informational attributes such as high source reliability and low information credibility, or vice versa," and conclude that "it is unclear whether information evaluators are capable of treating source reliability and information credibility as fully independent." Evaluators avoid the very cells (reliable-source / low-credibility-report, and the reverse) that a genuinely two-axis scheme should populate.

This is not a new observation. The paper cites Samet (1975), who found intra-individual coding "ambiguous or inconsistent for one-third of the cases." And when judging trustworthiness, evaluators systematically over-weight the source's track record over the content of the specific report — the source axis contaminates the report axis.

The finding indicts collapsing-into-one-number and keeping-two-numbers alike: if trained analysts cannot hold the axes apart, then the vault's single source-tier fusion may be an honest admission rather than a simplification loss. The same reliability-bleeds-into-credibility failure resurfaces in machine retrieval, where reading a document's own text degrades an LLM's judgment of its source's authority. It sits alongside the vault's other structured-analysis correctives, such as ACH's forced disconfirmation, as a documented gap between a tradecraft schema's design and its execution.