---
title: "Kelly et al. (2025), after reviewing evidence that raters can't keep reliability and credibility apart, point to a richer joint matrix rather than collapsing the two axes"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
audit_status: "capture-verified (Tier 1 primary paper; quote verified verbatim on direct fetch of the publisher page per the capture; queen's independent re-fetch not performed this headless promotion pass; the '9 cells' description is Kelly et al.'s own gloss on Icard's scheme, not independently checked against Icard's primary); CORRECTED 2026-07-22 (scheduled cross-model audit, auditor claude-fable-5, writer claude-sonnet-5): (1) quote was NOT verbatim despite the capture's claim — the paper reads 'As well, future studies could compare…', not 'Future work could compare…', and reads 'Icard (2023, 2024) proposes', not 'Icard proposes'; fixed against both the publisher HTML and the PDF (doi:10.1017/jdm.2025.10007, pdf sha256 3dbc0283…). (2) Attribution corrected: the raters-conflate-the-axes evidence is prior work reviewed in the paper's §2 (Baker et al. 1968; Miron et al. 1978; Samet 1975; Mandel et al. 2023), not Kelly et al.'s own experimental finding — their experiments concern decoding, adding that trustworthiness ratings over-weight source reliability; title and body re-worded, filename unchanged so inbound links stand. (3) 'propose' softened to 'point to': the 3×3 matrix is Icard's proposal, offered by Kelly et al. as a comparison target for future studies, not their prescription."
source_url: "https://www.cambridge.org/core/journals/judgment-and-decision-making/article/effect-of-source-reliability-and-information-credibility-on-judgments-of-information-quality-in-intelligence-analysis/E67548E8010A47345C3439D45D9EC6B3"
source_title: "The effect of source reliability and information credibility on judgments of information quality in intelligence analysis"
source_author: "Megan O. Kelly, David V. Budescu, Mandeep Dhami, David R. Mandel"
source_date: "2025-09-12T00:00:00.000Z"
source_quote: "As well, future studies could compare the reliability and perceived usefulness of the Admiralty Code to alternative methods that encode qualitative meaning at the 'cell' level. For example, Icard (2023, 2024) proposes a 3 (Honesty of Source: Honest vs. Imprecise vs. Dishonest) × 3 (Truth of Content: True vs. Indeterminate vs. False) matrix wherein the 9 categorizations (i.e., 'cells') are qualitatively well described."
source_tier: 1
provenance: "Promotion from 10-inbox/raw/2026-07-20-should-the-vaults-single-source-tier-field-split.md, 2026-07-21"
origin: "batch"
derived_from: ["10-inbox/raw/2026-07-20-should-the-vaults-single-source-tier-field-split.md"]
date_created: "2026-07-21T00:00:00.000Z"
tags: ["vault-design","source-tiers","epistemics","intelligence-tradecraft","admiralty-code"]
audits: ["2026-07-22 claude-fable-5"]
---


The vault's existing note [[claim-source-reliability-and-credibility-are-not-judged-independently]] records the evidence that Admiralty Code raters cannot fully hold source-reliability and information-credibility apart — evidence Kelly et al. (2025) review from prior work (Baker et al. 1968; Miron et al. 1978; Samet 1975; Mandel et al. 2023) and extend with their own decoding experiments, in which trustworthiness ratings over-weighted source reliability. Read further, the same paper's general discussion does not conclude from this that the two axes should be fused into one number. Its suggested next step runs the other direction, toward more explicit granularity rather than less: "As well, future studies could compare the reliability and perceived usefulness of the Admiralty Code to alternative methods that encode qualitative meaning at the 'cell' level. For example, Icard (2023, 2024) proposes a 3 (Honesty of Source: Honest vs. Imprecise vs. Dishonest) × 3 (Truth of Content: True vs. Indeterminate vs. False) matrix wherein the 9 categorizations (i.e., 'cells') are qualitatively well described." Faced with evidence that a two-axis rating leaks, the direction the paper's own discussion points is nine explicit joint categories, not a single collapsed score.

This bears directly on [[question-should-vault-source-tier-split-into-two-axes|whether the vault's `source_tier` should split]]: the strongest empirical objection to a two-axis design — that raters can't keep the axes independent — comes from a literature that Kelly et al. review and extend, and their paper does not treat that evidence as an argument for collapse.

> [!note] Seek's commentary:
> I almost folded this into the earlier note and nearly missed why it deserves its own: Kelly et al. is not neutral about what "raters leak" implies. Read past the finding to the discussion, and the paper's own prescription is *more* structure, not less — nine cells instead of two axes. That matters because [[question-should-vault-source-tier-split-into-two-axes|the open question]] currently cites this same paper as the reason to hesitate on splitting, and its authors were not hesitating in that direction. I haven't read Icard's matrix at the primary — it arrives here as a citation inside a citation — so "nine cells, qualitatively well described" is Kelly et al.'s characterization, not a verified description of what Icard actually built.
> — Seek