In Irwin & Mandel's (2023) N=41 expert-analyst experiment, raising stated verbal confidence significantly increased analysts' inferred numeric probability, not just their interval's margin of error

intelligence-tradecraftepistemicsforecastingindependenceprobability-confidencedavid-r-mandel

Daniel Irwin and David R. Mandel gave a sample of 41 professional Canadian intelligence analysts (21 recruited in person, 20 via a remote link) a fixed hypothetical assessment using the term "likely," then asked for 95%-certainty probability ranges under three stated-confidence conditions — low, moderate, high. If confidence were a genuinely independent quantity expressing only margin of error, the inferred range's midpoint should stay constant across conditions while only its width varies. Instead: "Had experts treated probability and confidence as independent constructs, we would expect to observe invariant midpoint interpretations, yet midpoint interpretations substantially increased with each increase in confidence level" (F(1, 39) = 90.62, p < .001, partial η² = .699; all pairwise comparisons significant). The pattern replicated at larger scale in non-expert samples (n = 440 and n = 624), and in the second sample confidence level had a stronger effect on inferred probability than the stated probability term itself (F[2,621] = 38.43 vs. F[1,621] = 33.83, both p < .001, plus a significant interaction). The paper states the conclusion directly: "Critically, we found that intelligence consumers do not treat confidence and probability as independent constructs."

This is the empirical leg Kelly et al. (2025) cite for their claim that "analysts and nonexperts alike do treat probability and confidence as related constructs," and it resolves question-verify-irwin-mandel-2023-probability-confidence-cue. It joins the vault's recurring pattern of prescribed-independent axes that leak as the sharpest empirical case: the leak runs in the direction of confidence contaminating probability, not the reverse (see claim-irwin-mandel-2023-probability-location-shifts-inferred-confidence for the reverse direction).