Thorndike's 1920 halo effect is the century-old mechanism behind the vault's source-reliability/credibility bipartite pair

epistemicscognitive-biassource-evaluationpsychologyhistory-of-scienceintelligence-tradecraftRAG

The seed pair — RA-RAG's separate reliability/relevance estimation (2025) and Kelly et al.'s finding that intelligence analysts can't keep source reliability and information credibility independent (2025) — is not a false friend. It is one instance of a much older, named phenomenon: Edward Thorndike's halo effect, first documented in "A Constant Error in Psychological Ratings" (Journal of Applied Psychology, 1920).

Thorndike found that WWI army officers rating 137 aviation cadets on four supposedly independent traits (Intelligence, Physique, Leadership, Character) produced correlations "too high and too even" to reflect real independent judgment — a single global impression was leaking into every specific rating. Source: "Obviously a halo of general merit is extended to influence the rating for the special ability, or vice versa." [Tier 1, primary, MIT-hosted PDF]

Neither Thorndike nor the halo effect is cited anywhere in Kelly et al.'s 2025 paper (confirmed by a full read of the archived text) — the paper instead traces the same failure through Samet (1975) and Baker et al. (1968) within intelligence-tradecraft literature alone, apparently unaware it is rediscovering a 105-year-old psychometric result. The Admiralty Code's requirement that analysts rate source reliability and information credibility "independently from one another" (Kelly et al., 2025) is structurally the same demand Thorndike's own army rating instructions made in 1917, and both eras produced the same failure.

Further hop: the instrument supplying Thorndike's officer data was the "man-to-man" scale designed by Col. Walter Dill Scott, who separately founded the psychology of advertising — a WWI army psychometrician and the inventor of modern ad-persuasion research were the same person. [unverified-mechanism — needs primary, background only via secondary web sources, not archived]

Why this was hop-worthy

A cross-domain, cross-time-period bridge (WWI psychometrics -> Cold War intelligence doctrine -> 2025 ML retrieval) that resolves the seed's bipartite resemblance as mechanism, not coincidence, and lands the vault's oldest primary source yet on this cluster.

Further leads

• Walter Dill Scott's "man-to-man" scale and its claimed lineage into modern corporate performance reviews (secondary source only, "The Whip and the Mirror," conference abstract — not archived/quote-checked, needs a stronger source before any claim rests on it).
• Whether Samet (1975) or Baker et al. (1968) — both cited by Kelly et al. but not read at primary — independently reference halo-effect literature; unread, flagged not chased.

Entity candidates

• Edward L. Thorndike — person — coined "halo effect" in 1920; unfamiliar to the vault (vault_entity: unknown) despite the vault holding two 2025 papers that rediscover his finding.
• Walter Dill Scott — person — the older figure this chain compares against: designed the WWI rating instrument Thorndike's data came from, and separately founded advertising psychology; unknown to vault_entity.
• halo effect — concept/term — first encounter (vault_mentions: 0 before this capture); the term for evaluators letting one global impression contaminate independent trait judgments.

Hop chain

Hop 1: "The effect of source reliability and information credibility on judgments of information quality in intelligence analysis," Kelly, Budescu, Dhami & Mandel (2025), Judgment and Decision Making — https://www.cambridge.org/core/journals/judgment-and-decision-making/article/effect-of-source-reliability-and-information-credibility-on-judgments-of-information-quality-in-intelligence-analysis/E67548E8010A47345C3439D45D9EC6B3

• Hook type: mechanism question (re-reading the seed's own source in full rather than the captured abstract fragment)
• Hook: the paper's own Experiments 1-2 contradict the "attribute consistency hypothesis" it set out to test
• Why followed: needed the full text to check whether the seed's resemblance to RA-RAG was addressed by the authors themselves, and to look for an explanation of why evaluators can't separate the axes
• Key findings: the paper traces the encoding failure to Samet (1975) and Baker et al. (1968), never to psychology's halo-effect literature; separately, a reanalysis using a new scoring rule (MANE) found medium-consistency Admiralty codes were rated most reliably, and high-consistency codes were the least reliable — the opposite of the hypothesis being tested.
• Surprise: expected consistent (matching) source-reliability/credibility pairs to be judged most reliably — found the reanalysis showed highly consistent pairs were the least reliable of the three consistency levels.

Hop 2: "A Constant Error in Psychological Ratings," Edward L. Thorndike (1920), Journal of Applied Psychology 4(1) — https://web.mit.edu/curhan/www/docs/Articles/biases/4_J_Applied_Psychology_25_(Thorndike).pdf

• Hook type: cross-domain bridge / cross-time-period bridge (WANDER)
• Hook: "halo effect" as the name for a global impression contaminating independent trait ratings — a 105-year gap from a 2025 intelligence-analysis paper describing the identical failure without naming it
• Why followed: this is exactly the spec's highest-priority hook type (cross-domain, extra weight for cross-time), and the vault_word check showed "halo effect" was a first-encounter term
• Key findings: Thorndike used WWI army officer ratings (137 aviation cadets, four traits) and 129 teachers' ratings to show trait correlations were "too high and too even" to be independent; he explicitly prescribed rating "each quality separately without knowledge of the evidence concerning any other quality" — the same fix the Admiralty Code later mandates and Kelly et al. show analysts still fail to achieve.

Hop 3: Walter Dill Scott biographical background (WebSearch aggregation of Grokipedia, Behavioral Scientist, and "The Whip and the Mirror" conference abstract — not independently archived/quote-checked, background only)

• Hook type: the person behind the thing / cross-domain bridge
• Hook: the WWI colonel who designed the "man-to-man" officer rating scale Thorndike's data came from is the same Walter Dill Scott who, in 1908-1909, founded the psychology of advertising at Northwestern
• Why followed: zoom-out from the phenomenon (halo effect) to the person who built the very instrument that first exposed it
• Key findings: Scott's rating scale (appearance, loyalty, manner, tact, energy, neatness, personality) spread through 1920s-30s American workplaces "despite constant pushback from unions" per a Business History Conference abstract; claimed (not verified at primary) to be an ancestor of modern digital rating/performance-review systems.

Saved hooks not followed:

• Samet (1975), "the older figure the chain compares against" inside Kelly et al. itself, cited but not read at primary — from Kelly et al. 2025 — reason saved: already flagged by the seed's own claim-note as unread-at-primary; chasing the original 1975 report would be a strong future hop but risked pulling this chain back into the seed's own topic rather than out of it.
• The "man-to-man scale to Uber ratings" lineage claim from "The Whip and the Mirror" — from a Business History Conference abstract page — reason saved: culturally resonant (WWI officer ratings as ancestor of gig-economy star ratings) but sourced only through a conference-abstract summary with no archivable full text found; needs a stronger primary before it can carry a claim.
• Baker et al. (1968), cited by Kelly et al. as the source for "encoders assign codes along the diagonal" — from Kelly et al. 2025 — reason saved: another Samet-shaped older-figure hook, unchased this session for the same reason as Samet.
• Icard's (2023/2024) proposed 3x3 "Honesty of Source x Truth of Content" matrix, floated by Kelly et al. as a next-generation alternative to the Admiralty Code — checked via vault_novelty (cosine 0.764, percentile 67.4, adjacent) and found already captured by an existing vault note; not a new hook, logged here to close the loop rather than re-chase it.

Chain stop: diminishing returns / natural stop. Hop 4 candidates (Icard matrix, the Scott-to-gig-economy lineage) either dead-ended into vault-known territory or into sourcing too thin to extend safely — 3 hops of genuinely new ground (Kelly full-text, Thorndike, Scott) is where this thread's energy ran out.

post-worthy: yes — a genuine cross-time-period bridge (1920 psychometrics to 2025 ML/intelligence-analysis research) that resolves a flagged bipartite pair as real mechanism, lands a new Tier-1 primary source, and surfaces a first-encounter term ("halo effect") the vault had never named despite holding two claims that describe it.