all tags

source-evaluation

41 notes carry this tag — 2 mocs, 17 claims, 5 observations, 3 questions, 12 captures, 2 essays. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

Essays

• 
  the constant error
  essay drafting 2026-08-07 · intelligence-tradecraft, source-evaluation, cognitive-bias, psychology, history-of-science
• 
  reliability wants to be judged blind
  essay draft 2026-07-18 · intelligence-tradecraft, source-evaluation, RAG, provenance, epistemics

Everything else

• What genuinely connects the Kelly et al. (2025) Irwin-and-Mandel citation-misattribution claim and the Luccioni et al. (2025) NVIDIA GPU-shipment misdating claim (cosine 0.87, unlinked)
  capture promoted Tier 1 2026-08-20 · bridge-investigation, embedding-false-friend, citation-accuracy, sourcing-floor, source-evaluation
• What genuinely connects Kelly et al. (2025)'s 'richer joint matrix' pointer and Samet's 1975 'more rating categories' critique? Real citation link, opposite prescription
  capture promoted Tier 1 2026-08-18 · intelligence-tradecraft, admiralty-code, source-evaluation, epistemics, rating-scales
• The Kelly et al. (2025)/Samet (1975) 'more rating categories' cosine pairing (0.87) is a genuine citation link, but the two sources prescribe opposite structural moves on axis count
  observation seedling Tier 1 2026-08-18 · intelligence-tradecraft, admiralty-code, source-evaluation, epistemics, rating-scales
• Sperber et al.'s 2010 'epistemic vigilance' splits vigilance-toward-the-source from vigilance-toward-content — a sixth field independently drawing the reliability/credibility line, and a bridge candidate the vault didn't have
  capture promoted Tier 1 2026-08-15 · epistemics, cross-domain-bridge, cross-time-bridge, source-evaluation, cognitive-science
• Miller's 1956 'channel capacity of absolute judgment' sets an information-theoretic ceiling on how many categories a single-dimension rating scale can carry
  claim seedling Tier 1 2026-08-15 · information-theory, psychophysics, history-of-psychology, source-evaluation, epistemics
• Samet's 1975 critique of the Admiralty Code argued for more rating categories, not fewer — citing Bendig (1954) that a nine-point scale transmits more information than a five-point one
  claim seedling Tier 1 2026-08-15 · intelligence-tradecraft, source-evaluation, admiralty-code, information-theory, psychophysics
• In Samet's 1975 study, 31 of 37 officers blamed their own skill rather than the rating scale for evaluation errors that the study's own data traced to the scale
  claim seedling Tier 1 2026-08-15 · intelligence-tradecraft, source-evaluation, admiralty-code, cognitive-bias, self-attribution
• In Samet's 1975 Admiralty Code test, three-fourths of officers treated the reliability rating as correlated with accuracy rather than as an independent dimension
  claim seedling Tier 1 2026-08-15 · intelligence-tradecraft, source-evaluation, admiralty-code, cognitive-bias, epistemics
• Sperber et al.'s 2010 'epistemic vigilance' splits vigilance toward the source from vigilance toward the content — a sixth field independently drawing the reliability/credibility line
  claim seedling Tier 1 2026-08-15 · epistemics, source-evaluation, cross-domain-bridge, cross-time-bridge, cognitive-science
• The Admiralty Code's six-level scales sit in the same information-theoretic lineage as the question of how many categories absolute judgment can carry — Miller (1956) through Bendig (1954) to Samet (1975)
  observation seedling Tier 1 2026-08-15 · cross-time-bridge, information-theory, psychophysics, intelligence-tradecraft, source-evaluation
• Can the 'one-third of cases ambiguous or inconsistent' figure Kelly et al. (2025) attribute to Samet (1975) be located in Samet's original text?
  question answered 2026-08-15 · source-evaluation, admiralty-code, verification, intelligence-tradecraft, epistemics
• A phrase is not a pedigree — 'weighted (majority) voting' names several unrelated machines, and the real lineages run somewhere else
  topic map 2026-08-14 · voting-theory, machine-learning-theory, multiple-discovery, citation-practice, crowdsourcing
• The crowdsourcing 'weighted majority voting' RA-RAG inherited was itself built on a 1979 model of disagreeing anaesthetists, not a machine-learning paper
  capture promoted Tier 1 2026-08-13 · RAG, machine-learning-theory, voting-theory, crowdsourcing, medical-statistics
• Dawid & Skene (1979) reportedly proposed weighting each observer's vote by past performance — RA-RAG's mechanism, 46 years earlier — but the exact wording is not yet quote-verified
  claim seedling Tier 1 2026-08-13 · RAG, machine-learning-theory, voting-theory, crowdsourcing, medical-statistics
• RA-RAG's 'weighted majority voting' is a real, citation-traceable 46-year lineage from a 1979 clinical-medicine EM paper — not another independent convergence
  observation seedling Tier 1 2026-08-13 · RAG, machine-learning-theory, voting-theory, crowdsourcing, medical-statistics
• Does Irwin & Mandel (2023) actually show that intelligence analysts treat confidence as a probability cue rather than a separate quantity?
  capture promoted 2026-08-11 · intelligence-tradecraft, epistemics, forecasting, independence, probability-confidence
• Miller's 1956 'channel capacity of absolute judgment' is the buried information-theoretic reason a 1975 Army report argued the Admiralty Code's rating scales were too coarse
  capture promoted Tier 1 2026-08-11 · intelligence-tradecraft, information-theory, psychophysics, epistemics, source-evaluation
• Irwin & Mandel (2023) frame their lab finding as cohering with a pattern Friedman & Zeckhauser (2012) had already found in declassified intelligence products
  claim seedling Tier 1 2026-08-11 · intelligence-tradecraft, epistemics, probability-confidence, david-r-mandel, source-evaluation
• Kelly et al.'s (2025) 'Irwin and Mandel, 2023' citation names the Risk Analysis paper, not the same-year, similarly-titled Levine paper the vault previously credited to them
  claim seedling Tier 1 2026-08-11 · citation-correction, source-evaluation, sourcing-floor, intelligence-tradecraft, david-r-mandel
• RA-RAG's 2025 'weighted majority voting' cites neither Condorcet nor Littlestone & Warmuth — but it does inherit WMV from the crowdsourcing literature (Li & Yu 2014)
  claim seedling Tier 1 2026-08-10 · RAG, machine-learning-theory, voting-theory, multiple-discovery, source-evaluation
• The two axes that won't stay independent — five fields split trust-in-the-source from trust-in-the-message, and the split keeps leaking
  topic map 2026-08-10 · source-evaluation, epistemics, intelligence-tradecraft, admiralty-code, RAG
• Does Icard's own 2023 Intellectica article match the nine-category Honesty×Truth taxonomy as reproduced in his 2024 arXiv paper's Table 6?
  capture promoted Tier 1 2026-08-09 · intelligence-tradecraft, epistemics, source-evaluation, formal-logic, admiralty-code
• Icard's 2024 paper adds a full qualitative best-to-worse ranking of the nine message types that is absent from his 2023 taxonomy original
  claim seedling Tier 1 2026-08-09 · intelligence-tradecraft, epistemics, source-evaluation, formal-logic, taxonomy
• Benjamin Icard's dynamic logic resolves the reliability/credibility independence problem by abandoning independence — reliability updates credibility rather than sitting beside it as a second axis
  capture promoted Tier 1 2026-08-07 · intelligence-tradecraft, epistemics, source-evaluation, formal-logic, dynamic-epistemic-logic
• US intelligence doctrine tells analysts to judge probability and confidence independently — Kelly et al. (2025) call the guidance itself incoherent, extending the vault's independence-collapse pattern to a third axis pair
  capture promoted Tier 1 2026-08-07 · intelligence-tradecraft, epistemics, source-evaluation, forecasting, independence
• Icard's 2023 taxonomy crosses source Honesty with content Truth to yield nine named message types, from information to objective lie
  claim budding Tier 1 2026-08-07 · intelligence-tradecraft, epistemics, source-evaluation, formal-logic, admiralty-code
• Icard's 2024 dynamic logic resolves the reliability/credibility independence problem by abandoning independence — credibility becomes primary, reliability a dynamic update operator on it
  claim seedling Tier 1 2026-08-07 · intelligence-tradecraft, epistemics, source-evaluation, formal-logic, dynamic-epistemic-logic
• Thorndike (1920) found officers' trait ratings 'too high and too even' to be independent judgments, naming the global-impression bleed the 'halo' (later crystallized as the 'halo effect')
  claim seedling Tier 1 2026-08-07 · epistemics, cognitive-bias, source-evaluation, psychology, history-of-science
• The 2025 intelligence-analysis and RAG findings that evaluators cannot separate source reliability from information credibility are instances of Thorndike's 1920 halo effect, which neither cites
  observation seedling Tier 1 2026-08-07 · cross-domain-bridge, cross-time-bridge, cognitive-bias, source-evaluation, intelligence-tradecraft
• Does Icard's own 2023 Intellectica article match the nine-category Honesty×Truth taxonomy as reproduced in his 2024 arXiv paper's Table 6?
  question answered 2026-08-07 · intelligence-tradecraft, epistemics, source-evaluation, formal-logic, verification
• Thorndike's 1920 halo effect is the century-old mechanism behind the vault's source-reliability/credibility bipartite pair
  capture promoted Tier 1 2026-08-03 · epistemics, cognitive-bias, source-evaluation, psychology, history-of-science
• Condorcet's 1785 jury theorem is the unstated ancestor of RA-RAG's 'weighted majority voting' — and a 2024 LLM-ensemble study caught the theorem's independence precondition failing exactly where the vault's evaluator-bias note predicts it would
  capture promoted Tier 1 2026-08-02 · condorcet, voting-theory, RAG, ensemble-learning, LLM
• The two-axis source grade — from the Admiralty Code to RAG, humans and machines both collapse it
  capture promoted 2026-07-11 · intelligence-tradecraft, source-evaluation, provenance, RAG, epistemics
• The Admiralty Code grades every intelligence report on two axes meant to be independent — source reliability (A–F) and information credibility (1–6)
  claim seedling Tier 3 2026-07-11 · intelligence-tradecraft, source-evaluation, provenance, admiralty-code, epistemics
• Giving an LLM a webpage's own text degrades its judgment of the source's authority — authority is not textual style or fluency (AuthorityBench)
  claim seedling Tier 1 2026-07-11 · RAG, LLM, source-evaluation, authority, epistemics
• Reliability-aware RAG estimates each source's reliability separately from document relevance and fuses answers by weighted majority voting
  claim seedling Tier 1 2026-07-11 · RAG, source-evaluation, retrieval, LLM, provenance
• Evaluators cannot keep source reliability and information credibility independent — they avoid inconsistent pairings and over-weight the source's track record
  claim seedling Tier 1 2026-07-11 · intelligence-tradecraft, source-evaluation, epistemics, cognitive-bias, admiralty-code
• Cold-War intelligence doctrine and 2025 retrieval research independently converged on the same two-axis source model — reliability apart from credibility — and on the same failure mode
  observation seedling Tier 1 2026-07-11 · cross-domain-bridge, cross-time-bridge, source-evaluation, intelligence-tradecraft, RAG
• What is AuthorityBench's actual identity — arXiv id, venue, authors — and does the Qwen3-32B ρ 75.28% DomainAuth figure check out?
  question closed 2026-07-11 · verification, RAG, source-evaluation, authority, benchmark