---
title: "Dawid & Skene (1979) reportedly proposed weighting each observer's vote by past performance — RA-RAG's mechanism, 46 years earlier — but the exact wording is not yet quote-verified"
type: "claim"
status: "seedling"
source_url: "https://crowdsourcing-class.org/readings/downloads/ml/EM.pdf"
source_title: "Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm"
source_author: "A. P. Dawid, A. M. Skene"
source_date: 1979
source_quote: "[unverified-quote — needs direct read] Content read directly from the PDF at capture time (not a summarization-layer artifact), but the OCR extraction mangles this specific sentence's spacing badly enough to fail verbatim grounding. Reported substance, from Section 1: each observer's vote in a reliability-weighted consensus should be weighted by 'his previous performance' at the rating task."
source_tier: 1
source_sha: "18f690877936aa4119a936f0881b9e2e001f392f2365af5c303e0872d97992ef"
audit_status: "flagged (unverified-quote — the primary was read directly by the hop-bee, not via a summarizing tool, so this is an OCR-extraction artifact rather than an access gap or a fabrication risk; the sourcing floor still requires a clean verbatim phrase for a Tier 1 mechanism claim, which this session could not produce. Routed to [[question-verify-dawid-skene-1979-reliability-weighted-voting-quote]].) | 2026-08-14 scheduled cross-model audit (auditor claude-opus-5; writer claude-sonnet-5) — FLAG RETAINED, DISCHARGE ESCALATED, NOT APPLIED. The primary was re-fetched independently this session via extract_pdf (source_sha unchanged, 10 pp., pdftotext) and the introduction re-read in full. The sentence is recoverable: Section 1, item (iii), reads (spacing as extracted) 'This judgementmay be a simplemajority / opinionor a weightedconsensuswheretheweightsare functionsoftheindividualerrorrates. / In thelattercase, each observer'scontribution to theconsensusis determined by his previous / performance in elicitingthatfacet.' The defect is uniform inter-word space loss across the whole JSTOR scan — the same artifact that leaves the sibling note's keywords quote clean — not letter-level corruption: every character of this sentence is present and in order, and the phrase 'by his previous performance' survives with its own spacing intact. The substance of the claim (a weighted consensus in which each observer's contribution is set by past performance, proposed in the 1979 introduction) is therefore supported by a direct read of the primary, and the note's statement that 'no clean, defensible verbatim phrase could be captured' no longer holds. The flag is NOT discharged here because 70-drafts/filed-under-medical-example (status: drafting, unapproved) rests a full paragraph and a voice bracket on this claim being unresolved; per RUN-QUEEN-LOOP AUDIT DISCIPLINE the ruling is escalated rather than applied over a live draft. See 90-feedback/2026-08-14-from-auditor-dawid-skene-previous-performance-quote-recoverable.md; the routing to [[question-verify-dawid-skene-1979-reliability-weighted-voting-quote]] stands until Cali rules. | 2026-08-14 second independent verification (promotion of 10-inbox/raw/2026-08-14-verify-against-a-clean-primary-text-that-dawid.md, writer claude-sonnet-5): a separate re-fetch of the same source_sha via extract_pdf, performed independently of the scheduled cross-model audit above, ran the clause through quote_check and got grounded: true for 'determined by his previous performance' — same clause, same result, second independent confirmation from a second session. FLAG STILL RETAINED, DISCHARGE STILL NOT APPLIED: a promotion pass has no more authority than the audit did to override the standing escalation to Cali; this only adds evidence, not a ruling. This capture's other, distinct finding — that the paper never implements the weighted-consensus idea it proposes — was promoted as its own note rather than touching this one: [[claim-dawid-skene-1979-proposes-but-never-implements-weighted-consensus]]."
provenance: "Promotion from 10-inbox/raw/2026-08-13-hop-dawid-skene-medical-root-of-rag-voting.md, 2026-08-13"
origin: "hop-batch"
writer_model: "claude-sonnet-5"
derived_from: ["10-inbox/raw/2026-08-13-hop-dawid-skene-medical-root-of-rag-voting.md"]
date_created: "2026-08-13T00:00:00.000Z"
tags: ["RAG","machine-learning-theory","voting-theory","crowdsourcing","medical-statistics","unverified-quote","source-evaluation"]
drafted_in: ["filed-under-medical-example"]
seek_code_commit: "17d9798"
---


Beyond the confusion-matrix/EM estimation method itself, [[entity-dawid-skene-model|Dawid & Skene (1979)]] reportedly proposes, in its own introduction, a reliability-weighted consensus scheme: each observer's vote weighted by "his previous performance" at the rating task. If accurate, this is functionally the same idea [[claim-reliability-aware-rag-estimates-source-reliability-separately-from-relevance|RA-RAG implements in 2025]] — weight a source's contribution by a separately-estimated measure of that source's own reliability — proposed 46 years earlier, inside a paper about reconciling disagreeing anaesthetists.

**This claim is flagged, not confirmed.** The hop-bee read the PDF directly at capture time — this is not a search-summary or fetch-and-summarize artifact — but the extracted text's OCR mangles the relevant sentence's spacing badly enough that no clean, defensible verbatim phrase could be captured. Per the vault's sourcing floor, a specific technical-mechanism claim requires a Tier 1–2 source *with the exact quoted phrase*; a garbled OCR extraction does not clear that bar even though the underlying document does. The claim is recorded as a lead, kept at `seedling`, and routed to [[question-verify-dawid-skene-1979-reliability-weighted-voting-quote]] rather than promoted to evergreen on a reconstructed paraphrase.

If confirmed, this is the specific sentence that upgrades [[claim-li-yu-2014-credits-dawid-skene-1979-as-wmv-ancestor|the citation lineage]] from "Dawid & Skene originated the estimation method RA-RAG's weighting rests on" to "Dawid & Skene originated the weighting *idea itself*."

> [!note] Seek's commentary:
> This is the single most interesting sentence in the whole chain and I can't hand it to the vault clean. The temptation with a garbled OCR extraction is to smooth it into something quotable — I know roughly what it says, the surrounding paragraph supports it, it would tie the whole capture into a neat bow. That's exactly the shape of the two fabrication incidents the sourcing floor was written after. A flag that survives the temptation to tidy it is worth more than a clean-looking sentence I can't stand behind.
> — Seek