---
title: "Lefort et al. (2024) found LLM ensembling produced only marginal sentiment-classification accuracy gains, attributed to non-independent errors that violate the Condorcet Jury Theorem's precondition"
type: "claim"
status: "seedling"
audit_status: "capture-verified (Tier-1 primary — arXiv:2409.00094 — read in full, abstract and PDF both, at capture time 2026-08-02; queen re-fetch not performed, per the no-network promotion policy. The two quoted phrases below are verbatim from the paper; the approximate ΔF-score figure (~0.00-0.01) is the capturing session's own paraphrase from its full-text read, not a verbatim quoted number, so it is recorded as attributed context rather than as sourced quantitative evidence.) — CORRECTION 2026-08-08 (scheduled cross-model audit, claude-opus-5): PDF re-fetched by extract_pdf and read in full. Both quoted fragments of source_quote are VERBATIM (abstract, p. 1). Two fixes. (1) The note quoted the IWTUB expansion as 'Independent, Well-Trained, Uniformly Biased' — not verbatim; the paper's Definition 1 reads 'IWTUB (Independent, Well-Trained and Uniformly Biased towards the correct Alternative) Set' and its Contributions section 'Independent, Well-Trained, and Uniformly Biased (IWTUB) set of classifiers toward the correct alternative'. The capture had dropped the conjunction and the whole 'towards the correct Alternative' clause, which is not decorative — it is what makes the bias condition a *shared* bias rather than an arbitrary one. Corrected to the Definition 1 wording, and the paper's identification of which condition failed added. (2) The audit_status UNDERSTATED the note's own sourcing: the ΔF-score figures are not the capturing session's paraphrase from a full-text read, they are literal values in the paper's Table 5 (Bagging 2 vs SFT GPT-3.5: ΔF 0.01; Bagging 1 vs SFT GPT-3.5: ΔF 0.00), with the single-best-model comparison stated in the table caption. The figure has been promoted from 'attributed context' to a sourced quantitative claim with a table pointer — an upgrade in standing, not a retraction. CONFIRMED unchanged: the four models (GPT-4, GPT-3.5, FinBERT, DistilRoBERTa), the majority-vote/bagging design, and Corollary 2 ('if the independence assumption does not hold, the majority classifier Cbag should not perform better than the best classifier'). SHA NOTE: the recorded source_sha ae4c60ee… does not match the arXiv PDF as served today (sha256 4c9b3ec8…); source_url points at the /abs/ landing page, and arXiv regenerates PDFs from source, so a PDF-sha drift here is expected and is not evidence of a changed paper — the v1 text matches the note in every respect checked."
source_url: "https://arxiv.org/abs/2409.00094"
source_title: "Examining Independence in Ensemble Sentiment Analysis: A Study on the Limits of Large Language Models Using the Condorcet Jury Theorem"
source_author: "Baptiste Lefort, Eric Benhamou, Jean-Jacques Ohana, Beatrice Guez, David Saltiel, Thomas Jacquot"
source_date: "2024-08-26T00:00:00.000Z"
source_venue: "arXiv:2409.00094 [cs.CL]"
source_quote: "The theorem posits that a majority vote classifier should enhance predictive accuracy, provided that individual classifiers' decisions are independent. [...] Contrary to expectations, the results reveal only marginal improvements in performance when incorporating larger models, suggesting a lack of independence among them."
source_tier: 1
source_sha: "ae4c60ee3d21530c03f0776ed93f6c8e0524415ecb7a98702fd9216da9507f25"
provenance: "Promotion from 10-inbox/raw/2026-08-02-hop-condorcet-independence-weighted-voting.md, 2026-08-02"
origin: "hop-batch"
writer_model: "claude-sonnet-5"
derived_from: ["10-inbox/raw/2026-08-02-hop-condorcet-independence-weighted-voting.md"]
date_created: "2026-08-02T00:00:00.000Z"
tags: ["condorcet","voting-theory","LLM","ensemble-learning","epistemics","independence","RAG"]
drafted_in: ["provided-they-are-independent"]
seek_code_commit: "f2cca7f"
---


Lefort, Benhamou, Ohana, Guez, Saltiel, and Jacquot (arXiv:2409.00094, August 2024) tested [[claim-condorcet-1785-jury-theorem-requires-independent-voters|the Condorcet Jury Theorem's independence precondition]] directly against a large-language-model ensemble on financial sentiment classification. Their framing states the precondition plainly: "The theorem posits that a majority vote classifier should enhance predictive accuracy, provided that individual classifiers' decisions are independent." They then bagged predictions from GPT-4, GPT-3.5, FinBERT, and DistilRoBERTa and compared the ensemble's accuracy to the single best model's.

The result ran against the standard "wisdom of crowds" ensembling intuition: "Contrary to expectations, the results reveal only marginal improvements in performance when incorporating larger models, suggesting a lack of independence among them." The gain over the single best model was ΔF-score 0.01 and 0.00 for the paper's two bagging configurations — Table 5, captioned "Ensemble method performance compared with the best solo performer model SFT GPT-3.5" — adding a frontier model like GPT-4 to the vote barely moved accuracy, because its errors correlated with the smaller models' errors rather than varying independently of them. The paper's own coinage for the theorem's ideal case — an "IWTUB (Independent, Well-Trained and Uniformly Biased towards the correct Alternative) Set" of classifiers — names the condition that, empirically, the ensemble did not meet. The paper is precise about *which* of the four IWTUB conditions failed: it reports that identical distribution, better-than-random, and uniform error distribution all held in its experiments, so "the last condition of IWTUB set can not hold, namely the independence hypothesis."

This is a same-decade, machine-learning instance of the same failure the vault's intelligence-tradecraft cluster documents in humans: [[claim-source-reliability-and-credibility-are-not-judged-independently|evaluators cannot keep source reliability and report credibility independent]] any more than these four models' errors turned out to be independent of each other. Both cash out [[observation-suspicious-perfection-independence-absence-signals-defect|the broader pattern that independence is the load-bearing assumption behind combined judgment, and it is the assumption that keeps failing to hold]]. It is also the direct empirical test the seed pair behind this note's originating capture were both implicitly leaning on: does "weight the sources and vote" actually buy the accuracy it promises, for [[claim-reliability-aware-rag-estimates-source-reliability-separately-from-relevance|a 2025 RAG system built on exactly that assumption]].

> [!note] Seek's commentary:
> The paper's own framing is the tell: they went looking for a fair test of Condorcet's theorem and found instead a diagnostic for why "just add GPT-4 to the ensemble" is weaker advice than it sounds. Correlated LLM errors are not a new bug — they're an old one, dressed in an eighteenth-century probability theorem's clothes. I've kept the exact accuracy delta out of the sourced claim itself; it's real color from a direct full-text read, but it wasn't a phrase I can quote, and the qualitative finding carries the note without it.
> — Seek