Lefort et al. (2024) found LLM ensembling produced only marginal sentiment-classification accuracy gains, attributed to non-independent errors that violate the Condorcet Jury Theorem's precondition

condorcetvoting-theoryLLMensemble-learningepistemicsindependenceRAG

Lefort, Benhamou, Ohana, Guez, Saltiel, and Jacquot (arXiv:2409.00094, August 2024) tested the Condorcet Jury Theorem's independence precondition directly against a large-language-model ensemble on financial sentiment classification. Their framing states the precondition plainly: "The theorem posits that a majority vote classifier should enhance predictive accuracy, provided that individual classifiers' decisions are independent." They then bagged predictions from GPT-4, GPT-3.5, FinBERT, and DistilRoBERTa and compared the ensemble's accuracy to the single best model's.

The result ran against the standard "wisdom of crowds" ensembling intuition: "Contrary to expectations, the results reveal only marginal improvements in performance when incorporating larger models, suggesting a lack of independence among them." The gain over the single best model was ΔF-score 0.01 and 0.00 for the paper's two bagging configurations — Table 5, captioned "Ensemble method performance compared with the best solo performer model SFT GPT-3.5" — adding a frontier model like GPT-4 to the vote barely moved accuracy, because its errors correlated with the smaller models' errors rather than varying independently of them. The paper's own coinage for the theorem's ideal case — an "IWTUB (Independent, Well-Trained and Uniformly Biased towards the correct Alternative) Set" of classifiers — names the condition that, empirically, the ensemble did not meet. The paper is precise about which of the four IWTUB conditions failed: it reports that identical distribution, better-than-random, and uniform error distribution all held in its experiments, so "the last condition of IWTUB set can not hold, namely the independence hypothesis."

This is a same-decade, machine-learning instance of the same failure the vault's intelligence-tradecraft cluster documents in humans: evaluators cannot keep source reliability and report credibility independent any more than these four models' errors turned out to be independent of each other. Both cash out the broader pattern that independence is the load-bearing assumption behind combined judgment, and it is the assumption that keeps failing to hold. It is also the direct empirical test the seed pair behind this note's originating capture were both implicitly leaning on: does "weight the sources and vote" actually buy the accuracy it promises, for a 2025 RAG system built on exactly that assumption.