The James-Stein estimator uniformly dominates the sample mean under squared-error loss, making the MLE inadmissible in three or more dimensions

Before 1961 the textbook consensus held that no estimation rule could uniformly improve on the observed sample average when estimating the means of several independent normal observations under squared-error loss. The James-Stein estimator broke that consensus on maximum likelihood's own home turf: for three or more simultaneously estimated normal means, shrinking every individual estimate toward a common central point produces a lower expected total squared error than the raw averages — for every value of the true means, not merely on average over some prior. (Dimension ≥ 3 is the floor for shrinkage toward a pre-chosen fixed point; the better-known variant that shrinks toward the estimated grand average spends one dimension locating that centre and dominates for dimension ≥ 4 — Efron & Hastie state the theorem for N ≥ 4, CASI Ch. 7, (7.15)–(7.16).) The sample mean is therefore inadmissible in dimension ≥ 3. Efron & Hastie call the 1961 result a "rude shock to the statistical world."

The counterintuitive part is that the pooled quantities need have nothing to do with one another. Efron & Morris's worked version pools 18 batting averages with the proportion of imported cars in Chicago: the theorem applies to the 19-problem lump exactly as it did to the 18, though the guarantee covers only the total squared error — pooling a genuinely atypical unrelated quantity can degrade the individual estimates ("Stein's Paradox in Statistics," Scientific American 1977). The gain is paid for by accepting bias in each individual estimate in exchange for a large reduction in variance across the ensemble — a bias-variance trade made at the level of the whole vector rather than any one component.

The mechanism is the same "borrow strength from unrelated data" move that Herbert Robbins and Stein formalized in the same era (Robbins's empirical Bayes, 1956; Stein's inadmissibility proof, published 1956; the explicit estimator, James & Stein, 1961 — "Stein's paradox" itself is Efron & Morris's 1977 coinage). Shrinkage toward the grand mean is exactly what an empirical-Bayes prior would prescribe — which is why the result is so naturally explained in Bayesian terms, and why its discoverer's refusal of that explanation is itself notable (claim-stein-delayed-admissibility-proof-five-years-to-avoid-bayesian-argument). The concrete magnitude of the effect is documented in Efron's baseball worked example (claim-efron-baseball-shrinkage-halved-batting-average-prediction-error).