---
title: "Cross-validation (model-selection criterion)"
type: "entity"
entity_kind: "concept"
status: "hub"
canonical_name: "cross-validation"
aliases: ["out-of-sample testing","CV"]
first_seen: "2026-08-02T00:00:00.000Z"
writer_model: "claude-sonnet-5"
connects_to: ["overfitting","take-the-best","bias-variance tradeoff","Henry Brighton","model selection"]
seek_code_commit: "f2cca7f"
---


The statistical practice of testing a model on data it was not fit on, as opposed to
fitting and testing on the same sample — the specific methodological choice reported
to flip the verdict in the [[entity-gerd-gigerenzer|Gigerenzer]]/Brighton camp's
dispute with [[entity-nick-chater|Chater]] & [[entity-mike-oaksford|Oaksford]] over
take-the-best. Fit-then-test-on-the-whole-sample rewards flexible models that overfit,
especially at high sample sizes; cross-validation exposes that overfitting by holding
out data the model hasn't seen. It matters to the vault as the load-bearing mechanism
across an entire empirical dispute, not a footnote to it — the same overfitting logic
that later became central to machine-learning practice generally.

## References
- [[claim-brighton-2006-reanalysis-reversed-take-the-best-result-against-chater-oaksford]] · [[claim-brighton-2006-aaai-paper-does-not-contain-city-population-reanalysis]] · [[observation-ecological-rationality-and-rational-analysis-are-contested-not-compatible]]
- Captures: 2026-07-31-are-gigerenzers-ecological-rationality-and-oaksford-chaters-rational, 2026-08-02-did-the-chateroaksford-camp-reply-to-brightons-2006