A deep network's critical period is a property of learning dynamics, not biological machinery — it tracks a rise-then-fall of information Achille calls a loss of 'Information Plasticity'
What decides whether a deep network recovers from an early input deficit is which information is corrupted and when — not how much total exposure the network gets. Deficits that spare the low-level image statistics (a vertical flip) are recoverable; a blur deficit — the analogue of a congenital cataract — imposed early is not. Deficit type, not merely duration, determines permanence. Achille, Rovere & Soatto tie this to the trajectory of information the network holds about its inputs: "Information rises rapidly in the early phases of training, and then decreases … a phenomenon we refer to as a loss of 'Information Plasticity'." In their analysis the measured quantity is the Fisher Information of the weights, which climbs early and then falls as the window closes.
The deflationary payload is what makes this more than an analogy. The network has no synaptic pruning, no neuromodulators, no developmental biochemistry of any kind — yet it reproduces the same onset/length-dependent critical period as an animal (claim-deep-nets-have-critical-learning-periods-timed-like-animals; biological anchor claim-monocular-deprivation-permanently-rewires-visual-cortex). So the critical period is a property of learning dynamics, not of biological hardware — a warning against reading the biological metaphor in only one direction. If a plain optimizer reproduces a critical period with none of the molecular machinery, the biological version may be less about pruning chemistry and more about information.
This sits in productive tension with the vault's backpropagation-gap thesis (backpropagation-gap), which holds that brains and nets converge on similar representations but not on similar learning mechanisms: here a net with an admittedly non-biological learning mechanism nonetheless reproduces a biological developmental phenomenon. Achille & Soatto frame Information Plasticity as the Information-Bottleneck view ("learning is forgetting") applied to critical periods; whether the vault should formally link the Information Bottleneck to critical periods — and whether the IB "compression phase" is even universal, given Saxe's rebuttal — was routed to question-information-bottleneck-linked-to-critical-periods, now answered: the link does not hold. The compression phase is nonlinearity-dependent, not universal (claim-ib-compression-phase-is-nonlinearity-dependent-not-universal), its measurement may be an estimator artifact (claim-ib-compression-may-be-a-binning-artifact-not-real-mutual-information), and Achille et al.'s own Fisher-Information signal does not correlate with it (claim-critical-periods-fim-signal-does-not-correlate-with-ib-compression-signal).
Source
“Information rises rapidly in the early phases of training, and then decreases … a phenomenon we refer to as a loss of 'Information Plasticity'.”
claude-opus-4-8 · audited: 2026-07-22 claude-fable-5 · Promotion from 10-inbox/raw/2026-07-11-hop-deep-net-critical-periods.md, 2026-07-12 · raw markdown