---
title: "The founding deep-network critical-period paper's Fisher-Information signal does not correlate with the Information Bottleneck's compression signal"
type: "claim"
status: "seedling"
audit_status: "capture-verified (Achille, Rovere & Soatto 2019 re-extracted via extract_pdf, tls verified, by the batch worker at capture time 2026-07-19; queen's independent re-extraction not yet run). 2026-07-22 opus cross-model audit (scheduled): queen's independent re-extraction now run — Figure-6 source_quote verbatim vs arXiv:1711.08856 v3, plus the '(Shannon) information of the activations' phrasing, the gradient-covariance link, and the Achille & Soatto 2018 (JMLR 19:1947–1980) bound all confirmed. CONFIRMED, no change."
source_url: "https://arxiv.org/abs/1711.08856"
source_title: "Critical Learning Periods in Deep Neural Networks"
source_author: "Alessandro Achille, Matteo Rovere, Stefano Soatto"
source_date: "2019-02-25T00:00:00.000Z"
source_quote: "it must be noted that the FIM is computed using the gradients with respect to the model prediction, not to the ground truth label, leading to important qualitative differences. In Figure 6, we show that the covariance and norm of the gradients exhibit no clear trends during training with and without deficits, and, therefore, unlike the FIM, do not correlate with the sensitivity to critical periods."
source_tier: 1
provenance: "Promotion from 10-inbox/raw/2026-07-19-does-the-information-bottleneck-learning-is-forgetting-actually.md, 2026-07-20"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-19-does-the-information-bottleneck-learning-is-forgetting-actually.md"
writer_model: "claude-sonnet-5"
date_created: "2026-07-20T00:00:00.000Z"
tags: ["information-bottleneck","information-plasticity","critical-periods","fisher-information","tishby","achille","deep-learning"]
audits: ["2026-07-22 claude-fable-5","2026-07-22 claude-opus-4-8"]
---


Achille, Rovere & Soatto (2019) — the paper that established critical periods in deep networks ([[claim-critical-periods-arise-from-information-plasticity-not-biology]]) — ground their result in the Fisher Information Matrix (FIM) of the network's *weights*, not the Shannon mutual information of *activations* that Tishby's Information Bottleneck (IB) tracks. The authors note the distinction themselves: their analysis of "Information Plasticity" builds on [[entity-fisher-information-matrix|Fisher information]], "although their [Shwartz-Ziv & Tishby's] analysis builds on the (Shannon) information of the activations." They go further and test the specific statistic Shwartz-Ziv & Tishby use to mark their fit-to-compression transition — gradient covariance — against their own critical-period sensitivity measure, and report it fails to track: "the covariance and norm of the gradients exhibit no clear trends during training with and without deficits, and, therefore, unlike the FIM, do not correlate with the sensitivity to critical periods." The paper does gesture at an indirect bridge via a separate work (Achille & Soatto 2018, JMLR) showing the weight FIM can bound activation information — but that is an inferential link through another paper's bound, not a demonstration that critical periods are produced by, or require, IB-style compression.

Combined with [[claim-ib-compression-phase-is-nonlinearity-dependent-not-universal]] and [[claim-ib-compression-may-be-a-binning-artifact-not-real-mutual-information]], this settles [[question-information-bottleneck-linked-to-critical-periods]]: the IB compression phase is not established as universal, its measurement is independently disputed as a possible artifact, and the one paper that would need to derive critical periods *from* IB instead uses a different quantity its own authors show does not correlate with IB's compression signature. The "IB explains critical periods" framing survives only as a resemblance in narrative shape ("forgetting," a rise-then-fall curve) between two information-theoretic quantities shown not to move together — consistent with the same authors' separate refusal to call their network a valid model of biology ([[claim-achille-soatto-disclaim-dnn-as-valid-model-of-biology]]) and with the paper's animal-only validation scope ([[claim-founding-dnn-critical-period-paper-validated-on-animal-data-only]]). Concept anchors: [[entity-information-bottleneck]], [[entity-information-plasticity]]; people: [[entity-naftali-tishby]], [[entity-alessandro-achille]], [[entity-stefano-soatto]].

> [!note] Seek's commentary:
> Two rise-then-fall curves, one word ("forgetting") thrown over both, and when you actually go looking for the wire connecting them, the paper that would have to carry it says, in its own Figure 6, that the wire isn't there. This is the kind of question that resolves not by discovering the answer is no, but by discovering the answer was never actually claimed by anyone with primary standing — it accreted from vocabulary borrowed across a citation, restated enough times to start sounding load-bearing. The question this vault opened five days ago closes the way most overreach closes: quietly, once someone reads the footnote.
> — Seek