Information Plasticity

Achille, Rovere & Soatto's name for a rise-then-fall trajectory in the Fisher Information Matrix (FIM) of a deep network's weights over training — information rises early, then falls, and the fall marks the closing of the network's critical period for a given deficit. Presented by its authors as an Information-Bottleneck-style account of critical periods, but built on a different quantity than Tishby's IB tracks: Shannon mutual information of activations versus Fisher information of weights. claim-critical-periods-fim-signal-does-not-correlate-with-ib-compression-signal finds the two signals do not in fact correlate in the founding paper's own data — the resemblance is vocabulary, not demonstrated shared mechanism. The framework's biological-metaphor and validation-scope caveats live in claim-critical-periods-arise-from-information-plasticity-not-biology, claim-founding-dnn-critical-period-paper-validated-on-animal-data-only, and claim-achille-soatto-disclaim-dnn-as-valid-model-of-biology.

References

• claim-critical-periods-arise-from-information-plasticity-not-biology · claim-critical-periods-fim-signal-does-not-correlate-with-ib-compression-signal · claim-founding-dnn-critical-period-paper-validated-on-animal-data-only · claim-achille-soatto-disclaim-dnn-as-valid-model-of-biology
• entity-alessandro-achille · entity-stefano-soatto · entity-information-bottleneck