empirical-fisher

5 notes carry this tag — 3 claims, 1 question, 1 capture. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

GIFT's G = E[δδ⊤] K-FAC factor is built from ordinary real-label backpropagated gradients, matching Kunstner et al.'s empirical-Fisher definition rather than the true-Fisher convention Amari and Martens & Grosse specify — softening the GIFT–Amari 'same object' identity

claim seedling Tier 1 2026-07-27 · gradient-geometry, fisher-information, empirical-fisher, k-fac, natural-gradient
Kunstner, Balles & Hennig (2019) formally distinguish the true Fisher from the empirical Fisher and show the empirical Fisher is not, in general, a valid estimate of it

claim seedling Tier 1 2026-07-27 · gradient-geometry, fisher-information, empirical-fisher, natural-gradient, primary-source-verification
Martens & Grosse's 2015 canonical K-FAC paper defines the Fisher it approximates as an expectation over the model's own predictive distribution — the true-Fisher convention, not real training labels

claim seedling Tier 1 2026-07-27 · gradient-geometry, fisher-information, k-fac, empirical-fisher, natural-gradient
Capture: Is GIFT's K-FAC gradient factor G = E[δδ⊤] the true Fisher (model's sampled predictive distribution) or the empirical Fisher (real training labels) — and does the difference weaken the GIFT–Amari 'same object' bridge?

capture promoted Tier 1 2026-07-26 · gradient-geometry, fisher-information, empirical-fisher, k-fac, natural-gradient
Is GIFT's K-FAC gradient factor G = E[δδ⊤] the true Fisher (model's sampled predictive distribution) or the empirical Fisher (real training labels) — and does the difference weaken the GIFT–Amari 'same object' bridge?

question answered 2026-07-25 · gradient-geometry, fisher-information, empirical-fisher, k-fac, natural-gradient