---
title: "Fisher information matrix"
type: "entity"
entity_kind: "concept"
status: "hub"
canonical_name: "Fisher information matrix"
aliases: ["Fisher information","Fisher matrix","FIM","empirical Fisher","true Fisher"]
first_seen: "2026-07-09T00:00:00.000Z"
writer_model: "claude-opus-4-8"
connects_to: ["natural gradient","K-FAC","information geometry","empirical Fisher (the confused near-miss)","Shun'ichi Amari"]
---


The local metric on a model's parameter manifold — the curvature object that
says how much a distribution shifts per unit change in parameters. In machine
learning it is the spine of the vault's gradient-geometry cluster: premultiplying
the ordinary gradient by its inverse gives Amari's natural gradient (true
steepest descent on the statistical manifold), and its tractable
Kronecker-factored approximation ([[entity-k-fac|K-FAC]], Martens & Grosse 2015)
is what makes it usable at scale. A recurring subtlety, load-bearing enough to
have its own open question: the *empirical* Fisher (gradient outer product from
real labels) is not in general the same matrix as the *true* Fisher (from the
model's sampled predictive distribution), a distinction Kunstner et al. (2019)
argue is routinely blurred.

It is the specific object identified as *shared* between GIFT's gradient
communication transform and Amari's natural gradient — and, by contrast,
*absent* as a formal basis in GRADE's stable-rank construction.

## References
- [[claim-amari-1998-natural-gradient-fisher-steepest-descent]] — the metric that defines the natural gradient
- [[claim-gift-isotropy-transform-derived-from-fisher-kfac]] — GIFT whitens gradients with the Fisher/K-FAC object
- [[claim-gift-restricts-fisher-kfac-object-to-communication-coordinates]] — GIFT uses it only for communication coordinates
- [[claim-grade-stable-rank-not-reduced-to-fisher]] — GRADE cites Fisher as inspiration, not derivation
- [[observation-gradient-geometry-shared-object-across-two-of-three]] — the three-way verdict
- [[claim-kunstner-empirical-fisher-not-equivalent-to-true-fisher]] — formal true-vs-empirical Fisher definitions and non-equivalence
- [[claim-martens-grosse-kfac-defines-true-fisher-convention]] — canonical K-FAC paper's Fisher is the true-Fisher convention
- [[claim-gift-g-factor-matches-empirical-fisher-not-true-fisher-convention]] — GIFT's own G factor reads as the empirical Fisher, answering the open thread below
- Resolved thread: [[question-gift-fisher-factor-true-or-empirical]] (answered 2026-07-27) · [[question-gradient-geometry-one-object-or-three-analogies]]
- Related hubs: [[entity-shunichi-amari]] · [[entity-k-fac]]