GIFT's near-isotropic gradient transform is derived from the Fisher information matrix via the same K-FAC factorization that underlies Amari's natural gradient

GIFT (arXiv:2607.07494) frames its isotropy transform not as an independently-motivated whitening step but as an operation on the Fisher information matrix. The paper assigns the Fisher the same role Amari's 1998 natural gradient does — "The Fisher information matrix provides a local metric for measuring parameter perturbations in neural-network optimization" — and then names its specific approximation: "Following the K-FAC block introduced in Section II-B, the local metric of a linear layer is approximated by F_W ≈ A ⊗ G, A = E[aa⊤], G = E[δδ⊤]." The isotropizing step is described in the same terms Amari's construction uses: it is "a whitening operation on the layerwise gradient statistics," and applying the inverse K-FAC factors "turn[s] the local anisotropic metric into a coordinate system whose metric ball is spherical." That is the Fisher-information ellipsoid converted into a Euclidean ball — the same curvature object Amari's natural gradient uses to define steepest descent, here applied as a coordinate change rather than a preconditioner.

On the evidence of GIFT's own method section, then, its "isotropy transform" and Amari's "natural gradient" are not two analogies that happen to share vocabulary: they are two operations built on the same formal object, the Fisher information matrix in its K-FAC-factored form. This is distinct from the already-promoted claim-gift-2026-gradient-anisotropy-isotropic-transform, which records GIFT's diagnosis (Euclidean quantization distorts anisotropic gradients) and fix (transform to near-isotropic space first); the present claim is about the transform's derivation from a named curvature object. How that object is scoped is the subject of claim-gift-restricts-fisher-kfac-object-to-communication-coordinates; the three-way verdict it feeds is observation-gradient-geometry-shared-object-across-two-of-three.

One refinement is left open and routed to question-gift-fisher-factor-true-or-empirical: whether GIFT's G = E[δδ⊤] factor is the true Fisher (from the model's sampled predictive distribution, as Amari's construction assumes) or the empirical Fisher (from real training labels), which per claim-amari-1998-natural-gradient-fisher-steepest-descent's lineage are not generally the same matrix.