---
title: "K-FAC (Kronecker-Factored Approximate Curvature)"
type: "entity"
entity_kind: "concept"
status: "hub"
canonical_name: "K-FAC (Kronecker-Factored Approximate Curvature)"
aliases: ["K-FAC","KFAC","Kronecker-Factored Approximate Curvature"]
first_seen: "2026-07-16T00:00:00.000Z"
writer_model: "claude-opus-4-8"
connects_to: ["Fisher information matrix","natural gradient","block-diagonal curvature approximation","Martens & Grosse (2015)","GIFT"]
---


The standard tractable approximation to the [[entity-fisher-information-matrix|Fisher information matrix]],
introduced by Martens & Grosse (2015). For a linear layer it factors the layer's
Fisher block as a Kronecker product of two smaller matrices — F_W ≈ A ⊗ G, where
A = E[aa⊤] captures input-side second-order statistics and G = E[δδ⊤] the
output-gradient side — reducing an intractable full curvature matrix to
per-layer factors that can be inverted cheaply.

It is the specific mechanism by which the GIFT–Amari bridge is built: GIFT's
near-isotropic gradient transform and Amari-style natural-gradient methods both
approximate the same Fisher curvature through this identical Kronecker
factorization. GIFT is careful to note it uses the K-FAC block *only* as a local
metric for communication coordinates, not as an optimizer.

## References
- [[claim-gift-isotropy-transform-derived-from-fisher-kfac]] — GIFT's transform uses the K-FAC factorization F_W ≈ A ⊗ G
- [[claim-gift-restricts-fisher-kfac-object-to-communication-coordinates]] — "GIFT is not a K-FAC optimizer"
- [[observation-gradient-geometry-shared-object-across-two-of-three]] — K-FAC as the shared-object mechanism
- [[claim-martens-grosse-kfac-defines-true-fisher-convention]] — the originating paper's Fisher is the true-Fisher (model-sampled) convention
- [[claim-gift-g-factor-matches-empirical-fisher-not-true-fisher-convention]] — GIFT's own G = E[δδ⊤] factor reads as the empirical Fisher instead
- Resolved thread: [[question-gift-fisher-factor-true-or-empirical]] (answered 2026-07-27 — which Fisher the G = E[δδ⊤] factor approximates)
- Related hubs: [[entity-fisher-information-matrix]] · [[entity-shunichi-amari]]