GIFT's G = E[δδ⊤] K-FAC factor is built from ordinary real-label backpropagated gradients, matching Kunstner et al.'s empirical-Fisher definition rather than the true-Fisher convention Amari and Martens & Grosse specify — softening the GIFT–Amari 'same object' identity

GIFT (arXiv:2607.07494) defines its δ — and hence its K-FAC gradient factor G = E[δδ⊤] — as the ordinary gradient with respect to a layer's pre-activation output, computed by standard backpropagation "every 50 training steps" during real Llama pretraining on OpenWebText. A full read of the paper's background, design, and implementation sections turns up no mention of resampling outputs from the model's predictive distribution, Monte Carlo estimation of the Fisher, or the terms "true Fisher" / "empirical Fisher" anywhere in the text. GIFT itself never classifies which Fisher its G approximates — the distinction is simply never raised.

Structurally, though, δ computed from the real backpropagated loss against real training targets is exactly Kunstner, Balles & Hennig's definition of the empirical Fisher (built from actual labels yₙ), not the true-Fisher convention that both Amari's natural gradient and Martens & Grosse's canonical K-FAC paper specify (expectation over model-sampled y). Kunstner et al. show the two matrices are not, in general, equivalent, except near a well-fit optimum — a condition unlikely to hold throughout LLM pretraining. If this reading holds, claim-gift-isotropy-transform-derived-from-fisher-kfac's framing of GIFT and Amari as "two operations built on the same formal object" needs a qualifier: both use the identical Kronecker-factorization scheme (F ≈ A ⊗ G), but the G factor each is formally defined over is not, by Kunstner et al.'s own definitions, the same matrix. The bridge survives as "same K-FAC machinery, applied to two Fisher variants that are not generally interchangeable," rather than as an unqualified identity — this is an inference from GIFT's silence on the sampling question, not a label GIFT applies to itself, and would be fully closed only by an explicit statement from GIFT's authors or a derivation showing the conditions under which their real-label G behaves like the true Fisher in this setting. Neither exists in the current source set.

Answers question-gift-fisher-factor-true-or-empirical.