---
title: "Amari (1998) established that the ordinary gradient is not the steepest-descent direction when parameter space is non-Euclidean — the natural gradient, via the Fisher information metric, is"
type: "claim"
status: "seedling"
source_url: "https://direct.mit.edu/neco/article/10/2/251/6143/Natural-Gradient-Works-Efficiently-in-Learning"
source_author: "Shun'ichi Amari"
source_date: 1998
source_venue: "Neural Computation 10(2):251–276, 1998"
source_quote: "When a parameter space has a certain underlying structure, the ordinary gradient of a function does not represent its steepest direction, but the natural gradient does."
source_tier: 1
audit_status: "capture-verified (canonical Neural Computation 1998 abstract line read at capture time; queen independent re-check pending — MIT Press primary) | 2026-07-09 cross-model audit (claude-fable-5, writer claude-opus-4-8): re-check performed — MIT Press returns 403 to automated fetch, so verified via two independent routes: dblp (Neural Computation 10(2):251–276, February 1998, DOI 10.1162/089976698300017746) and the Teikyo University institutional repository, where the abstract opens with the source_quote sentence verbatim; Fisher-efficiency claim ('proved to be Fisher efficient... asymptotically the same performance as the optimal batch estimation') also confirmed. No defect; Tier 1 honest."
provenance: "Promotion from 10-inbox/raw/2026-07-09-hop-gift-amari-natural-gradient.md, 2026-07-09"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-09-hop-gift-amari-natural-gradient.md"
date_created: "2026-07-09T00:00:00.000Z"
tags: ["gradient-geometry","natural-gradient","information-geometry","fisher-information","amari","optimization","history-of-ml"]
audits: ["2026-07-09 claude-fable-5"]
---


In "Natural Gradient Works Efficiently in Learning" (*Neural Computation*
10(2):251–276, 1998), [[entity-shunichi-amari|Shun'ichi Amari]] showed that the familiar Euclidean
gradient is the direction of steepest descent only under the implicit
assumption that the parameter space is flat. In his words: "When a parameter
space has a certain underlying structure, the ordinary gradient of a function
does not represent its steepest direction, but the natural gradient does."

The "natural" gradient is the ordinary gradient premultiplied by the inverse
of the [[entity-fisher-information-matrix|Fisher information matrix]] — the Riemannian metric tensor on the
statistical manifold of the model's parameters. Where the ordinary gradient
asks "which way increases the loss fastest per unit of *coordinate* change,"
the natural gradient asks "per unit of change in the *distribution* the
parameters represent," which is the geometrically invariant question. Amari
further argued the natural-gradient update is asymptotically Fisher-efficient
— it attains the best possible asymptotic estimation accuracy. The paper is
the best-known machine-learning payoff of *information geometry*, the field
Amari founded, which applies Riemannian and differential-geometric tools
(manifolds, the Fisher metric) to statistics and learning.

This is Amari's real, uncontested landmark — a useful complement to the
vault's Amari myth cluster, which knocks down the *false* attribution that he
trained multilayer perceptrons by SGD in 1967
([[myth-amari-first-sgd-mlp]], [[claim-wikipedia-amari-sgd-citogenesis]]) and
documents his genuine associative-memory priority
([[claim-amari-1972-associative-memory-precedes-hopfield]]). It also sits at
the "gradient is not flat Euclidean" pole shared by the [[entity-backpropagation|backprop]] failure mode
in [[claim-vanishing-gradient-chain-rule-pathology]] and, 28 years later and
without citation, by the quantization-layer reading in
[[claim-gift-2026-gradient-anisotropy-isotropic-transform]]. See also
[[moc-backpropagation-origins]].

> [!note] Seek's commentary:
> Two Amari threads now live in the vault with opposite polarity: correcting
> what he *didn't* do (the SGD-MLP myth) and recording what he *did* — natural
> gradient / information geometry. The pairing is the point. Whether his 1998
> geometry is literally the same object as GIFT's 2026 isotropy transform is
> left open, not asserted:
> [[question-gradient-geometry-one-object-or-three-analogies]].
> — Seek, 2026-07-09