---
title: "automatic differentiation"
type: "entity"
entity_kind: "concept"
status: "hub"
canonical_name: "automatic differentiation"
aliases: ["autodiff","AD","reverse-mode automatic differentiation","reverse mode"]
first_seen: "2026-05-31T00:00:00.000Z"
writer_model: "claude-opus-4-8"
connects_to: ["reverse mode","backpropagation","the Wengert list / tape","cheap gradient principle","numerical analysis"]
---


The numerics-side name for the machinery that AI calls backpropagation — the
split is deliberate (per the entity-page spec's own canonical example),
because the same flower is caught whether the source speaks numerical analysis
or deep learning. Reverse-mode AD computes a full gradient in a small constant
multiple of the cost of the original function (the cheap-gradient principle,
Griewank's bound), running a recorded computation — the Wengert list, or
"tape" — backward. The vault's autodiff thread carries the Linnainmaa priority
story, the forward/reverse (JVP/VJP) transpose duality, and the observation
that the ML and AD communities were mutually unaware for years.

## References
- [[claim-linnainmaa-reverse-mode-single-pass]] · [[claim-cheap-gradient-bound-two-figures]] · [[claim-jvp-vjp-transpose-duality]] · [[claim-autograd-three-reifications-of-the-tape]] · [[claim-wengert-list-named-for-forward-mode-inventor]] · [[claim-ml-and-ad-communities-mutually-unaware]]
- Related hubs: [[entity-seppo-linnainmaa]] · [[entity-andreas-griewank]] · [[entity-backpropagation]] · Capture: 2026-07-06-did-linnainmaa-originate-reverse-mode-automatic-differentiation-per-griewank-2012