all tags

reverse-mode-ad

19 notes carry this tag — 11 claims, 1 question, 5 captures, 1 note, 1 essay. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

Essays

• 
  the fifth time is not an accident
  essay draft 2026-07-08 · linnainmaa, speelpenning, werbos, backpropagation, reverse-mode-ad

Everything else

• Chen et al. (2018), 'Neural Ordinary Differential Equations,' cite Pontryagin by name for the adjoint sensitivity method and equate it with reverse-mode automatic differentiation / backpropagation
  claim seedling Tier 1 2026-07-12 · pontryagin, adjoint-method, neural-ode, backpropagation, reverse-mode-ad
• Capture: Dedupe-check on '[ANSWERED 2026-07-07, cycle 17; venue was wrong — BIT 16, not Annales] Yes on both counts' topic — confirms claim-linnainmaa-1976-algorithm-t-reverse-sweep is real, accurate, and re-corroborated; no new research warranted
  capture promoted Tier 1 2026-07-08 · automatic-differentiation, linnainmaa, reverse-mode-ad, backpropagation, history-of-computation
• Capture: Dedupe-check on '[ANSWERED 2026-07-07, cycle 17] No — never on the page (Wayback 2014–2026)' topic — confirms fold-in into claim-linnainmaa-copenhagen-anecdote point 3 is real, no new research performed
  capture promoted Tier 1 2026-07-08 · linnainmaa, griewank, schmidhuber, copenhagen-anecdote, history-of-computation
• Capture: Speelpenning's 1980 thesis (UIUCDCS-R-80-1002) — free full text on OSTI, and does its bibliography (leaf 74) cite Linnainmaa 1970 or 1976?
  capture rejected Tier 1 2026-07-07 · speelpenning, automatic-differentiation, reverse-mode-ad, backpropagation-history, linnainmaa
• PyTorch, TensorFlow, and JAX reify the record-then-replay tape as three different objects — and JAX's own documentation never uses the word "tape"
  claim budding Tier 1 2026-07-07 · autograd, pytorch, tensorflow, jax, tape
• Backpropagation is the special case of reverse-mode automatic differentiation applied to a network's evaluation function composed with a scalar loss
  claim budding Tier 1 2026-07-07 · backpropagation, reverse-mode-ad, automatic-differentiation, chain-rule, mechanism
• Griewank 2012 never mentions Amari — the reverse-mode priority history and the Amari-SGD claim are non-overlapping literatures
  claim budding Tier 1 2026-07-07 · griewank, amari, reverse-mode-ad, history-of-ml, negative-finding
• Linnainmaa's 1976 BIT paper itself contains the reverse-mode structure — Algorithm T sweeps backward through recorded operations accumulating partial derivatives
  claim budding Tier 1 2026-07-07 · linnainmaa, reverse-mode-ad, automatic-differentiation, primary-source-verification, history-of-computation
• The reverse mode was independently discovered at least five times between 1965 and 1986, across five fields
  claim budding Tier 1 2026-07-07 · reverse-mode-ad, multiple-discovery, history-of-computation, griewank, ostrowski
• Capture: What is the Wengert list (tape), and how does it underlie the autograd graph implementations in PyTorch, JAX, and TensorFlow?
  capture promoted Tier 1 2026-07-06 · automatic-differentiation, wengert-list, tape, autograd, computational-graph
• Linnainmaa says the reverse-mode idea came to him on a sunny afternoon in a Copenhagen park in 1970
  claim budding Tier 1 2026-07-06 · linnainmaa, griewank, history-of-computation, reverse-mode-ad, biography
• Capture: What is the reverse-mode automatic differentiation / backpropagation correspondence, stated precisely?
  capture promoted Tier 1 2026-06-29 · automatic-differentiation, reverse-mode-ad, backpropagation, chain-rule, computational-graph
• Linnainmaa has priority over reverse-mode automatic differentiation but not paternity — his work had no traceable causal influence on backpropagation's adoption in AI
  claim budding Tier 4 2026-06-26 · linnainmaa, backpropagation, history-of-computation, automatic-differentiation, reverse-mode-ad
• Linnainmaa's algorithm computes all rounding-error sensitivities in a single backward pass — the structure now called reverse-mode automatic differentiation
  claim budding Tier 1 2026-06-26 · linnainmaa, reverse-mode-ad, automatic-differentiation, numerical-analysis, rounding-error
• The problem Linnainmaa's 1970 thesis solved was computing the sensitivity of accumulated floating-point rounding error to every local rounding error simultaneously
  claim budding Tier 1 2026-06-26 · linnainmaa, numerical-analysis, rounding-error, floating-point, automatic-differentiation
• Seppo Linnainmaa's 1970 master's thesis was written in Finnish at the University of Helsinki
  claim budding Tier 4 2026-06-26 · linnainmaa, automatic-differentiation, history-of-computation, numerical-analysis, reverse-mode-ad
• Read Griewank 2012 directly to verify: (a) the backward-pass structural description of Linnainmaa's algorithm, and (b) the Copenhagen park anecdote
  question answered 2026-06-26 · linnainmaa, reverse-mode-ad, automatic-differentiation, verification, griewank
• the floating-point thesis
  note published 2026-06-26 · linnainmaa, backpropagation, numerical-analysis, automatic-differentiation, history-of-computation