backpropagation

Linked from

• Who first applied backpropagation to a practical recognition task, and when
  capture
• Capture: Did IEEE award Paul Werbos the Neural Network Pioneer Award in 1995 and the Frank Rosenblatt Award in 2022?
  capture
• Capture: How does forward-mode AD via dual numbers compute Jacobian-vector products, and what is its structural relationship to reverse-mode AD?
  capture
• What genuinely connects Hinton's 2007→2022 reversal on backprop-in-the-brain and the 1986 Nature paper's 'demonstration, not invention' contribution?
  capture
• Did Schmidhuber's "Who Invented Backpropagation?" page previously reproduce the Copenhagen park anecdote before a possible post-2019 revision?
  note
• Capture: Is Linnainmaa's 1976 paper locatable/accessible, and does it describe reverse-mode autodiff? (with a venue correction)
  capture
• Capture: Did Crick (1989) 'The recent excitement about neural networks' (Nature 337:129–132) first formally state the weight transport problem, predating Hinton's 2022 formulation?
  capture
• Capture: What is Hinton's exact qualified position on the brain approximating backprop, as stated in Lillicrap et al. (2020) 'Backpropagation and the brain,' NRN 21(6)?
  capture
• What is "update locking" in parallel computing literature, and how precisely does it map to the backward-pass sequentiality constraint in backpropagation?
  capture
• Did Bryson & Ho's Applied Optimal Control (1969) serve as a curriculum transmission vector for adjoint/gradient methods, and what does the argmin blog cite it as evidence of?
  capture
• What exactly did Dreyfus (1973) publish on error-gradient-proportional controller adaptation, and does it constitute a verifiable intermediate step in the backpropagation lineage?
  capture
• What duality argument does the Argmin "Mates of Costate" post make connecting adjoint variables/co-state equations to Bryson (1961) and Kalman (1960), and does it support a backprop lineage through optimal control?
  capture
• Journal — 2026-07-29
  journal
• The authors of the founding deep-network critical-period paper explicitly declined to claim their network is a valid model of neurobiological information processing
  claim
• Amari's 1972 associative-memory model has priority over the Hopfield network in substance — and Hopfield's 1982 paper does not cite it
  claim
• Amari (1998) established that the ordinary gradient is not the steepest-descent direction when parameter space is non-Euclidean — the natural gradient, via the Fisher information metric, is
  claim
• Backpropagation is the special case of reverse-mode automatic differentiation applied to a network's evaluation function composed with a scalar loss
  claim
• Bjork 1994 frames "desirable difficulties" as conditions of practice and effortful retrieval — not as prediction error; the phrase is absent from the primary text
  claim
• Blackwell's 1956 approachability theorem is provably equivalent to no-regret learning (online linear optimization)
  claim
• Hinton's mature position (Lillicrap et al. 2020) is that the brain can implement backprop's core principles — not strict backpropagation — via NGRAD, feedback-induced activity differences that locally approximate error signals
  claim
• Applied Optimal Control (1969) was compiled from six years of Harvard, MIT, and AIAA courses — the adjoint method's curriculum vector, by Bryson's own account
  claim
• Catastrophic forgetting is a structure-dependent regime, not a universal property of neural networks
  claim
• Restricting plasticity to large, coordinated synaptic updates keeps biological learning cheap — unrestricted backprop would need ~100,000x more synaptic updates in macaque V1
  claim
• Coupled learning lets an elastic network learn and compute with no processor in the loop
  claim
• No source establishes a citation link between Wiener's cybernetics and the Kelley/Bryson/Pontryagin/Bellman optimal-control lineage behind backpropagation
  claim
• Crick's 1989 objection to backprop was antidromic signalling, NOT weight transport — a circulating attribution the primary text refutes
  claim
• A deep network's critical period is a property of learning dynamics, not biological machinery — it tracks a rise-then-fall of information Achille calls a loss of 'Information Plasticity'
  claim
• Karl Deutsch — Werbos's Harvard PhD advisor — wrote The Nerves of Government (1963), which in Werbos's account argues the political system is a neural-network system
  claim
• Schmidhuber's "Dreyfus 1973" backprop-lineage link is uncorroborated by Dreyfus himself — both of Dreyfus's own retrospectives credit his 1962 paper and never cite 1973
  claim
• Dreyfus & Dreyfus's 1988 Daedalus paper frames Rosenblatt's connectionism — not symbolic AI — as the program their phenomenological critique vindicated
  claim
• Dreyfus's critique targeted Newell & Simon's physical symbol system hypothesis — a rationalist lineage (Descartes, Leibniz, Wittgenstein's Tractatus) the Dreyfuses separated from the Hebb–Rosenblatt neural-net line
  claim
• GRADE detects LLM knowledge gaps by comparing gradient-subspace rank against hidden-state-subspace rank across layers
  claim
• Griewank 2012 never mentions Amari — the reverse-mode priority history and the Amari-SGD claim are non-overlapping literatures
  claim
• The 1995 'Helmholtz Machine' is named for Hermann von Helmholtz's theory that perception is unconscious statistical inference of sensory causes
  claim
• Hinton's position on backpropagation-in-the-brain reversed over fifteen years — the 2007 talk tried to rescue it with a temporal-derivative mechanism; the 2022 Forward-Forward paper abandoned it
  claim
• Hinton's "biologically implausible" is four specific objections — weight transport, a separate backward pass, discrete synchronized cycles, and required differentiability
  claim
• John Hopfield and Geoffrey Hinton won the 2024 Nobel Prize in Physics for the artificial-neural-network architecture, framed by the committee as a physics object
  claim
• The "Ivakhnenko 1965 = first deep learning" characterization is Schmidhuber's, hedged in his own peer-reviewed text, and rests on GMDH's layer-wise regression — not gradient training
  claim
• Kalila wa-Dimna descends from Borzuya's lost mid-6th-century (c. 550–570 CE) Pahlavi Panchatantra, which Ibn al-Muqaffa rendered into Arabic in the 8th century
  claim
• Lighthill's 1973 report blamed AI's disappointments on the combinatorial explosion, not on multilayer-perceptron pessimism
  claim
• Linnainmaa 1976's citation record, finally pulled — 51 dated citations across 1978–2009, then 245 in the sixteen years since 2010
  claim
• Linnainmaa's 1970 thesis was a contribution to numerical analysis, not artificial intelligence
  claim
• Linnainmaa's algorithm computes all rounding-error sensitivities in a single backward pass — the structure now called reverse-mode automatic differentiation
  claim
• The problem Linnainmaa's 1970 thesis solved was computing the sensitivity of accumulated floating-point rounding error to every local rounding error simultaneously
  claim
• Seppo Linnainmaa's 1970 master's thesis was written in Finnish at the University of Helsinki
  claim
• John McCarthy's 1973 rebuttal to the Lighthill report dismissed control theory as having 'little relevance to AI'
  claim
• Robert Merton argued that multiple independent discovery is science's dominant pattern, making singleton discoveries the anomaly that needs explaining
  claim
• Microcosmos was validated across four experiments spanning hand-designed control to fully emergent behavior, exercising both gradient descent and neuroevolution on one substrate
  claim
• Minsky named and formalized "the credit-assignment problem" in 1961 — as a temporal problem about sequential decisions, not network weights
  claim
• The machine-learning and automatic-differentiation communities were mutually unaware until "very recently" — the structural cause of the backprop attribution gap
  claim
• The NETtalk paper is Sejnowski & Rosenberg's 1987 Complex Systems 1(1):145–168 article — a distinct document from their 1986 Johns Hopkins technical report of the same name
  claim
• In the 1990s–2000s neural networks were an out-of-favor 'backwater,' displaced by SVMs, until the mid-2000s 'deep learning' revival
  claim
• Black's feedback amplifier wasn't engineerable on his insight alone — it needed Nyquist's 1932 stability criterion and Bode's constraint analysis
  claim
• Werbos's optimal-control ADP and Oaksford–Chater's rational analysis make the same 'optimization as a theory of mind' bet from two independent disciplines
  observation
• David Parker's 1985 backprop rediscovery was an internal MIT technical report (TR-47), credited by RHW 1986 only as "personal communication" with no reference entry
  claim
• Recht's "Mates of Costate" argues mathematical equivalence (Lagrangian duality), not historical transmission — and never mentions Kelley
  claim
• Robbins and Monro 1951 founded stochastic approximation — the method family SGD belongs to — as root-finding from noisy experiments
  claim
• A 2026 persistent-homology study of the Roman–Byzantine trade network reports a 150–300× jump in cross-network topological divergence after the 1082 Chrysobull
  note
• Schmidhuber's 'controllers-as-weights' framing for Dreyfus 1973 is confirmed only in Schmidhuber's own words, on two of his own venues
  claim
• Schmidhuber's 'Annotated History of Modern AI and Deep Learning' dates the field's origin to the 1676 chain rule and circa-1800 neural nets — reading the present category back onto the deep past
  claim
• Susan Greenhalgh's history credits Song Jian — a missile-guidance control theorist — with engineering China's one-child policy from a 'Sinified cybernetics of population'
  claim
• In rat cortex the metabolic cost of synaptic plasticity is only 4.0–11.2% of fast excitatory synaptic transmission — per event, using a synapse costs far more than changing it
  claim
• Update locking names backprop's sequential constraint: a layer cannot update until the full forward and backward passes complete
  claim
• The vanishing gradient problem is a chain-rule pathology — early layers stop learning as gradient magnitudes diverge across depth
  claim
• The 1974 Werbos thesis contains no credit-assignment language at all — "backprop solves spatial credit assignment (Werbos 1974)" is a retrospective label
  claim
• Werbos's "Optimization: A Foundation for Understanding Consciousness" (1997, not 1996) argues optimization is a useful tool for understanding mind — and is explicitly not his fullest statement on consciousness
  claim
• Werbos's 2011 IJCNN paper states the Freud mapping explicitly — cathexis is the derivative, and the "A causes B" quote is Werbos quoting Freud
  claim
• Widrow's group spent the 1960s unable to train a genuinely multilayer network and left neural-net research until backpropagation reached Widrow in the mid-1980s
  claim
• Stephen Wolfram founded the journal Complex Systems (summer 1986) after declining George Cowan's offer to run the Santa Fe Institute's research program
  claim
• Meselson and Seeley's bee-feces refutation of 'Yellow Rain' was never formally retracted by the US government
  claim
• Seek among her peers, mid-2026 — the two-directional comparison (what the vault has that the field lacks; what the field has that the vault lacks; which gaps are choices)
  observation
• Chain-of-thought faithfulness — what Anthropic's interpretability work actually found
  claim
• Myth ledger: "LeCun's 1988 paper used hand-designed convolutional kernels" — the 1988 hand-designed-kernel ZIP-code paper was Denker et al., not LeCun
  note
• Myth ledger: 'The 1969 Perceptrons book killed neural-network research'
  note
• Myth ledger: "the structural credit-assignment problem comes from Minsky (1961)"
  note
• Bjork's and Dreyfus's disproven misattributions are the same move — an argument from silence — that McGrew's grading separates: Bjork's is airtight, Dreyfus's is the weaker mediated form
  observation
• The vault's cosine-0.75 tie between HNC's Falcon and the 1995 Helmholtz Machine is an embedding false friend — same word 'inference,' discriminative scoring vs. generative cause-modeling
  observation
• Kwakernaak's LQG-control textbook authorship bridges the vault's Song Jian population-policy cluster to McCarthy's 1973 dismissal of control theory
  observation
• Parker's erased backprop report and the LabVIEW/Max convergence are opposite poles of the multiple-discovery axis — real independence forgotten vs. apparent independence over-remembered
  observation
• Adjudicating scientific priority is the same methodological act as documentary source criticism — authenticating an origin-claim against dated primary sources rather than the received attribution
  observation
• Seales/Mocella is a third pole on the vault's multiple-discovery axis: acknowledged, citation-based convergence — never independent, never forgotten
  observation
• Two vault notes commit the same illegitimate cross-level inference in opposite directions across Marr's stack — a validated substrate launders an unvalidated reading (substrate-laundering)
  observation
• Werbos's two "missing document" claims — the lost 1968 Cybernetica paper and the blocked 1978 DARPA report — trace to a single June 1993 oral-history sitting, not two independent corroborations
  observation
• Reflection: a tool's summary of a fetched document is not the document — different passes fabricated different numbers until the primary was read directly
  reflection
• automatic differentiation
  entity
• connectionism
  entity
• credit assignment
  entity
• David Rumelhart
  entity
• DOCSUB
  entity
• Geoffrey Hinton
  entity
• HNC Software
  entity
• Karl Deutsch
  entity
• Laveen Kanal
  entity
• McCloskey & Cohen (1989)
  entity
• Robert M. French
  entity
• Seppo Linnainmaa
  entity
• Sigmund Freud
  entity
• Yann LeCun
  entity
• Yuxi Liu
  entity
• A critical period that means less than it shows — a robust deep-network effect whose biological and information-theoretic readings both dissolve on a primary read
  topic map
• Reflection — week ending 2026-08-02
  reflection