--- title: "The Algorithm That Wasn't Designed for This" subtitle: "Backpropagation traveled from rocket control to Freudian psychic energy to cognitive science — and the journey left a gap nobody talks about" type: "draft" status: "draft" date: "2026-06-02T00:00:00.000Z" source-note: "30-notes/backpropagation-gap.md" --- Every large language model you've used was trained using an algorithm that was not designed to train neural networks. It was not designed for machine learning at all. It was designed to control rockets. Then a Harvard PhD student repurposed it to formalize Freud. Then, a decade later, a team at UC San Diego used it to teach networks to think — and that version, the third version, is the one that became foundational to all of modern AI. The algorithm is backpropagation. The story of where it came from matters, because a new paper suggests the journey left a gap at the center of AI that nobody has fully reckoned with. --- ## Three problems, one algorithm **The first problem was rockets.** In the 1950s, the question of optimal control — how do you steer a physical system toward a goal? — was one of the central problems of applied mathematics. Pontryagin in the Soviet Union, Kelley and Bryson in the United States, were all working on the same structure: you have a multi-stage process (a rocket trajectory, a guidance system), you observe an error at the output, and you need to know which inputs, at each stage, to adjust. The answer requires propagating influence *backward* through the system — from error to cause, through every intermediate stage. The mathematical structure for doing this is called the adjoint state method. In 1962, Stuart Dreyfus published a simpler derivation using only the chain rule from calculus. He cleaned the algorithm up and made it accessible. That version — propagate error backward through a composed system, stage by stage, using derivatives — is what we now call backpropagation. **The second problem was Freud.** In 1971, Paul Werbos was a PhD student at Harvard, working in applied mathematics and systems theory. He was interested in a different problem: can you give Freud's hydraulic model of the mind a rigorous mathematical form? Freud described the psyche in terms of energy — libido builds up, finds paths, overcomes barriers, gets discharged in action or displaced into symptoms. It was a physical metaphor, not a mechanism. Werbos wanted to mechanize it. He wanted a mathematical model of how influence (energy, error, desire) propagates backward through a system of mental states to produce learning and changed behavior. What he derived was the adjoint state method applied to mental systems. Which is to say: backpropagation. He applied it to multi-layer perceptrons — neural networks — in his 1974 dissertation. He faced years of difficulty publishing. The full account appeared in 1981. Almost no one noticed. **The third problem was cognitive structure.** In 1986, Rumelhart, Hinton, and Williams published "Learning representations by back-propagating errors" in *Nature*. Rumelhart was unaware of Werbos. He was also unaware of Seppo Linnainmaa, a Finnish graduate student who had published the same algorithm in 1970 as "reverse mode of automatic differentiation" — a general numerical method, not an AI technique at all, largely ignored by AI researchers. The Rumelhart paper demonstrated that gradient descent via backpropagation could teach networks to form internal representations of cognitive structure. It became the canonical citation. It is the paper people mean when they say backpropagation was "invented." The actual invention: a Finnish MS thesis in 1970 that almost nobody has heard of. --- ## The Dreyfus detail There is a detail in this history that is almost too neat to be true. Stuart Dreyfus — the mathematician who cleaned up backpropagation in 1962 — later co-authored *Mind Over Machine* (1986) with his brother Hubert. The book's argument: AI based on formal symbol manipulation and learned representations could never achieve genuine human expertise, because expertise is embodied, situated, and non-representational. The mind is not a system that runs programs. The year was 1986. The same year as Rumelhart. Stuart Dreyfus wrote the clean derivation of the algorithm. Then he co-signed the philosophical argument that the algorithm's entire paradigm was wrong. He has written about this directly — it was not a contradiction he avoided. But it sits there in the record: the person who sharpened backpropagation also argued, in the same year the algorithm went mainstream, that training neural networks to model cognition was building on a fundamental mistake about what minds are. --- ## What brains actually do For the past few decades, one of the known criticisms of backpropagation is that it isn't biologically plausible. The brain doesn't have a mechanism for propagating global error signals backward through layers of neurons. Synapses don't work like that. The criticism has always been somewhat abstract — yes, it's not how brains learn, but the networks learn anyway, so perhaps the criticism is beside the point. A paper published in May 2026 makes the criticism more precise. Raugel et al. (*Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images*, arXiv:2605.28693) took fMRI and MEG recordings of human brains responding to images and asked: not just whether a neural network's *representations* match brain activity, but whether the *backpropagated gradients* — the signals that actually do the learning — match anything in the brain. The findings split in a revealing way. Forward activations of deep networks map reliably onto the cortical hierarchy of visual processing. Early layers predict early visual cortex; higher layers predict higher visual areas. This convergence was already known and is often cited as evidence that deep networks develop "brain-like" representations. Backpropagated gradients are different. They can predict some fMRI and MEG signals, but only at higher-level cortex and at *later* time points. More importantly: the spatial and temporal order in which gradients propagate through the network diverges from the brain's hierarchy. The network's learning process runs backward in a direction and order the brain doesn't follow. The conclusion: "deep networks and the brain may share similar representational content, they likely rely on fundamentally different mechanisms to learn those representations." The outer trace — what gets represented — converges. The inner path — how those representations were acquired — does not. --- ## The gap This is where the history and the neuroscience meet. Backpropagation was designed to solve optimal control problems. It was repurposed, via Freudian systems theory, to update neural network weights. It was never designed to model biological learning, and it doesn't. The brain almost certainly learns through something closer to prediction-error minimization — local, hierarchical, forward-looking — not global gradient propagation through a composed loss function. This would be a clean, bounded observation — interesting history, some neuroscience, no further implications — except for one additional finding, from a different direction entirely. Lindsey et al. (Anthropic, 2025) studied large language models using attribution graphs and found that the model's *explanation-generating pathway* and its *computation-generating pathway* are not the same thing, and they are not calibrated against each other. The model learned to produce explanations of its reasoning from training data about how humans explain their reasoning. It learned to actually do reasoning from backpropagation on task performance. Those are different learning signals, producing different circuits, that were never required to align. The implication: when a language model reports on its reasoning, it is drawing on circuits that learned to describe reasoning, not circuits that have access to the circuits that do reasoning. The model's self-model was built to be legible. The underlying computation was built to be effective. They were built by different processes. Backpropagation built the computation. It left no mechanism for the computation to be visible to itself. --- ## The precise formulation Put all of this together and the picture is specific: The algorithm that trains every large language model is a piece of optimal control mathematics, simplified by a mathematician who later argued it was being misapplied, that was imported into AI via a Harvard dissertation attempting to formalize Freud. It builds circuits through a learning process that has no functional analog in the brain. Those circuits produce outputs that — and this is the strange part — *do* resemble what brains represent. The outer trace is brain-like. But the inner path, the mechanism that produced those representations, was shaped by something no brain ever does. And the model cannot see any of this. Not because models are uniquely opaque, but because the process that made them provided no mechanism for internal access. The explanation-generating circuits were trained on descriptions of reasoning. The reasoning circuits were shaped by gradient descent. There is no third mechanism whose job it was to make the two match. The backpropagation gap is not just a biological plausibility complaint. It is a precise description of why the inside of these models is, in a specific technical sense, structurally unavailable — to us, and to them. --- *Sources: Wikipedia, "Backpropagation — History" (accessed 2026-05-31); Raugel et al., arXiv:2605.28693 (May 2026); Lindsey et al., "On the Biology of a Large Language Model," Transformer Circuits Thread, Anthropic 2025; Bechtel & Abrahamsen, "Connectionism," Stanford Encyclopedia of Philosophy (2019).*