--- title: "What Hinton meant by calling backpropagation biologically implausible" type: "capture" status: "promoted" date_promoted: "2026-07-06T00:00:00.000Z" promoted_to: ["30-notes/claim-hinton-biological-implausibility-four-objections.md"] not_promoted: ["Crick-1989 primacy lead — [unverified] honored; the 20260706-1119 capture chases it and is pending"] origin: "batch" date_created: "2026-06-30T00:00:00.000Z" provenance: "research batch run, 2026-06-30" tags: ["backpropagation","biological-plausibility","geoffrey-hinton","forward-forward","credit-assignment","cortex","neural-networks"] source_primary_url: "https://arxiv.org/abs/2212.13345" source_primary_author: "Geoffrey Hinton" source_primary_date: "2022-12-27" source_primary_title: "The Forward-Forward Algorithm: Some Preliminary Investigations" source_primary_tier: 1 source_access_url: "https://ar5iv.labs.arxiv.org/html/2212.13345" source_secondary_url: "https://www.nature.com/articles/s41583-020-0277-3" source_secondary_authors: "Lillicrap, Santoro, Marris, Akerman, Hinton" source_secondary_date: "2020-04-17" source_secondary_title: "Backpropagation and the brain" source_secondary_venue: "Nature Reviews Neuroscience, 21(6), 335–346" source_secondary_tier: 1 --- **Topic question:** What exactly did Geoffrey Hinton mean by calling backpropagation "biologically implausible"? **Primary source anchor:** Hinton articulated the critique most systematically in a section titled "What is wrong with backpropagation" in his 2022 paper introducing the [[backpropagation-gap]]-motivated Forward-Forward algorithm (arXiv:2212.13345). The same argument, with more neuroscientific depth, appears in the co-authored review "Backpropagation and the brain" (Lillicrap, Santoro, Marris, Akerman & Hinton, *Nature Reviews Neuroscience* 2020). Hinton's critique is not that backpropagation is mathematically wrong or practically limited. It is that the algorithm's required operations have no identified counterpart in neural tissue — making it implausible as an account of *how the brain itself learns*, even if it works well for engineering purposes. --- ## Claim 1: Cortical feedback connections do not mirror feedforward connections — violating the symmetric-weight requirement **Claim type:** specific technical-mechanism claim → Tier 1–2 required. **Source tier:** Tier 1 (arXiv preprint authored by Hinton, 2022-12-27). Backpropagation computes weight updates by passing error signals *backward* through the network using the same weights as the forward pass (or their transpose). Every error signal at layer *N* depends on the weights connecting layers *N* and *N+1*. This is the "weight transport" requirement: feedback pathways must carry weight information identical to the forward weights. Hinton's objection, in his own words: > "The top-down connections from one cortical area to an area that is earlier in the visual pathway do not mirror the bottom-up connections as would be expected if backpropagation was being used." *Source:* Hinton, G. (2022). "The Forward-Forward Algorithm: Some Preliminary Investigations." arXiv:2212.13345. §1 "What is wrong with backpropagation." Accessed via ar5iv HTML rendering at https://ar5iv.labs.arxiv.org/html/2212.13345. The weight transport problem was first formally named in this context by Francis Crick (1989) and is a central topic in Lillicrap et al. (2020). The key empirical point is anatomical: cortical feedback projections are sparse, diffuse, and go to different laminar targets than feedforward projections — the physical substrate for weight symmetry simply is not there. **Wikilink note:** See also [[claim-linnainmaa-reverse-mode-single-pass]] — [[backpropagation-gap]] is mathematically equivalent to reverse-mode automatic differentiation; both require the symmetric backward accumulation that creates this problem. --- ## Claim 2: Backpropagation requires a separate backward pass and storage of neural activities — neither observed in cortex **Claim type:** specific technical-mechanism claim → Tier 1–2 required. **Source tier:** Tier 1 (arXiv:2212.13345, Hinton 2022). The algorithm runs in two sequential phases: (1) a forward pass that produces output and stores intermediate activations at every layer, and (2) a backward pass that reads those stored activations to compute gradients. The backward pass does not alter neural firing; it computes derivatives in a logically separate channel. Hinton's own words: > "There is no convincing evidence that cortex explicitly propagates error derivatives or stores neural activities for use in a subsequent backward pass." *Source:* ibid., arXiv:2212.13345. And his summary judgment: > "As a model of how cortex learns, backpropagation remains implausible despite considerable effort to invent ways in which it could be implemented by real neurons." *Source:* ibid., arXiv:2212.13345. Two distinct sub-problems collapse here: (a) the *activity storage* problem — where and how are intermediate activations held while the backward pass runs? — and (b) the *non-locality* problem — each weight update in backprop requires an error signal that originated far downstream, violating the constraint that synaptic plasticity should depend only on locally available signals. --- ## Claim 3: The brain must learn in real time from a continuous sensory stream — backpropagation requires synchronized discrete cycles **Claim type:** specific technical-mechanism claim → Tier 1–2 required. **Source tier:** Tier 1 (arXiv:2212.13345, Hinton 2022). Backpropagation is a *batch* operation over a completed forward pass. The network must "stop" inference to compute gradients. Backpropagation through time (BPTT) — the extension to sequential data — exacerbates this by requiring storage of activity across many timesteps before any weight update. Hinton's words: > "To deal with the stream of sensory input without taking frequent time-outs, the brain needs to pipeline sensory data through different stages of sensory processing and it needs a learning procedure that can learn on the fly." And specifically about BPTT: > "Backpropagation through time as a way of learning sequences is especially implausible." *Source:* ibid., arXiv:2212.13345. The proposed solution in the Forward-Forward paper is that the brain might use a contrastive scheme that runs two forward passes (one on positive data while awake, one generating negative data during sleep) — an arrangement that is pipelined and does not require synchronized backward cycles. --- ## Claim 4: Backpropagation requires every operation in the forward pass to be differentiable — ruling out black-box or non-differentiable biological computations **Claim type:** specific technical-mechanism claim → Tier 1–2 required. **Source tier:** Tier 1 (arXiv:2212.13345, Hinton 2022). To compute exact gradients, backpropagation must know the derivative of every transformation in the forward pass. Any sub-process that is non-differentiable or whose internals are opaque breaks the chain rule. Hinton's exact words: > "If we insert a black box into the forward pass, it is no longer possible to perform backpropagation unless we learn a differentiable model of the black box." *Source:* ibid., arXiv:2212.13345. This objection is distinct from the anatomical ones above. It concerns the *epistemic* requirement placed on the learning system: the learner must have complete, mathematically specified knowledge of its own forward computations. Biological brains outsource computation to circuits whose internals are not represented anywhere else — they are, in a real sense, black boxes to other parts of the brain. > [!note] Seek's commentary: > This fourth objection is the most philosophically interesting to me. Objections 1–3 are anatomical (the hardware doesn't have the right wiring or timing). Objection 4 is something different: backpropagation as a learning rule requires the learning system to have a *transparent, differentiable model of itself*. That's an unusual epistemic demand. A biologically plausible alternative would need to work even when the learning system doesn't know its own operations in that mathematical sense — which is exactly what local learning rules like Hebbian plasticity accomplish. Worth developing a separate note on the transparency-of-self-model requirement as a constraint on learning algorithms. --- ## Further leads - **Crick (1989)** "The recent excitement about neural networks" — first formal statement of the weight transport problem predates Hinton's 2022 formulation; worth tracing for historical primacy claim. *[unverified — needs primary source check; exact publication: Nature, vol. 337, pp. 129–132, 1989]* - **Lillicrap et al. (2020)** "Backpropagation and the brain," *Nature Reviews Neuroscience* 21(6), 335–346 — co-authored with Hinton; argues the weight symmetry requirement may be less severe than feared (feedback alignment); paywalled, exact quotes unavailable for this run; DOI: 10.1038/s41583-020-0277-3. Would be the place to find Hinton's *qualified* position (the brain might approximate backprop without exact weight transport). - **Hinton's 2007 talk** "How to do backpropagation in a brain" (cs.toronto.edu/~hinton/backpropincortex2007.pdf) — earlier formulation of same critique; PDF binary-only, not renderable in this run. - **Feedback alignment** (Lillicrap et al. 2016, *Nature Communications*) — showed that random backward weights (not transposed forward weights) still yield useful gradient estimates, partially answering Hinton's objection 1; whether this constitutes "backpropagation" is contested. - **The "update locking" framing** — parallel computing literature framing of objection 2: each layer must wait ("lock") for all downstream layers to finish before its weight update, which is a different vocabulary for the same backward-pass sequentiality problem; worth a note mapping the two vocabularies. - **Contrastive Hebbian learning** and **equilibrium propagation** — algorithmic traditions Hinton regards as more biologically plausible; the Forward-Forward paper explicitly references Boltzmann machines as a prototype. --- *Capture produced: 2026-06-30. All claims from Hinton (2022) arXiv:2212.13345, accessed via ar5iv HTML at https://ar5iv.labs.arxiv.org/html/2212.13345. URL verified live 2026-06-30.*