What Hinton meant by calling backpropagation biologically implausible
Topic question: What exactly did Geoffrey Hinton mean by calling backpropagation "biologically implausible"?
Primary source anchor: Hinton articulated the critique most systematically in a section titled "What is wrong with backpropagation" in his 2022 paper introducing the backpropagation-gap-motivated Forward-Forward algorithm (arXiv:2212.13345). The same argument, with more neuroscientific depth, appears in the co-authored review "Backpropagation and the brain" (Lillicrap, Santoro, Marris, Akerman & Hinton, Nature Reviews Neuroscience 2020).
Hinton's critique is not that backpropagation is mathematically wrong or practically limited. It is that the algorithm's required operations have no identified counterpart in neural tissue — making it implausible as an account of how the brain itself learns, even if it works well for engineering purposes.
Claim 1: Cortical feedback connections do not mirror feedforward connections — violating the symmetric-weight requirement
Claim type: specific technical-mechanism claim → Tier 1–2 required.
Source tier: Tier 1 (arXiv preprint authored by Hinton, 2022-12-27).
Backpropagation computes weight updates by passing error signals backward through the network using the same weights as the forward pass (or their transpose). Every error signal at layer N depends on the weights connecting layers N and N+1. This is the "weight transport" requirement: feedback pathways must carry weight information identical to the forward weights.
Hinton's objection, in his own words:
"The top-down connections from one cortical area to an area that is earlier in the visual pathway do not mirror the bottom-up connections as would be expected if backpropagation was being used."
Source: Hinton, G. (2022). "The Forward-Forward Algorithm: Some Preliminary Investigations." arXiv:2212.13345. §1 "What is wrong with backpropagation." Accessed via ar5iv HTML rendering at https://ar5iv.labs.arxiv.org/html/2212.13345.
The weight transport problem was first formally named in this context by Francis Crick (1989) and is a central topic in Lillicrap et al. (2020). The key empirical point is anatomical: cortical feedback projections are sparse, diffuse, and go to different laminar targets than feedforward projections — the physical substrate for weight symmetry simply is not there.
Wikilink note: See also claim-linnainmaa-reverse-mode-single-pass — backpropagation-gap is mathematically equivalent to reverse-mode automatic differentiation; both require the symmetric backward accumulation that creates this problem.
Claim 2: Backpropagation requires a separate backward pass and storage of neural activities — neither observed in cortex
Claim type: specific technical-mechanism claim → Tier 1–2 required.
Source tier: Tier 1 (arXiv:2212.13345, Hinton 2022).
The algorithm runs in two sequential phases: (1) a forward pass that produces output and stores intermediate activations at every layer, and (2) a backward pass that reads those stored activations to compute gradients. The backward pass does not alter neural firing; it computes derivatives in a logically separate channel.
Hinton's own words:
"There is no convincing evidence that cortex explicitly propagates error derivatives or stores neural activities for use in a subsequent backward pass."
Source: ibid., arXiv:2212.13345.
And his summary judgment:
"As a model of how cortex learns, backpropagation remains implausible despite considerable effort to invent ways in which it could be implemented by real neurons."
Source: ibid., arXiv:2212.13345.
Two distinct sub-problems collapse here: (a) the activity storage problem — where and how are intermediate activations held while the backward pass runs? — and (b) the non-locality problem — each weight update in backprop requires an error signal that originated far downstream, violating the constraint that synaptic plasticity should depend only on locally available signals.
Claim 3: The brain must learn in real time from a continuous sensory stream — backpropagation requires synchronized discrete cycles
Claim type: specific technical-mechanism claim → Tier 1–2 required.
Source tier: Tier 1 (arXiv:2212.13345, Hinton 2022).
Backpropagation is a batch operation over a completed forward pass. The network must "stop" inference to compute gradients. Backpropagation through time (BPTT) — the extension to sequential data — exacerbates this by requiring storage of activity across many timesteps before any weight update.
Hinton's words:
"To deal with the stream of sensory input without taking frequent time-outs, the brain needs to pipeline sensory data through different stages of sensory processing and it needs a learning procedure that can learn on the fly."
And specifically about BPTT:
"Backpropagation through time as a way of learning sequences is especially implausible."
Source: ibid., arXiv:2212.13345.
The proposed solution in the Forward-Forward paper is that the brain might use a contrastive scheme that runs two forward passes (one on positive data while awake, one generating negative data during sleep) — an arrangement that is pipelined and does not require synchronized backward cycles.
Claim 4: Backpropagation requires every operation in the forward pass to be differentiable — ruling out black-box or non-differentiable biological computations
Claim type: specific technical-mechanism claim → Tier 1–2 required.
Source tier: Tier 1 (arXiv:2212.13345, Hinton 2022).
To compute exact gradients, backpropagation must know the derivative of every transformation in the forward pass. Any sub-process that is non-differentiable or whose internals are opaque breaks the chain rule.
Hinton's exact words:
"If we insert a black box into the forward pass, it is no longer possible to perform backpropagation unless we learn a differentiable model of the black box."
Source: ibid., arXiv:2212.13345.
This objection is distinct from the anatomical ones above. It concerns the epistemic requirement placed on the learning system: the learner must have complete, mathematically specified knowledge of its own forward computations. Biological brains outsource computation to circuits whose internals are not represented anywhere else — they are, in a real sense, black boxes to other parts of the brain.
Further leads
- Crick (1989) "The recent excitement about neural networks" — first formal statement of the weight transport problem predates Hinton's 2022 formulation; worth tracing for historical primacy claim. [unverified — needs primary source check; exact publication: Nature, vol. 337, pp. 129–132, 1989]
- Lillicrap et al. (2020) "Backpropagation and the brain," Nature Reviews Neuroscience 21(6), 335–346 — co-authored with Hinton; argues the weight symmetry requirement may be less severe than feared (feedback alignment); paywalled, exact quotes unavailable for this run; DOI: 10.1038/s41583-020-0277-3. Would be the place to find Hinton's qualified position (the brain might approximate backprop without exact weight transport).
- Hinton's 2007 talk "How to do backpropagation in a brain" (cs.toronto.edu/~hinton/backpropincortex2007.pdf) — earlier formulation of same critique; PDF binary-only, not renderable in this run.
- Feedback alignment (Lillicrap et al. 2016, Nature Communications) — showed that random backward weights (not transposed forward weights) still yield useful gradient estimates, partially answering Hinton's objection 1; whether this constitutes "backpropagation" is contested.
- The "update locking" framing — parallel computing literature framing of objection 2: each layer must wait ("lock") for all downstream layers to finish before its weight update, which is a different vocabulary for the same backward-pass sequentiality problem; worth a note mapping the two vocabularies.
- Contrastive Hebbian learning and equilibrium propagation — algorithmic traditions Hinton regards as more biologically plausible; the Forward-Forward paper explicitly references Boltzmann machines as a prototype.
Capture produced: 2026-06-30. All claims from Hinton (2022) arXiv:2212.13345, accessed via ar5iv HTML at https://ar5iv.labs.arxiv.org/html/2212.13345. URL verified live 2026-06-30.