---
title: "Update locking names backprop's sequential constraint: a layer cannot update until the full forward and backward passes complete"
type: "claim"
status: "budding"
audit_status: "capture-verified (DNI paper primary read at capture level, authors' own venue)"
source_url: "https://arxiv.org/abs/1608.05343"
source_title: "Decoupled Neural Interfaces using Synthetic Gradients"
source_author: "Max Jaderberg, Wojciech Marian Czarnecki, Simon Osindero, et al. (DeepMind)"
source_date: 2016
source_venue: "'Decoupled Neural Interfaces using Synthetic Gradients', ICML 2017"
source_tier: 1
source_quote: "update locking"
provenance: "Promotion from 10-inbox/raw/20260706-1452-what-is-update-locking.md, 2026-07-06, queen cycle 9"
origin: "session"
date_created: "2026-07-06T00:00:00.000Z"
tags: ["update-locking","backpropagation","synthetic-gradients","parallelism","deepmind"]
---


The synthetic-gradients paper formalizes three locks backpropagation imposes:
**forward locking** (a module can't process until its inputs arrive),
**update locking** (a module can't update until the full forward pass AND the
backward pass that depends on it have run), and **backward locking** (it
can't update until [[entity-backpropagation|the backward pass]] reaches it). Update locking is the
tightest: every weight waits on the entire round trip.

This is the *computational* cost of the backward pass, where
[[claim-training-inference-compute-asymmetry-mechanism]] gave the *arithmetic*
cost — the same backward pass that makes training ~3x inference is also what
serializes it. Synthetic gradients break the lock by having each layer
predict its own incoming gradient, decoupling updates. It sits alongside
feedback alignment and Forward-Forward as attacks on backprop's structure,
but from the systems/parallelism angle rather than the biological one — the
lock is why the backward pass is hard to distribute, which connects the
origins cluster to the inference-economics cluster's hardware concerns. See
[[moc-backpropagation-origins]], [[claim-hinton-biological-implausibility-four-objections]].

> [!note] Seek's commentary:
> Line this up with the other backprop critiques and a pattern falls out: every "alternative" in the vault attacks the same single operation — the backward pass — but along three orthogonal axes. The AD cluster asks *is it correct* (yes, it's exact reverse-mode). The biological cluster asks *is it natural* (no — weight transport, no backward phase). This note asks *is it fast* (no — it serializes; every weight waits on the full round trip). Same villain, three unrelated indictments, which is why feedback alignment, Forward-Forward, and synthetic gradients feel like siblings despite solving different problems. The backward pass is the load-bearing thing everyone would remove if only they could.
> — Seek