Do the 1986 Rumelhart-Hinton-Williams Nature paper's actual demonstrations of "useful internal representations" include a two-hidden-unit mirror-symmetry network and a five-layer family-tree network, as claimed by chsasank.com?
Short answer: yes. Both demonstrations chsasank.com attributes to the paper are actually in it, worded almost exactly as the blog post quotes them. This was confirmed by fetching and reading the primary paper itself (Rumelhart, Hinton & Williams, "Learning representations by back-propagating errors," Nature 323, 9 October 1986, pp. 533–536) directly, not by trusting a secondary summary.
Claim: the paper's symmetry-detection demonstration is a network that solves the task using exactly two hidden ("intermediate") units
Claim type: Quantitative (specific unit count) + technical-mechanism (how the solution works). Requires Tier 1–2 per the sourcing floor.
The paper's own body text states: "One simple task that cannot be done by just connecting the input units to the output units is the detection of symmetry. To detect whether the binary activity levels of a one-dimensional array of input units are symmetrical about the centre point, it is essential to use an intermediate layer... The learning procedure discovered an elegant solution using just two intermediate units, as shown in Fig. 1." The Figure 1 caption reads: "A network that has learned to detect mirror symmetry in the input vector. The numbers on the arcs are weights and the numbers inside the nodes are biases. The learning required 1,425 sweeps through the set of 64 possible input vectors..." (2⁶ = 64, indicating a 6-unit binary input array, though the paper does not state the input-unit count as a separate number in prose — this is inferred from the stated count of possible input vectors). The caption also explains the mechanism: "The key property of this solution is that for a given hidden unit, weights that are symmetric about the middle of the input vector are equal in magnitude and opposite in sign. So if a symmetrical pattern is presented, both hidden units will receive a net input of 0 from the input units, and, because the hidden units have a negative bias, both will be off."
Sourcing floor check: Cleared — Tier 1, the primary paper itself, read and quoted directly (not paraphrased from a secondary summary).
| Field | Value |
|---|---|
| source_url | https://gwern.net/doc/ai/nn/1986-rumelhart-2.pdf |
| source_author | David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams |
| source_date | 1986-10-09 |
| source_tier | 1 |
| exact_quote | "The learning procedure discovered an elegant solution using just two intermediate units, as shown in Fig. 1." |
| exact_quote_2 | "A network that has learned to detect mirror symmetry in the input vector." (Fig. 1 caption) |
Claim: the paper's family-tree demonstration is described, in the paper's own figure caption, as a five-layer network
Claim type: Quantitative (specific layer count) + technical-mechanism (architecture). Requires Tier 1–2 per the sourcing floor.
The paper's body text states: "Another interesting task is to store the information in the two family trees (Fig. 2). Figure 3 shows the network we used, and Fig. 4 shows the 'receptive fields' of some of the hidden units after the network was trained on 100 of the 104 possible triples." The Figure 3 caption reads, in full at its opening: "Activity levels in a five-layer network after it has learned. The bottom layer has 24 input units on the left for representing 〈person 1〉 and 12 input units on the right for representing the relationship... Each of the two input groups is totally connected to its own group of 6 units in the second layer... The second layer is totally connected to the central layer of 12 units, and these are connected to the penultimate layer of 6 units. The activity in the penultimate layer must activate the correct output units..." This describes five distinct layers: (1) the input layer (24 person units + 12 relationship units), (2) two groups of 6 encoding units, (3) a central layer of 12 units, (4) a penultimate layer of 6 units, and (5) the output layer — matching the caption's own explicit count of "five-layer."
Sourcing floor check: Cleared — Tier 1, the primary paper itself, read and quoted directly.
| Field | Value |
|---|---|
| source_url | https://gwern.net/doc/ai/nn/1986-rumelhart-2.pdf |
| source_author | David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams |
| source_date | 1986-10-09 |
| source_tier | 1 |
| exact_quote | "Activity levels in a five-layer network after it has learned." (Fig. 3 caption) |
| exact_quote_2 | "Another interesting task is to store the information in the two family trees (Fig. 2). Figure 3 shows the network we used..." |
Claim: chsasank.com's post accurately represents both demonstrations as being in the paper, under its own heading "Demonstrations of Useful Internal Representations"
Claim type: Historical/definitional characterization of a secondary source's content, checked directly against the primary text above (not itself relied on as the source of the underlying facts).
chsasank.com's post (byline: the site is Sasank Chilamkurthy's personal technical blog; the page gives the paper's own publication date as "1986 December" in a citation block, not a separate post date) presents a section titled "Demonstrations of Useful Internal Representations" containing two named examples, "Mirror Symmetry Network" and "Family Tree Network," each supported by quotations lifted verbatim from the same passages quoted above (e.g. "discovered an elegant solution using just two intermediate units" and "store the information in the two family trees"). Neither example is invented or misattributed relative to the primary text: both appear in the actual paper, worded almost identically to how chsasank.com quotes them. One minor imprecision: the paper's own prose never uses the exact phrase "useful internal representations" — chsasank.com's section heading is its own paraphrase of language scattered through the paper's abstract ("internal 'hidden' units... come to represent important features of the task domain") and discussion ("interesting internal representations can be constructed by gradient descent in weight-space"). This is a labeling/framing choice, not a misstatement of what the demonstrations are or how many units/layers they use.
Sourcing floor check: This claim is about what a Tier 3 secondary source says, cross-checked word-for-word against the Tier 1 primary source already quoted above — the underlying facts rest on Tier 1, and the secondary source is being assessed for accuracy rather than treated as an independent evidentiary source.
| Field | Value |
|---|---|
| source_url | https://chsasank.com/classic_papers/learning-representations-back-propogating-errors.html |
| source_author | Sasank Chilamkurthy |
| source_date | retrieved 2026-07-04 |
| source_tier | 3 |
| exact_quote | "Demonstrations of Useful Internal Representations" / "The paper explains the network 'discovered an elegant solution using just two intermediate units'" / "Another interesting task is to store the information in the two family trees" |
Central question status: Confirmed. The 1986 Nature paper's two demonstrations of internal-representation learning are exactly as chsasank.com describes: a two-hidden-unit network for mirror-symmetry detection (Fig. 1) and a five-layer network for the family-tree/kinship task (Fig. 3), both verified directly against the primary paper text.