The Pathfinder project rebuilt the same pathology-diagnosis task from rule-based reasoning onto Bayesian-network inference; its expert's blind verdict of improvement came at the Dempster–Shafer-to-simple-Bayes step

AI-historypathfinderbayesian-networksdavid-heckermaneric-horvitzexpert-systemsmedical-diagnosisinference

Pathfinder was a diagnostic expert system for lymph-node pathology, and it is the concrete case in which one team carried a single task across both "inference" senses — the same lymph-node diagnosis solved first by rule-chaining, then by statistical inference. It is a sequence rather than a controlled comparison: the two ends were never measured against each other, and the note's load-bearing quote belongs to a later step (below). Heckerman, Horvitz, and Nathwani's own account describes an early rule-based version that "employed propositional logic for reasoning" and frustrated the collaborating pathologist because "the rule-based methodology generated recommendations for additional observations based on a fixed traversal through the rule base," producing many irrelevant questions — the characteristic rigidity of a classic inference engine walking its rule base in fixed order.

The rebuild was staged, and the stages matter. The rule-based first version was abandoned before any probabilistic comparison — not because it lost a bake-off, but because its fixed traversal asked irrelevant questions and it ignored uncertainty altogether. The team then modelled a second version on QMR, rejected fuzzy decision theory, and built the second working version on a Dempster–Shafer–Barnett scheme, with which the expert was satisfied.

The blind test came at the next step. "Without informing the expert, we switched the scoring scheme of Pathfinder from the Dempster–Shafer–Barnett approach to the simple-Bayes model. To our surprise, after running several cases with the probabilistic scheme, the expert exclaimed excitedly that the diagnostic accuracy of the program had improved significantly." So the expert's unprompted verdict compares two probabilistic-era scoring schemes, not rules against Bayes.

A later formal study confirmed the informal impression and extended it: simple-Bayes gave greater diagnostic accuracy (measured as agreement with the expert) than the Dempster–Shafer–Barnett model, and also greater accuracy than the certainty-factor model. The final version, once graphical representations let the team encode conditional dependencies, was evaluated as "at least as good as that of the Pathfinder expert."

This is a second instance — alongside claim-valtorta-reformulated-ach-as-bayesian-networks — of an AI heuristic method superseded by Bayesian-network inference, and the empirical companion to Pearl's constructive argument (claim-pearl-built-bayesian-networks-as-coherent-alternative-to-certainty-factors) and Heckerman's formal critique (claim-heckerman-1986-certainty-factors-require-independence-assumptions).

Sourcing: the 1992 "Part I" paper is self-hosted by co-author Eric Horvitz (Tier 1 by venue). It could not be receipted at capture time (extract-tool outage), so the before/after account was held at [unverified-mechanism -- needs primary]; the 2026-08-11 audit read the PDF directly and discharged that hold, with all four quoted fragments matching verbatim. The magnitude of the accuracy improvement is explicitly not a number in this paper; the figures live in a separate, still-unlocated companion paper (Heckerman & Nathwani, "An evaluation of the diagnostic accuracy of Pathfinder", ref [50]), so any percentage remains [unverified-quant -- needs primary] and nothing is asserted here beyond "qualitatively, an improvement." See question-verify-pathfinder-rule-based-to-bayes-accuracy.