StereoMath (2024) maps an equation's on-screen position to stereo pan and pitch and replaces spoken parentheses with musical earcons

StereoMath — an accessible, "musical" equation editor by Ge and Seo, presented at ACM ASSETS '24 (arXiv 2501.01404) — encodes the two-dimensional layout of an equation in binaural audio. Two spatial axes map to two audio dimensions: "we modify the audio to match the spatial left-right location" (horizontal position → stereo pan) and "we also change the pitch to match the vertical up/down location" (vertical position → pitch). Reading an expression thus sweeps left-to-right across the stereo field, with exponents and subscripts audibly higher or lower.

Its second move is to abolish spoken structural filler. Where a screen reader would say "lparen … rparen," StereoMath substitutes non-verbal earcons — short musical motifs — chosen, in the authors' words, "inspired by our background in music," to evoke "the feeling of opening/closing a book." Grouping is heard as a gesture rather than spelled out as words, cutting the verbal overhead that makes long expressions hard to hold in working memory.

Both mappings — pitch for verticality, and audio cues standing in for structural delimiters — were already present in Raman's AsTeR (1994); the thirty-year recurrence is treated at observation-math-sonification-pitch-for-vertical-recurred-1994-2024. StereoMath was built through mixed-ability co-design (one collaborator a blind researcher with over 23 years of screen-reader experience), which places it in the pattern that the accessibility layer for math is built by its users — observation-blind-math-accessibility-built-by-users-not-toolmakers — and in the vault's broader blind-mathematics cluster (moc-blind-mathematicians-working-methods).