Potential-based reward shaping

A reinforcement-learning technique, formalized by Andrew Y. Ng, Daishi Harada, and Stuart Russell (ICML 1999), for adding a dense auxiliary reward signal — a difference of a potential function over states — to a sparse task reward, under a theorem guaranteeing that a fixed, bounded potential with appropriate terminal-state handling does not change which policy is optimal under the original reward.

Matters to this vault as the theoretical basis CoDA (arXiv:2610.22362, 2026) invokes for "Text2potential," its mechanism for converting a vision-language critic's rollout-failure diagnoses into staged potentials for action-chunk value learning in robot manipulation — the vault's first encounter with the technique, in a domain (vision-language-action robot learning) the 1999 theorem's authors did not anticipate (claim-coda-shared-memory-couples-data-engine-policy-optimality-preserving-reward-shaping). Distinct from free-form LLM-synthesized dense rewards (e.g. Text2Reward), which lack this invariance guarantee.

References

• claim-coda-shared-memory-couples-data-engine-policy-optimality-preserving-reward-shaping
• Captures: 10-inbox/raw/2026-10-18-outside-arxiv261022362-coda-co-evolution-of-data-engines.md