Exploring the Role of Dopamine in Curiosity-Driven Learning Using Reinforcement Learning Models

Abstract

Curiosity-driven learning is a core feature of biological intelligence, manifested in the active exploration of environments by organisms even in the absence of external rewards. This study integrates neurobiological findings with reinforcement learning theory to construct a unified computational framework that reconceptualizes dopamine as an encoder of "total prediction error." In this framework, behavioral selection maximizes total value, which comprises both external rewards and intrinsic informational value. Intrinsic rewards are quantified based on novelty or prediction error, while the weight assigned to exploration is dynamically modulated by the prefrontal cortex. Dopamine neurons encode total prediction error, thereby extending the conventional role of dopamine from a reward prediction error encoder to a more general encoder of violated expectations. The hippocampus generates intrinsic reward signals, the prefrontal cortex regulates the exploration weight, the ventral tegmental area integrates these signals to compute total prediction error, and the striatum translates this error into action selection. This model bridges the explanatory gap between neuroscience and computational theory, offering a unified perspective on the intrinsic motivation underlying curiosity and exploratory behavior. In practical terms, it also provides new insights into the mechanisms of psychiatric disorders.