Learning to Believe: Prediction error adaptation as an efficient neurocomputational mechanism for learning about risky outcomes
Abstract: Learning to accurately predict future outcomes facilitates optimal choice behaviour, and correct inferences about the world. Learning typically occurs when we encounter mismatches between our expectations and experiences, called prediction errors. Whereas signed prediction errors indicate the extent to which an outcome is better or worse than expected, unsigned prediction errors signal the degree to which an outcome is unexpected (surprising), independent of its sign. An important determinant of how much we should learn from prediction errors is the amount of reliability, or risk, in the environment. Specifically, there is a trade-off between beliefs being flexible in response to change, yet robust to risk. It has been hypothesised that a breakdown in this trade-off might elicit inaccurate and even odd beliefs as seen in psychosis. To investigate the neural signature of learning about risky outcomes, participants predicted the magnitude of upcoming rewards drawn from distributions with varying standard deviations (i.e., risk). After each prediction, participants received a reward, yielding trial-by-trial prediction errors. In healthy individuals, error-driven learning decreased in more risky environments, as revealed through the use of a combined Rescorla-Wagner --- Pearce-Hall reinforcement learning model. This process was paralleled by midbrain and striatal dopaminergic brain areas that coded signed prediction errors relative to risk, and superior frontal regions coding unsigned prediction errors to risk. Such prediction error adaptation sensitizes the detection of smaller prediction errors when outcome risk is smaller and makes optimal use of the brain's limited encoding capacity. The dopaminergic antagonist sulpiride perturbed the adaptive process, thus suggesting a crucial role for dopamine in this process, in line with previous work in non-human primates. In addition, preliminary work shows that an increase in positive psychotic symptoms in people with early psychosis and those at risk of the condition, is associated with decreased (unsigned) prediction error adaptation, and impaired learning about uncertain outcomes. These findings suggest the presence of a dedicated neurocomputational mechanism for learning about risky outcomes; impairment of which may relate to positive psychotic symptoms.