Note to users. If you're seeing this message, it means that your browser cannot find this page's style/presentation instructions -- or possibly that you are using a browser that does not support current Web standards. Find out more about why this message is appearing, and what you can do to make your experience of our site the best it can be.
Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
Christopher D. Fiorillo,*Philippe N. Tobler,Wolfram Schultz
Uncertainty is critical in the measure of information
and in assessing the accuracy of predictions. It is determined by
probabilityP, being maximal at P = 0.5 and
decreasing at higher and lowerprobabilities. Using distinct stimuli to
indicate the probabilityof reward, we found that the phasic activation
of dopamine neuronsvaried monotonically across the full range of
probabilities, supportingpast claims that this response codes the
discrepancy between predictedand actual reward. In contrast, a
previously unobserved responsecovaried with uncertainty and consisted
of a gradual increasein activity until the potential time of reward.
The coding ofuncertainty suggests a possible role for dopamine signals
in attention-basedlearning and risk-taking behavior.
Institute of Physiology, University of Fribourg, CH-1700 Fribourg,
Switzerland, and Department of Anatomy, University of Cambridge,
Downing Street, Cambridge CB2 3DY, UK.
*
To whom correspondence should be addressed. E-mail:
cdf28{at}cam.ac.uk
The editors suggest the following Related Resources on Science sites:
M. Ishikawa, M. Otaka, Y. H. Huang, P. A. Neumann, B. D. Winters, A. A. Grace, O. M. Schluter, and Y. Dong (2013)
J. Neurosci.
33, 6759-6765
|Abstract »|Full Text »|PDF »
Action controls dopaminergic enhancement of reward representations.
M. Guitart-Masip, R. Chowdhury, T. Sharot, P. Dayan, E. Duzel, and R. J. Dolan (2012)
PNAS
109, 7511-7516
|Abstract »|Full Text »|PDF »
Apathy: A Complex Symptom Specific to the Clinical Pattern of Presentation of Parkinson's Disease?.
R. Moretti, P. Torre, R. M. Antonello, M. V. Rosin, F. Esposito, M. R. Furman, and G. Bellini (2012)
American Journal of Alzheimer's Disease and Other Dementias
27, 196-201
|Abstract »|PDF »
Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequences.
S. L. Cowen, G. A. Davis, and D. A. Nitz (2012)
J Neurophysiol
107, 2393-2407
|Abstract »|Full Text »|PDF »
Cocaine sensitization inhibits the hyperpolarization-activated cation current Ih and reduces cell size in dopamine neurons of the ventral tegmental area.
F. Arencibia-Albite, R. Vazquez, M. C. Velasquez-Martinez, and C. A. Jimenez-Rivera (2012)
J Neurophysiol
107, 2271-2282
|Abstract »|Full Text »|PDF »
Frontal theta and beta synchronizations for monetary reward increase visual working memory capacity.
Attenuating GABAA Receptor Signaling in Dopamine Neurons Selectively Enhances Reward Learning and Alters Risk Preference in Mice.
J. G. Parker, M. J. Wanat, M. E. Soden, K. Ahmad, L. S. Zweifel, N. S. Bamford, and R. D. Palmiter (2011)
J. Neurosci.
31, 17103-17112
|Abstract »|Full Text »|PDF »
Behavioral and Neural Properties of Social Reinforcement Learning.
R. M. Jones, L. H. Somerville, J. Li, E. J. Ruberry, V. Libby, G. Glover, H. U. Voss, D. J. Ballon, and B. J. Casey (2011)
J. Neurosci.
31, 13039-13045
|Abstract »|Full Text »|PDF »
Dopamine neurons learn to encode the long-term value of multiple future rewards.
K. Enomoto, N. Matsumoto, S. Nakai, T. Satoh, T. K. Sato, Y. Ueda, H. Inokawa, M. Haruno, and M. Kimura (2011)
PNAS
108, 15462-15467
|Abstract »|Full Text »|PDF »
Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis.
Optogenetic Interrogation of Dopaminergic Modulation of the Multiple Phases of Reward-Seeking Behavior.
A. R. Adamantidis, H.-C. Tsai, B. Boutrel, F. Zhang, G. D. Stuber, E. A. Budygin, C. Tourino, A. Bonci, K. Deisseroth, and L. de Lecea (2011)
J. Neurosci.
31, 10829-10835
|Abstract »|Full Text »|PDF »
Electroencephalographic Topography Measures of Experienced Utility.
A. Pedroni, N. Langer, T. Koenig, M. Allemand, and L. Jancke (2011)
J. Neurosci.
31, 10474-10480
|Abstract »|Full Text »|PDF »
Dorsolateral Prefrontal Cortex Drives Mesolimbic Dopaminergic Regions to Initiate Motivated Behavior.
I. C. Ballard, V. P. Murty, R. M. Carter, J. J. MacInnes, S. A. Huettel, and R. A. Adcock (2011)
J. Neurosci.
31, 10340-10346
|Abstract »|Full Text »|PDF »
The placebo response in clinical trials: more questions than answers.
P. Enck, S. Klosterhalfen, K. Weimer, B. Horing, and S. Zipfel (2011)
Phil Trans R Soc B
366, 1889-1895
|Abstract »|Full Text »|PDF »
High-frequency, short-latency disinhibition bursting of midbrain dopaminergic neurons.
C. J. Lobb, C. J. Wilson, and C. A. Paladini (2011)
J Neurophysiol
105, 2501-2511
|Abstract »|Full Text »|PDF »
Dopamine agonists and risk: impulse control disorders in Parkinson's; disease.
V. Voon, J. Gao, C. Brezing, M. Symmonds, V. Ekanayake, H. Fernandez, R. J. Dolan, and M. Hallett (2011)
Brain
134, 1438-1446
|Abstract »|Full Text »|PDF »
Behavioral specifications of reward-associated long-term memory enhancement in humans.
Dopaminergic modulation of the human reward system: a placebo-controlled dopamine depletion fMRI study.
F. da Silva Alves, N. Schmitz, M. Figee, N. Abeling, G. Hasler, J. van der Meer, A. Nederveen, L. de Haan, D. Linszen, and T. van Amelsvoort (2011)
J Psychopharmacol
25, 538-549
|Abstract »|Full Text »|PDF »
Risk preference following adolescent alcohol use is associated with corrupted encoding of costs but not rewards by mesolimbic dopamine.
N. A. Nasrallah, J. J. Clark, A. L. Collins, C. A. Akers, P. E. Phillips, and I. L. Bernstein (2011)
PNAS
108, 5466-5471
|Abstract »|Full Text »|PDF »
Surprise Signals in Anterior Cingulate Cortex: Neuronal Encoding of Unsigned Reward Prediction Errors Driving Adjustment in Behavior.
B. Y. Hayden, S. R. Heilbronner, J. M. Pearson, and M. L. Platt (2011)
J. Neurosci.
31, 4178-4187
|Abstract »|Full Text »|PDF »
Double-Dissociation of the Catecholaminergic Modulation of Synaptic Transmission in the Oval Bed Nucleus of the Stria Terminalis.
M. Krawczyk, F. Georges, R. Sharma, X. Mason, A. Berthet, E. Bezard, and E. C. Dumont (2011)
J Neurophysiol
105, 145-153
|Abstract »|Full Text »|PDF »
Reward Magnitude Coding in Primate Amygdala Neurons.
Delays Conferred by Escalating Costs Modulate Dopamine Release to Rewards But Not Their Predictors.
M. J. Wanat, C. M. Kuhnen, and P. E. M. Phillips (2010)
J. Neurosci.
30, 12020-12027
|Abstract »|Full Text »|PDF »
Reward Prediction Error Coding in Dorsal Striatal Neurons.
K. Oyama, I. Hernadi, T. Iijima, and K.-I. Tsutsui (2010)
J. Neurosci.
30, 11447-11457
|Abstract »|Full Text »|PDF »
Temporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli.
K. Nomoto, W. Schultz, T. Watanabe, and M. Sakagami (2010)
J. Neurosci.
30, 10692-10702
|Abstract »|Full Text »|PDF »
The Effects of COMT (Val108/158Met) and DRD4 (SNP -521) Dopamine Genotypes on Brain Activations Related to Valence and Magnitude of Rewards.
E. Camara, U. M. Kramer, T. Cunillera, J. Marco-Pallares, D. Cucurell, W. Nager, A. Mestres-Misse, P. Bauer, R. Schule, L. Schols, et al. (2010)
Cereb Cortex
20, 1985-1996
|Abstract »|Full Text »|PDF »
Tradeoffs and Constraints on Neural Representation in Networks of Cortical Neurons.
E. Kermany, A. Gal, V. Lyakhov, R. Meir, S. Marom, and D. Eytan (2010)
J. Neurosci.
30, 9588-9596
|Abstract »|Full Text »|PDF »
A new tool for your novelty centre.
B. Liss and J. Roeper (2010)
J. Physiol.
588, 2013
|Full Text »|PDF »
Learning and Reversal Learning in the Subcortical Limbic System: A Computational Model.
A. M. Thompson, B. Porr, and F. Worgotter (2010)
Adaptive Behavior
18, 211-236
|Abstract »|PDF »
Contextual Novelty Changes Reward Representations in the Striatum.
M. Guitart-Masip, N. Bunzeck, K. E. Stephan, R. J. Dolan, and E. Duzel (2010)
J. Neurosci.
30, 1721-1726
|Abstract »|Full Text »|PDF »
Adaptation of Reward Sensitivity in Orbitofrontal Neurons.
S. Kobayashi, O. Pinto de Carvalho, and W. Schultz (2010)
J. Neurosci.
30, 534-544
|Abstract »|Full Text »|PDF »
At-risk for pathological gambling: imaging neural reward processing under chronic dopamine agonists.
B. Abler, R. Hahlbrock, A. Unrath, G. Gron, and J. Kassubek (2009)
Brain
132, 2396-2402
|Abstract »|Full Text »|PDF »
Single-Cell and Population Coding of Expected Reward Probability in the Orbitofrontal Cortex of the Rat.
E. van Duuren, G. van der Plasse, J. Lankelma, R. N. J. M. A. Joosten, M. G. P. Feenstra, and C. M. A. Pennartz (2009)
J. Neurosci.
29, 8965-8976
|Abstract »|Full Text »|PDF »
Functional Dissociations of Risk and Reward Processing in the Medial Prefrontal Cortex.
G. Xue, Z. Lu, I. P. Levin, J. A. Weller, X. Li, and A. Bechara (2009)
Cereb Cortex
19, 1019-1027
|Abstract »|Full Text »|PDF »
Stimulus-outcome learnability differentially activates anterior cingulate and hippocampus at feedback processing.
W. Schultz, K. Preuschoff, C. Camerer, M. Hsu, C. D Fiorillo, P. N Tobler, and P. Bossaerts (2008)
Phil Trans R Soc B
363, 3801-3811
|Abstract »|Full Text »|PDF »
Midbrain Dopaminergic Neurons and Striatal Cholinergic Interneurons Encode the Difference between Reward and Aversive Events at Different Epochs of Probabilistic Classical Conditioning Trials.
M. Joshua, A. Adler, R. Mitelman, E. Vaadia, and H. Bergman (2008)
J. Neurosci.
28, 11673-11684
|Abstract »|Full Text »|PDF »
A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward.
BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area.
K. D'Ardenne, S. M. McClure, L. E. Nystrom, and J. D. Cohen (2008)
Science
319, 1264-1267
|Abstract »|Full Text »|PDF »
Neural correlates of trust.
F. Krueger, K. McCabe, J. Moll, N. Kriegeskorte, R. Zahn, M. Strenziok, A. Heinecke, and J. Grafman (2007)
PNAS
104, 20084-20089
|Abstract »|Full Text »|PDF »
History- and Current Instruction-Based Coding of Forthcoming Behavioral Outcomes in the Striatum.
H. Yamada, N. Matsumoto, and M. Kimura (2007)
J Neurophysiol
98, 3557-3567
|Abstract »|Full Text »|PDF »
Lateral Habenula Stimulation Inhibits Rat Midbrain Dopamine Neurons through a GABAA Receptor-Mediated Mechanism.
Processing of Temporal Unpredictability in Human and Animal Amygdala.
C. Herry, D. R. Bach, F. Esposito, F. Di Salle, W. J. Perrig, K. Scheffler, A. Luthi, and E. Seifritz (2007)
J. Neurosci.
27, 5958-5966
|Abstract »|Full Text »|PDF »
Gene gene interaction associated with neural reward sensitivity.
J. Yacubian, T. Sommer, K. Schroeder, J. Glascher, R. Kalisch, B. Leuenberger, D. F. Braus, and C. Buchel (2007)
PNAS
104, 8125-8130
|Abstract »|Full Text »|PDF »
Differential Encoding of Losses and Gains in the Human Striatum.
B. Seymour, N. Daw, P. Dayan, T. Singer, and R. Dolan (2007)
J. Neurosci.
27, 4826-4831
|Abstract »|Full Text »|PDF »
From Preferences to Experiences: Valuing the Intangible Victim Costs of Crime.
P. Dolan and S. C. Moore (2007)
International Review of Victimology
14, 265-280
|Abstract »|PDF »
From prediction error to psychosis: ketamine as a pharmacological model of delusions.
The Nucleus Accumbens and Pavlovian Reward Learning.
J. J. Day and R. M. Carelli (2007)
Neuroscientist
13, 148-159
|Abstract »|PDF »
Adaptation of Prefrontal Cortical Firing Patterns and Their Fidelity to Changes in Action-Reward Contingencies.
W. J. Kargo, B. Szatmary, and D. A. Nitz (2007)
J. Neurosci.
27, 3548-3559
|Abstract »|Full Text »|PDF »
Menstrual cycle phase modulates reward-related neural function in women.
J.-C. Dreher, P. J. Schmidt, P. Kohn, D. Furman, D. Rubinow, and K. F. Berman (2007)
PNAS
104, 2465-2470
|Abstract »|Full Text »|PDF »
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems.
P. N. Tobler, J. P. O'Doherty, R. J. Dolan, and W. Schultz (2007)
J Neurophysiol
97, 1621-1632
|Abstract »|Full Text »|PDF »
Shaping of Motor Responses by Incentive Values through the Basal Ganglia.
B. Pasquereau, A. Nadjar, D. Arkadir, E. Bezard, M. Goillandeau, B. Bioulac, C. E. Gross, and T. Boraud (2007)
J. Neurosci.
27, 1176-1183
|Abstract »|Full Text »|PDF »