Uncertainty and Learning

It is a commonplace in statistics that uncertainty about parameters drives learning. Indeed one of the most influential models of behavioural learning has uncertainty at its heart. However, many popular theoretical models of learning focus exclusively on error, and ignore uncertainty. Here we review the links between learning and uncertainty from three perspectives: statistical theories such as the Kalman filter, psychological models in which differential attention is paid to stimuli with an effect on the speed of learning associated with those stimuli, and neurobiological data on the influence of the neuromodulators acetylcholine and norepinephrine on learning and inference.

[1]  Yasuyoshi Watanabe,et al.  Selective suppression of horizontal propagation in rat visual cortex by norepinephrine , 2000, The European journal of neuroscience.

[2]  Jonathan D. Cohen,et al.  Role of locus coeruleus in attention and behavioral flexibility , 1999, Biological Psychiatry.

[3]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[4]  C. Gallistel,et al.  Time, rate, and conditioning. , 2000, Psychological review.

[5]  P. Holland,et al.  Removal of Cholinergic Input to Rat Posterior Parietal Cortex Disrupts Incremental Processing of Conditioned Stimuli , 1998, The Journal of Neuroscience.

[6]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[7]  T. Tsumoto,et al.  Acetylcholine suppresses the spread of excitation in the visual cortex revealed by optical recording: possible differential effect depending on the source of input , 1999, The European journal of neuroscience.

[8]  M. Hasselmo Neuromodulation and cortical function: modeling the physiological basis of behavior , 1995, Behavioural Brain Research.

[9]  Ralph R. Miller,et al.  Biological significance in forward and backward blocking: Resolution of a discrepancy between animal conditioning and human causal judgment , 1996 .

[10]  B. Connors,et al.  Differential Regulation of Neocortical Synapses by Neuromodulators and Activity , 1997, Neuron.

[11]  S. Kakade,et al.  Acquisition and extinction in autoshaping. , 2002, Psychological review.

[12]  Charles D. Kolstad George Bush versus Al Gore : irreversibilities in greenhouse gas accumulation and emission control investment , 1994 .

[13]  Peter Dayan,et al.  Acetylcholine in cortical inference , 2002, Neural Networks.

[14]  A. Dickinson Contemporary Animal Learning Theory , 1981 .

[15]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[16]  S. Sara,et al.  Locus coeruleus-evoked responses in behaving rats: A clue to the role of noradrenaline in memory , 1994, Brain Research Bulletin.

[17]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[18]  Refractor Vision , 2000, The Lancet.

[19]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[20]  G. Aston-Jones,et al.  Locus coeruleus activity in monkey: Phasic and tonic changes are associated with altered vigilance , 1994, Brain Research Bulletin.

[21]  John R. Anderson The Adaptive Character of Thought , 1990 .

[22]  Thomas G. Dietterich,et al.  Editors. Advances in Neural Information Processing Systems , 2002 .

[23]  P. Holland Excitation and inhibition in unblocking. , 1988, Journal of experimental psychology. Animal behavior processes.

[24]  S. Cruikshank,et al.  Differential modulation of auditory thalamocortical and intracortical synaptic transmission by cholinergic agonist , 2000, Brain Research.

[25]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  A. Dickinson Conditioning and associative learning. , 1981, British medical bulletin.

[27]  Catherine E. Myers,et al.  Gateway to Memory: An Introduction to Neural Network Modeling of the Hippocampus and Learning , 2000 .

[28]  M. Gallagher,et al.  Disconnection of the amygdala central nucleus and substantia innominata/nucleus basalis disrupts increments in conditioned stimulus processing in rats. , 1999, Behavioral neuroscience.

[29]  Peter Dayan,et al.  Explaining Away in Weight Space , 2000, NIPS.

[30]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[31]  T. Robbins,et al.  Cortical noradrenaline, attention and arousal , 1984, Psychological Medicine.

[32]  M. Hasselmo,et al.  Suppression of synaptic transmission may allow combination of associative feedback and self-organizing feedforward connections in the neocortex , 1996, Behavioural Brain Research.

[33]  S. Sara,et al.  Response to Novelty and its Rapid Habituation in Locus Coeruleus Neurons of the Freely Exploring Rat , 1995, The European journal of neuroscience.

[34]  J. Kruschke Toward a unified model of attention in associative learning , 2001 .

[35]  M. Hasselmo,et al.  Noradrenergic suppression of synaptic transmission may influence cortical signal-to-noise ratio. , 1997, Journal of neurophysiology.

[36]  S. Sara,et al.  Learning by neurones: role of attention, reinforcement and behaviour. , 1998, Comptes rendus de l'Academie des sciences. Serie III, Sciences de la vie.

[37]  DeLiang Wang,et al.  Unsupervised Learning: Foundations of Neural Computation , 2001, AI Mag..

[38]  J McGaughy,et al.  Lack of effects of lesions of the dorsal noradrenergic bundle on behavioral vigilance. , 1997, Behavioral neuroscience.

[39]  P. Holland Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: Implications for behavior theory , 1997 .

[40]  S. Grossberg,et al.  Pattern Recognition by Self-Organizing Neural Networks , 1991 .

[41]  Peter Dayan,et al.  Statistical Models of Conditioning , 1997, NIPS.

[42]  N. Mackintosh,et al.  Conditioning And Associative Learning , 1983 .

[43]  D. Rasmusson The role of acetylcholine in cortical synaptic plasticity , 2000, Behavioural Brain Research.

[44]  Peter Dayan,et al.  ACh, Uncertainty, and Cortical Inference , 2001, NIPS.

[45]  R. Giering,et al.  Sensitivity Study of Optimal CO2 Emission Paths Using a Simplified Structural Integrated Assessment Model (SIAM) , 1997 .

[46]  S. Kakade,et al.  Learning and selective attention , 2000, Nature Neuroscience.

[47]  B. Anderson,et al.  Optimal Filtering , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[48]  L. Kamin Predictability, surprise, attention, and conditioning , 1967 .

[49]  P. Holland,et al.  Amygdala circuitry in attentional and representational processes , 1999, Trends in Cognitive Sciences.

[50]  Q. Gu,et al.  Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity , 2002, Neuroscience.

[51]  Disconnection of the amygdala central nucleus and substantia innominata/nucleus basalis disrupts increments in conditioned stimulus processing in rats. , 1999 .

[52]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[53]  D. Shanks Forward and Backward Blocking in Human Contingency Judgement , 1985 .

[54]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[55]  R. R. Miller,et al.  Biological significance in forward and backward blocking: resolution of a discrepancy between animal conditioning and human causal judgment. , 1996, Journal of experimental psychology. General.

[56]  Peter Dayan,et al.  Expected and Unexpected Uncertainty: ACh and NE in the Neocortex , 2002, NIPS.

[57]  R. Rescorla A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .