Processing of Expected and Unexpected Events During Conditioning and Attention: A Psychophysiological Theory

Some recent formal models of Pavlovian and instrumental conditioning contain internal paradoxes that restrict their predictive power. These paradoxes can be traced to an inadequate formulation of how mechanisms of short-term memory and long-term memory work together to control the shifting balance between the processing of expected and unexpected events. Once this formulation is strengthened, a unified processing framework is suggested wherein attentional and orienting subsystems coexist in a complementary relationship that controls the adaptive self-organization of internal representations in response to expected and unexpected events. In this framework, conditioning and attentional constructs can be more directly validated by interdisciplinary paradigms in which seemingly disparate phenomena can be shown to share similar physiological and pharmacological mechanisms. A model of cholinergic-catecholaminergic interactions suggests how drive, reinforcer, and arousal inputs regulate motivational baseline, hysteresis, and rebound, with the hippocampus as a final common path. Extinction, conditioned emotional responses, conditioned avoidance responses, secondary conditioning, and inverted U effects also occur. A similar design in sensory and cognitive representations suggests how short-term memory reset and attentional resonance occur and are related to evoked potentials such as N200, P300, and contingent negative variation (CNV). Competitive feedback properties such as pattern matching, contrast enhancement, and normalization of short-term memory patterns make possible the hypothesis testing procedures that search for arid define new internal representations in response to unexpected events. Longterm memory traces regulate adaptive filtering, expectancy learning, conditioned reinforcer learning, incentive motivational learning, and habit learning. When these mechanisms act together, conditioning phenomena such as overshadowing, unblocking, latent inhibition, overexpectation, and behavioral contrast emerge. Internal Problems of Some Mackintosh (1971), Rescorla and Wagner

[1]  B. Skinner,et al.  Some quantitative properties of anxiety , 1941 .

[2]  H. Janowitz,et al.  Effect of intravenously administered glucose on food intake in the dog. , 1949, The American journal of physiology.

[3]  R. Solomon,et al.  Traumatic avoidance learning: the outcomes of several extinction procedures with dogs. , 1953, Journal of abnormal psychology.

[4]  B. Campbell,et al.  Response strength as a function of drive level and amount of drive reduction. , 1953, Journal of experimental psychology.

[5]  D. Hebb Drives and the C.N.S. (conceptual nervous system). , 1955, Psychological review.

[6]  D. Elkind,et al.  Evidence for the interaction of drive and reward. , 1958, The American journal of psychology.

[7]  J. P. Seward,et al.  Further evidence for the interaction of drive and reward. , 1960, The American journal of psychology.

[8]  J. P. Seward,et al.  Performance as a function of drive, reward, and habit-strength. , 1960, The American journal of psychology.

[9]  N. Miller,et al.  Learning resistance to pain and fear: effects of overlearning, exposure, and rewarded expsure in context. , 1960, Journal of experimental psychology.

[10]  F. Morrell,et al.  Electrophysiological contributions to the neural basis of learning. , 1961, Physiological reviews.

[11]  R. Held Exposure-history as a factor in maintaining stability of perception and coordination. , 1961, The Journal of nervous and mental disease.

[12]  Richard Held,et al.  Exposure-history as a factor in maintaining stability of perception and coordination. , 1961, The Journal of nervous and mental disease.

[13]  B. Anand,et al.  Role of stomach in regulation of activities of hypothalamic feeding centers. , 1961, The American journal of physiology.

[14]  H. Wallach,et al.  Modification of stereoscopic depth-perception. , 1963, The American journal of psychology.

[15]  J. Keehn,et al.  Effect of Shock Duration on Sidman Avoidance Response Rates , 1963 .

[16]  R. Held,et al.  MOVEMENT-PRODUCED STIMULATION IN THE DEVELOPMENT OF VISUALLY GUIDED BEHAVIOR. , 1963, Journal of comparative and physiological psychology.

[17]  H. Wallach,et al.  The modification of stereoscopic depth-perception and the kinetic depth-effect. , 1963, The American journal of psychology.

[18]  H. Wallach,et al.  Why the modification of stereoscopic depth-perception is so rapid. , 1963, The American journal of psychology.

[19]  W. E. Broen,et al.  RESPONSE SUPPRESSION AND RECOVERY OF RESPONDING AT DIFFERENT DEPRIVATION LEVELS AS FUNCTIONS OF INTENSITY AND DURATION OF PUNISHMENT. , 1964, Journal of comparative and physiological psychology.

[20]  A STROUTHES,et al.  EFFECT OF CS-ONSET UCS-TERMINATION DELAY, UCS DURATION, CS-ONSET UCS-ONSET INTERVAL, AND NUMBER OF CS-UCS PAIRINGS ON CONDITIONED FEAR RESPONSE. , 1965, Journal of experimental psychology.

[21]  R. Rescorla,et al.  INHIBITION OF AVOIDANCE BEHAVIOR. , 1965, Journal of comparative and physiological psychology.

[22]  M. R. Baron,et al.  STIMULUS GENERALIZATION ALONG THE DIMENSION OF ANGULARITY: A COMPARISON OF TRAINING PROCEDURES. , 1965, Journal of comparative and physiological psychology.

[23]  C. H. Graham,et al.  Vision and visual perception , 1965 .

[24]  E. E. Boe,et al.  Effect of punishment duration and intensity on the extinction of an instrumental response. , 1966, Journal of experimental psychology.

[25]  R. Russell Frontiers in physiological psychology , 1966 .

[26]  E. Roy John,et al.  Mechanisms of memory , 1967 .

[27]  B. Anand,et al.  Activity of single neurones in the hypothalamic feeding centres: effect of gastric distension , 1967, The Journal of physiology.

[28]  L. Kamin Predictability, surprise, attention, and conditioning , 1967 .

[29]  S. Grossberg Nonlinear difference-differential equations in prediction and learning theory. , 1967, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Liberson Cw,et al.  Eye movements during the alert state (psychophysiological correlations under resting conditions). , 1967 .

[31]  R. Bolles Theory of Motivation , 1967 .

[32]  R M Church,et al.  Response suppression as a function of intensity and duration of a punishment. , 1967, Journal of comparative and physiological psychology.

[33]  L. Kamin Attention-like processes in classical conditioning , 1967 .

[34]  Hammond Lj,et al.  Retardation of fear acquisition by a previously inhibitory CS. , 1968 .

[35]  A. Routtenberg The two-arousal hypothesis: reticular formation and limbic system. , 1968, Psychological review.

[36]  R. Held Dissociation of visual functions by deprivation and rearrangement , 1968 .

[37]  S. Grossberg,et al.  Some physiological and biochemical consequences of psychological postulates. , 1968, Proceedings of the National Academy of Sciences of the United States of America.

[38]  R. Weisman,et al.  The course of Pavlovian excitation and inhibition of fear in rats. , 1969, Journal of comparative and physiological psychology.

[39]  P. Dunham,et al.  Enhancement of off-key pecking by on-key punishment. , 1969, Journal of the experimental analysis of behavior.

[40]  B. Campbell,et al.  Punishment and aversive behavior , 1969 .

[41]  S. Grossberg Some Networks That Can Learn, Remember, and Reproduce any Number of Complicated Space-Time Patterns, I , 1969 .

[42]  Stephen Grossberg,et al.  On the serial learning of lists , 1969 .

[43]  R. Rescorla,et al.  Establishment of a positive reinforcer through contrast with shock. , 1969, Journal of comparative and physiological psychology.

[44]  Dexter Wr,et al.  Role of contextual discrimination in fear conditioning. , 1969 .

[45]  Stephen Grossberg,et al.  Embedding fields: A theory of learning with physiological implications , 1969 .

[46]  D. E. Berlyne,et al.  CHAPTER 7 – The Reward-Value of Indifferent Stimulation1 , 1969 .

[47]  F. Masterson Is termination of a warning signal an effective reward for the rat , 1970 .

[48]  N. J. Mackintosh,et al.  An Analysis of Overshadowing and Blocking , 1971 .

[49]  S. Grossberg,et al.  Spiking threshold and overarousal effects in serial learning , 1971 .

[50]  S Grossberg,et al.  On the dynamics of operant conditioning. , 1971, Journal of theoretical biology.

[51]  P. Dunham Punishment: Method and Theory. , 1971 .

[52]  M. Denny,et al.  Relaxation Theory and Experiments , 1971 .

[53]  A. Black,et al.  Aversive conditioning and learning , 1971 .

[54]  Klaus Schmitt,et al.  Delay and functional differential equations and their applications : [proceedings] , 1972 .

[55]  J. Magnen Regulation of Food Intake , 1972 .

[56]  Stephen Grossberg,et al.  A Neural Theory of Punishment and Avoidance, II: Quantitative Theory , 1972 .

[57]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[58]  M. Seligman,et al.  Biological boundaries of learning. , 1972 .

[59]  Hunger and satiety in health and disease , 1972 .

[60]  J. W. Rudy,et al.  Rehearsal in animal conditioning. , 1973, Journal of experimental psychology.

[61]  J. C. Johnston,et al.  A cognitive theory of avoidance learning. , 1973 .

[62]  Stephen Grossberg,et al.  Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks , 1973 .

[63]  P. W. Frey,et al.  Inhibition and learning , 1973 .

[64]  Alma E. Lantz Effect of number of trials, interstimulus interval, and dishabituation during CS habituation on subsequent conditioning in a CER paradigm , 1973 .

[65]  F. J. McGuigan,et al.  Contemporary approaches to conditioning and learning , 1973 .

[66]  A. Friedhoff Catecholamines and Behavior , 1975 .

[67]  S Grossberg,et al.  Some developmental and attentional biases in the contrast enhancement and short term memory of recurrent neural networks. , 1975, Journal of theoretical biology.

[68]  N. Mackintosh A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[69]  S. Grossberg A neural model of attention, reinforcement and discrimination learning. , 1975, International review of neurobiology.

[70]  A. N. Epstein,et al.  The Neuropsychology of thirst : new findings and advances in concepts , 1975 .

[71]  Donald O. Walter,et al.  Mass action in the nervous system , 1975 .

[72]  S. Grossberg,et al.  Visual illusions in neural networks: line neutralization, tilt after effect, and angle expansion. , 1976, Journal of theoretical biology.

[73]  N. Mackintosh Overshadowing and stimulus intensity , 1976, Animal learning & behavior.

[74]  R. E. Lubow,et al.  The Context Effect: The Relationship Between Stimulus Preexposure and Environmental Preexposure Determines Subsequent Learning. , 1976 .

[75]  N. Mackintosh,et al.  Surprise and the attenuation of blocking. , 1976 .

[76]  N. Mackintosh,et al.  Locus of the Effect of a Surprising Reinforcer in the Attenuation of Blocking , 1977 .

[77]  W Singer,et al.  Control of thalamic transmission by corticofugal and ascending reticular pathways in the visual system. , 1977, Physiological reviews.

[78]  L. Nadel,et al.  The Hippocampus as a Cognitive Map , 1978 .

[79]  P. W. Frey,et al.  Model of conditioning incorporating the Rescorla-Wagner associative axiom, a dynamic attention process, and a catastrophe rule. , 1978 .

[80]  J. Hinson,et al.  Behavioral competition: a mechanism for schedule interactions. , 1978, Science.

[81]  L. Butcher Cholinergic-monoaminergic interactions in the brain , 1978 .

[82]  S. Grossberg Behavioral Contrast in Short Term Memory: Serial Binary Memory Models or Parallel Continuous Memory Models? , 1978 .

[83]  J. Pearce,et al.  Latent inhibition of a CS during CS-US pairings. , 1979, Journal of experimental psychology. Animal behavior processes.

[84]  N. Mackintosh,et al.  One-Trial Overshadowing , 1979 .

[85]  J. Gibson The Ecological Approach to Visual Perception , 1979 .

[86]  N. Mackintosh,et al.  Reinforcer specificity in the enhancement of conditioning by posttrial surprise. , 1979 .

[87]  E. Kremer Effect of posttrial episodes on conditioning in compound conditioned stimuli. , 1979, Journal of experimental psychology. Animal behavior processes.

[88]  W. K. Honig,et al.  Cognitive Processes in Animal Behavior , 1979 .

[89]  Eric L. Schwartz,et al.  Computational anatomy and functional architecture of striate cortex: A spatial mapping approach to perceptual coding , 1980, Vision Research.

[90]  S. Ullman Against direct perception , 1980, Behavioral and Brain Sciences.

[91]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[92]  R. Näätänen,et al.  Magnitude of stimulus deviance and brain potentials. , 1980, Progress in brain research.

[93]  S. Grossberg How does a brain build a cognitive code , 1980 .

[94]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[95]  S. Grossberg Mathematical Psychology and Psychophysiology , 1982 .

[96]  S. Grossberg Pattern Learning by Functional-Differential Neural Networks with Arbitrary Path Weights , 1982 .

[97]  S. Grossberg Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[98]  Stephen Grossberg,et al.  Classical and Instrumental Learning by Neural Networks , 1982 .

[99]  Stephen Grossberg,et al.  A Theory of Human Memory: Self-Organization and Performance of Sensory-Motor Codes, Maps, and Plans , 1982 .