Parallel Distributed Processing: Implications for Cognition and Development

Abstract : This paper provides a brief overview of the connectionist or parallel distributed processing framework for modeling cognitive processes, and considers the application of the connectionist framework to problems of cognitive development. Several aspects of cognitive development might result from the process of learning as it occurs in multi-layer networks. This learning process has the characteristic that it reduces the discrepancy between expected and observed events. As it does this, representations develop on hidden units which dramatically change both the way in which the network represents the environment from which it learns and the expectations that the network generates about environmental events. The learning process exhibits relatively abrupt transitions corresponding to stage shifts in cognitive development. These points are illustrated using a network that learns to anticipate which side of a balance beam will go down, based on the number of weights on each side of the fulcrum and their distance from the fulcrum on each side of the beam. The network is trained in an environment in which weight more frequently governs which side will go down. It recapitulates the states of development seen in children, as well as the stage transitions, as it learns to represent weight and distance information. Keywords: Parallel processing; Data processing.

[1]  J. Piaget,et al.  The Growth Of Logical Thinking From Childhood To Adolescence: An Essay On The Construction Of Formal Operational Structures , 1958 .

[2]  J. Flavell.  The Developmental psychology of Jean Piaget , 1963 .

[3]  J. Hayes.  Cognition and the development of language , 1970 .

[4]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[5]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[6]  R. Siegler.  Three aspects of cognitive development , 1976, Cognitive Psychology.

[7]  H. F. J. M. Buffart,et al.  Formal theories of visual perception , 1978 .

[8]  D. Klahr,et al.  The representation of children's knowledge. , 1978, Advances in child development and behavior.

[9]  R. Glaser.  Advances in Instructional Psychology , 1978 .

[10]  R. Siegler.  Developmental Sequences within and between Concepts. , 1981 .

[11]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[12]  Charles E. Caton,et al.  Semantic and Conceptual Development: An Ontological Perspective , 1982 .

[13]  Donald A. Norman,et al.  Simulating a Skilled Typist: A Study of Skilled Cognitive-Motor Performance , 1982, Cogn. Sci..

[14]  E. Butterfield,et al.  The classification of children's knowledge: development on the balance-scale and inclined-plane tasks. , 1985, Journal of experimental child psychology.

[15]  James L. McClelland.  Putting Knowledge in its Place: A Scheme for Programming Parallel Processing Structures on the Fly , 1988, Cogn. Sci..

[16]  James L. McClelland,et al.  Distributed memory and the representation of general and specific information. , 1985, Journal of experimental psychology. General.

[17]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation columns. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[19]  James L. McClelland,et al.  James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.

[20]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[21]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[22]  R. J. Williams,et al.  The logic of activation functions , 1986 .

[23]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation-selective cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[24]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[25]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[26]  Lawrence D. Jackel,et al.  Large Automatic Learning, Rule Extraction, and Generalization , 1987, Complex Syst..

[27]  Jeff Shrager,et al.  Observation of Phase Transitions in Spreading Activation Networks , 1987, Science.

[28]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[29]  J. Feldman,et al.  Connectionist models and their implications: readings from cognitive science , 1988 .

[30]  Jerome A. Feldman.  Connectionist representation of concepts , 1988 .

[31]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[32]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[33]  Geoffrey E. Hinton,et al.  Parallel Models of Associative Memory , 1989 .

[34]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[35]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[36]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[37]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .