A Connectionist Approach to Learn Association between Sentences and Behavioral Patterns of a Robot

We present a novel connectionist model for acquiring the semantics of a simple language through the behavioral experiences of a real robot. We focus on the “compositionality” of semantics, a fundamental characteristic of human language, which is the ability to understand the meaning of a sentence as a combination of the meanings of words. We also pay much attention to the “embodiment” of a robot, which means that the robot should acquire semantics which matches its body, or sensory-motor system. The essential claim is that an embodied compositional semantic representation can be selforganized from generalized correspondences between sentences and behavioral patterns. This claim is examined and confirmed through simpie experiments in which a robot generates corresponding behaviors from unlearned sentences by analogy with the correspondences between learned sentences and behaviors.

[1]  Ferdinand de Saussure.  Course in General Linguistics , 1916 .

[2]  Terry Winograd,et al.  Understanding natural language , 1974 .

[3]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[4]  Stevan Harnad,et al.  Symbol grounding problem , 1990, Scholarpedia.

[5]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[6]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[7]  Robert F. Hadley.  Systematicity revisited : reply to Christiansen and Chater and Niklasson and van Gelder , 1994 .

[8]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[9]  Jun Tani,et al.  Model-based learning for mobile robot navigation from the dynamical systems perspective , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[10]  Raymond J. Mooney,et al.  Semantic Lexicon Acquisition for Learning Natural Language Interfaces , 1998, VLC@COLING/ACL.

[11]  Luc Steels,et al.  The Emergence of Grammar in Communicating Autonomous Robotic Agents , 2000, ECAI.

[12]  Yuuya Sugita.  A Connectionist Model Which Unifies the Behavioral and the Linguistic Processes: Results from Robot , 2000 .

[13]  J. Fodor,et al.  Why Compositionality Won’t Go Away: Reflections on Horwich’s ‘Deflationary’ Theory , 2001 .

[14]  Jeffrey Mark Siskind,et al.  Grounding the Lexical Semantics of Verbs in Visual Perception using Force Dynamics and Event Logic , 2001, J. Artif. Intell. Res..

[15]  S. Kirby,et al.  The emergence of linguistic structure: an overview of the iterated learning model , 2002 .

[16]  Deb K. Roy,et al.  Learning visually grounded words and syntax for a scene description task , 2002, Comput. Speech Lang..

[17]  Aude Billard,et al.  Imitation: a means to enhance learning of a synthetic protolanguage in autonomous robots , 2002 .

[18]  Jun Tani,et al.  Learning to generate articulated behavior through the bottom-up and the top-down interaction processes , 2003, Neural Networks.

[19]  Jun Tani,et al.  Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment , 2003, IEEE Trans. Syst. Man Cybern. Part A.