Combining Visual Attention, Object Recognition and Associative Information Processing in a NeuroBotic System

We have implemented a neurobiologically plausible system on a robot that integrates visual attention, object recognition, language and action processing using a coherent cortex-like architecture based on neural associative memories. This system enables the robot to respond to spoken commands like ”bot show plum” or ”bot put apple to yellow cup”. The scenario for this is a robot close to one or two tables carrying certain kinds of fruit and other simple objects. Tasks such as finding and pointing to certain fruits in a complex visual scene according to spoken or typed commands can be demonstrated. This involves parsing and understanding of simple sentences, relating the nouns to concrete objects sensed by the camera, and coordinating motor output with planning and sensory processing.

[1]  Friedhelm Schwenker,et al.  Learning Object Recognition in a NeuroBotic System , 2004 .

[2]  Alessandro Saffiotti,et al.  Anchoring Symbols to Sensor Data: Preliminary Report , 2000, AAAI/IAAI.

[3]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[4]  G. Palm,et al.  Associating words to visually recognized objects ∗ , 2004 .

[5]  Alvy Ray Smith,et al.  Color gamut transform pairs , 1978, SIGGRAPH.

[6]  Friedemann Pulvermüller,et al.  The Neuroscience of Language: On Brain Circuits of Words and Serial Order , 2003 .

[7]  大野 義夫,et al.  Computer Graphics : Principles and Practice, 2nd edition, J.D. Foley, A.van Dam, S.K. Feiner, J.F. Hughes, Addison-Wesley, 1990 , 1991 .

[8]  Gèunther Palm,et al.  Neural Assemblies: An Alternative Approach to Artificial Intelligence , 1982 .

[9]  G. Palm,et al.  On associative memory , 2004, Biological Cybernetics.

[10]  Günther Palm,et al.  Improved bidirectional retrieval of sparse patterns stored by Hebbian learning , 1999, Neural Networks.

[11]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[12]  Richard D. Alexander,et al.  A Comparative Review , 1968 .

[13]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[14]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[15]  Günther Palm,et al.  Iterative retrieval of sparsely coded associative memory patterns , 1996, Neural Networks.

[16]  Stefan Wermter,et al.  Hybrid neural systems: from simple coupling to fully integrated neural networks , 1999 .

[17]  Andreas Knoblauch,et al.  Synchronization and pattern separation in spiking associative memories and visual cortical areas , 2004 .

[18]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[19]  Günther Palm,et al.  SFB 527: Integration symbolischer und subsymbolischer Informationsverarbeitung in adaptiven sensomotorischen Systemen , 1997, GI Jahrestagung.

[20]  Jürgen Schürmann,et al.  Pattern classification , 1996 .

[21]  John E. Howland,et al.  Computer graphics , 1990, IEEE Potentials.

[22]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .

[23]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[24]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[25]  A R Smith,et al.  Color Gamut Transformation Pairs , 1978 .

[26]  David G. Stork,et al.  Pattern Classification , 1973 .

[27]  G. Palm.  Neural Assemblies , 1982, Studies of Brain Function.

[28]  Andreas Knoblauch,et al.  Pattern separation and synchronization in spiking associative memories and visual areas , 2001, Neural Networks.

[29]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[30]  Hans Utz,et al.  Miro - middleware for mobile robot applications , 2002, IEEE Trans. Robotics Autom..

[31]  Antony Browne,et al.  Connectionist inference models , 2001, Neural Networks.

[32]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[33]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[34]  Friedhelm Schwenker,et al.  Three learning phases for radial-basis-function networks , 2001, Neural Networks.