Humanoid learns to detect its own hands

Robust object manipulation is still a hard problem in robotics, even more so in high degree-of-freedom (DOF) humanoid robots. To improve performance a closer integration of visual and motor systems is needed. We herein present a novel method for a robot to learn robust detection of its own hands and fingers enabling sensorimotor coordination. It does so solely using its own camera images and does not require any external systems or markers. Our system based on Cartesian Genetic Programming (CGP) allows to evolve programs to perform this image segmentation task in real-time on the real hardware. We show results for a Nao and an iCub humanoid each detecting its own hands and fingers.

[1]  Mathias Kölsch,et al.  Robust hand detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[2]  TomasM art ´ inek.  An Evolvable Image Filter: Experimental Evaluation of a Complete Hardware Implementation in FPGA , 2005 .

[3]  Julian F. Miller,et al.  A Survey of Self Modifying Cartesian Genetic Programming , 2011 .

[4]  Shinji Miyazaki,et al.  Comparison of the performance of 3D camera systems , 1995 .

[5]  Alexander H. Waibel,et al.  Segmenting hands of arbitrary color , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[6]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[7]  Javier A. Montoya-Zegarra,et al.  Fast interactive segmentation of natural images using the image foresting transform , 2009, 2009 16th International Conference on Digital Signal Processing.

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[10]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[11]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[12]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[13]  Tomoharu Nagao,et al.  Feed Forward Genetic Image Network: Toward Efficient Automatic Construction of Image Processing Algorithm , 2007, ISVC.

[14]  Jürgen Leitner,et al.  Mars terrain image classification using Cartesian genetic programming , 2012 .

[15]  Simon Harding,et al.  Evolution of image filters on graphics processor units using Cartesian Genetic Programming , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[16]  Jürgen Leitner,et al.  Cartesian Genetic Programming for Image Processing , 2013 .

[17]  Nikolaos G. Tsagarakis,et al.  iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research , 2007, Adv. Robotics.

[18]  Mark Lee,et al.  Robotic hand-eye coordination without global reference: A biologically inspired learning scheme , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[19]  Wolfgang Banzhaf,et al.  Image Processing and CGP , 2011, Cartesian Genetic Programming.

[20]  Wolfgang Banzhaf,et al.  Genetic programming on GPUs for image processing , 2008, Int. J. High Perform. Syst. Archit..

[21]  J. Miller.  An empirical study of the efficiency of learning boolean functions using a Cartesian Genetic Programming approach , 1999 .

[22]  Matthew Turk,et al.  View-based interpretation of real-time optical flow for gesture recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[23]  Julian Francis Miller,et al.  Cartesian genetic programming , 2010, GECCO.

[24]  B. Matthews.  Comparison of the predicted and observed secondary structure of T4 phage lysozyme. , 1975, Biochimica et biophysica acta.

[25]  Giulio Sandini,et al.  The iCub humanoid robot: An open-systems platform for research in cognitive development , 2010, Neural Networks.

[26]  Jürgen Leitner,et al.  MT-CGP: mixed type cartesian genetic programming , 2012, GECCO '12.

[27]  Julian Francis Miller,et al.  Cartesian genetic programming , 2000, GECCO '10.

[28]  Wolfgang Banzhaf,et al.  Distributed genetic programming on GPUs using CUDA , 2011 .

[29]  Dario Izzo,et al.  The Generalized Island Model , 2012, Parallel Architectures and Bioinspired Algorithms.

[30]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[31]  M A Goodale,et al.  Visuomotor control: Where does vision end and action begin? , 1998, Current Biology.

[32]  William B. Langdon,et al.  Evolving Hand-Eye Coordination for a Humanoid Robot with Machine Code Genetic Programming , 2001, EuroGP.

[33]  Jürgen Leitner,et al.  An Integrated, Modular Framework for Computer Vision and Cognitive Robotics Research (icVision) , 2012, BICA.

[34]  Julian Francis Miller,et al.  Redundancy and computational efficiency in Cartesian genetic programming , 2006, IEEE Transactions on Evolutionary Computation.

[35]  R. Paul.  Robot Manipulators: Mathematics, Programming, and Control , 1981 .

[36]  Giorgio Metta,et al.  Online multiple instance learning applied to hand detection in a humanoid robot , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.