Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components

We describe RTblob, a high speed vision system that detects objects in cluttered scenes based on their color and shape at a speed of over 800 frames/s. Because the system is available as open-source software and relies only on off-the-shelf PC hardware components, it can provide the basis for multiple application scenarios. As an illustrative example, we show how RTblob can be used in a robotic table tennis scenario to estimate ball trajectories through 3D space simultaneously from four cameras images at a speed of 200 Hz.

[1]  Masatoshi Ishikawa,et al.  Tweezers type tool manipulation by a multifingered hand using a high-speed visusal servoing , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Hubert Nguyen,et al.  GPU Gems 3 , 2007 .

[3]  Éric Marchand,et al.  Statistically robust 2-D visual servoing , 2006, IEEE Transactions on Robotics.

[4]  Masatoshi Ishikawa,et al.  955-fps Real-time Shape Measurement of a Moving/Deforming Object using High-speed Vision for Numerous-point Analysis , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[5]  Christoph H. Lampert,et al.  Active Structured Learning for High-Speed Object Detection , 2009, DAGM-Symposium.

[6]  Ian Craw,et al.  Tracking and measuring drivers' eyes , 1995, Image Vis. Comput..

[7]  Yves Jean,et al.  Real time tracking for enhanced tennis broadcasts , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[8]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[9]  David W. Murray,et al.  Real-time localization and mapping with wearable active vision , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[10]  Franklin C. Crow,et al.  Summed-area tables for texture mapping , 1984, SIGGRAPH.

[11]  Guy E. Blelloch,et al.  Prefix sums and their applications , 1990 .

[12]  Bernd Jähne,et al.  Digital image processing (3rd ed.): concepts, algorithms, and scientific applications , 1995 .

[13]  Günter Schreiber,et al.  Off-the-shelf vision for a robotic ball catcher , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[14]  Ian Craw,et al.  Tracking and Measuring Drivers Eyes , 1995, BMVC.

[15]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Ian D. Reid,et al.  Real-Time SLAM Relocalisation , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Qingshan Liu,et al.  An effective and fast soccer ball detection and tracking method , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[19]  Shinichi Hirai,et al.  CMOS+FPGA vision system for visual feedback of mechanical systems , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[20]  Kosuke Sato,et al.  Real-time gesture recognition by learning and selective control of visual interest points , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Andrew Zisserman,et al.  Multiple View Geometry , 2009, Encyclopedia of Biometrics.

[22]  Ulrich Muehlmann,et al.  A new high speed cmos camera for real-time tracking applications , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[23]  Thomas Kalinke,et al.  An image processing system for driver assistance , 2000, Image Vis. Comput..

[24]  J. P. Lewis,et al.  Fast Template Matching , 2009 .

[25]  Roberto Brunelli,et al.  MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES , 2001 .

[26]  Yoshiaki Shirai,et al.  An active vision system for real-time traffic sign recognition , 2000, ITSC2000. 2000 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.00TH8493).

[27]  Masatoshi Ishikawa,et al.  1 ms column parallel vision system and its application of high speed target tracking , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[28]  Shai Avidan,et al.  Support Vector Tracking , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  PoggioTomaso,et al.  Example-Based Learning for View-Based Human Face Detection , 1998 .

[30]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Mark J. Harris,et al.  Parallel Prefix Sum (Scan) with CUDA , 2011 .

[32]  Cataldo Guaragnella,et al.  A new algorithm for ball recognition using circle Hough transform and neural classifier , 2004, Pattern Recognit..

[33]  G. Turin,et al.  An introduction to matched filters , 1960, IRE Trans. Inf. Theory.

[34]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Stefan Hinz,et al.  Fast and subpixel precise blob detection and attribution , 2005, IEEE International Conference on Image Processing 2005.

[36]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[37]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[38]  Bernd Jähne,et al.  Digital Image Processing: Concepts, Algorithms, and Scientific Applications , 1991 .

[39]  Ron Kimmel,et al.  Demosaicing: Image Reconstruction from Color CCD Samples , 1998, ECCV.

[40]  Qi Tian,et al.  Algorithms for subpixel registration , 1986 .

[41]  Richard Szeliski,et al.  Vision Algorithms: Theory and Practice , 2002, Lecture Notes in Computer Science.