Real-time pose estimation of 3D objects from camera images using neural networks

This paper deals with the problem of obtaining a rough estimate of three dimensional object position and orientation from a single two dimensional camera image. Such an estimate is required by most 3-D to 2-D registration and tracking methods that can efficiently refine an initial value by numerical optimization to precisely recover 3-D pose. However the analytic computation of an initial pose guess requires the solution of an extremely complex correspondence problem that is due to the large number of topologically distinct aspects that arise when a three dimensional opaque object is imaged by a camera. Hence general analytic methods fail to achieve real-time performance and most tracking and registration systems are initialized interactively or by ad hoc heuristics. To overcome these limitations we present a novel method for approximate object pose estimation that is based on a neural net and that can easily be implemented in real-time. A modification of Kohonen's self-organizing feature map is systematically trained with computer generated object views such that it responds to a preprocessed image with one or more sets of object orientation parameters. The key idea proposed here is to choose network topology in accordance with the representation of 3-D orientation. Experimental results from both simulated and real images demonstrate that a pose estimate within the accuracy requirements can be found in more than 81% of all cases. The current implementation operates at 10 Hz on real world images.

[1]  K. Schulten,et al.  Kohonen's self-organizing maps: exploring their computational capabilities , 1988, IEEE 1988 International Conference on Neural Networks.

[2]  T. Poggio,et al.  A network that learns to recognize three-dimensional objects , 1990, Nature.

[3]  Helge Ritter,et al.  Learning with the Self-Organizing Map , 1991 .

[4]  Pasi Koikkalainen,et al.  Self-organizing hierarchical feature maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[5]  Helge J. Ritter,et al.  Rapid learning with parametrized self-organizing maps , 1996, Neurocomputing.

[6]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[7]  Philip E. Brou.  Using the Gaussian Image to Find the Orientation of Objects , 1984 .

[8]  Helge J. Ritter,et al.  A neural 3-D object recognition architecture using optimized Gabor filters , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[9]  R.P. Lippmann,et al.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[10]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[11]  George Nagy.  Candide's Practical Principles of Experimental Pattern Recognition , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  N. Ranganathan,et al.  Gabor filter-based edge detection , 1992, Pattern Recognit..

[13]  Helge J. Ritter,et al.  Selbstorganisierende neuronale Karten , 1988 .

[14]  Kang Park,et al.  Recognition and localization of a 3D polyhedral object using a neural network , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[15]  Olaf Munkelt.  Aspect-Trees: Generation and Interpretation , 1995, Comput. Vis. Image Underst..

[16]  Hon-Son Don,et al.  A neural network approach to 3D object identification and pose estimation , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[17]  Michel Dhome,et al.  Determination of the Attitude of 3D Objects from a Single Perspective View , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[19]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[20]  Helge J. Ritter,et al.  Three-dimensional neural net for learning visuomotor coordination of a robot arm , 1990, IEEE Trans. Neural Networks.

[21]  Volker Tresp,et al.  Die besonderen Eigenschaften Neuronaler Netze bei der Approximation von Funktionen , 1995, Künstliche Intell..

[22]  Anil K. Jain,et al.  BONSAI: 3D Object Recognition Using Constrained Search , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Stéphane Mallat,et al.  Singularity detection and processing with wavelets , 1992, IEEE Trans. Inf. Theory.

[24]  Ken Shoemake,et al.  Animating rotation with quaternion curves , 1985, SIGGRAPH.

[25]  Helge Ritter,et al.  Local PSOMs and Chebyshev PSOMs -Improving the Parametrised Self-Organizing Maps , 1995 .

[26]  John F. Gilmore,et al.  Object recognition using neural networks and high-order perspective-invariant relational descriptions , 1992, Other Conferences.

[27]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[28]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[29]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[30]  H. Ritter.  Combining self-organizing maps , 1989, International 1989 Joint Conference on Neural Networks.

[31]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[32]  Jorma Laaksonen,et al.  Variants of self-organizing maps , 1990, International 1989 Joint Conference on Neural Networks.

[33]  Helge Ritter.  Parametrized Self-Organizing Maps for Vision Learning Tasks , 1994 .

[34]  William E. Higgins,et al.  Texture Segmentation using 2-D Gabor Elementary Functions , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Gerd Hirzinger,et al.  Registration of CAD-models to images by iterative inverse perspective matching , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[36]  Helge Ritter,et al.  Extending Kohonens Self-Organizing Mapping Algorithm to Learn Ballistic Movements , 1988 .

[37]  Teuvo Kohonen.  What Generalizations of the Self-Organizing Map Make Sense? , 1994 .

[38]  C. C. Li,et al.  A performance study of two wavelet-based edge detectors , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol. III. Conference C: Image, Speech and Signal Analysis,.