The fundamental matrix: Theory, algorithms, and stability analysis

In this paper we analyze in some detail the geometry of a pair of cameras, i.e., a stereo rig. Contrarily to what has been done in the past and is still done currently, for example in stereo or motion analysis, we do not assume that the intrinsic parameters of the cameras are known (coordinates of the principal points, pixels aspect ratio and focal lengths). This is important for two reasons. First, it is more realistic in applications where these parameters may vary according to the task (active vision). Second, the general case considered here, captures all the relevant information that is necessary for establishing correspondences between two pairs of images. This information is fundamentally projective and is hidden in a confusing manner in the commonly used formalism of the Essential matrix introduced by Longuet-Higgins (1981). This paper clarifies the projective nature of the correspondence problem in stereo and shows that the epipolar geometry can be summarized in one 3×3 matrix of rank 2 which we propose to call the Fundamental matrix.After this theoretical analysis, we embark on the task of estimating the Fundamental matrix from point correspondences, a task which is of practical importance. We analyze theoretically, and compare experimentally using synthetic and real data, several methods of estimation. The problem of the stability of the estimation is studied from two complementary viewpoints. First we show that there is an interesting relationship between the Fundamental matrix and three-dimensional planes which induce homographies between the images and create unstabilities in the estimation procedures. Second, we point to a deep relation between the unstability of the estimation procedure and the presence in the scene of so-called critical surfaces which have been studied in the context of motion analysis. Finally we conclude by stressing the fact that we believe that the Fundamental matrix will play a crucial role in future applications of three-dimensional Computer Vision by greatly increasing its versatility, robustness and hence applicability to real difficult problems.

[1]  G. M.,et al.  Projective Geometry , 1938, Nature.

[2]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[3]  F. Bookstein.  Fitting conic sections to scattered data , 1979 .

[4]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[5]  Lynn E. Garner.  An outline of projective geometry , 1980 .

[6]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[7]  Thomas S. Huang,et al.  Estimating three-dimensional motion parameters of a rigid planar patch, II: Singular value decomposition , 1982 .

[8]  PAUL D. SAMPSON,et al.  Fitting conic sections to "very scattered" data: An iterative refinement of the bookstein algorithm , 1982, Comput. Graph. Image Process..

[9]  Thomas S. Huang,et al.  Uniqueness and Estimation of Three-Dimensional Motion Parameters of Rigid Objects with Curved Surfaces , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Thomas S. Huang,et al.  Some Experiments on Estimating the 3-D Motion Parameters of a Rigid Body from Two Consecutive Image Frames , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  S. Maybank,et al.  The angular Velocity associated with the optical flowfield arising from motion through a rigid environment , 1985, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[12]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[13]  Olivier Faugeras,et al.  Motion and Structure from Motion in a piecewise Planar Environment , 1988, Int. J. Pattern Recognit. Artif. Intell..

[14]  H. C. Longuet-Higgins.  Multiple interpretations of a pair of images of a surface , 1988, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[15]  Gilad Adiv,et al.  Inherent Ambiguities in Recovering 3-D Motion and Structure from a Noisy Flow Field , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Narendra Ahuja,et al.  Motion and Structure From Two Perspective Views: Algorithms, Error Analysis, and Error Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Olivier D. Faugeras,et al.  Some Properties of the E Matrix in Two-View Motion Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Hans-Hellmut Nagel,et al.  Analytical results on error sensitivity of motion estimation from two views , 1990, Image Vis. Comput..

[19]  Hans-Hellmut Nagel,et al.  Analytical Results on Error Sensitivity of Motion Estimation from Two Views , 1990, ECCV.

[20]  S. Maybank.  The projective geometry of ambiguous surfaces , 1990, Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences.

[21]  S. Maybank.  Properties of essential matrices , 1990, Int. J. Imaging Syst. Technol..

[22]  Kenichi Kanatani,et al.  Computational projective geometry , 1991, CVGIP Image Underst..

[23]  Chia-Hoang Lee,et al.  Time-varying images: The effect of finite resolution on uniqueness , 1991, CVGIP Image Underst..

[24]  Johan Philip.  Estimation Three-Dimensional Motion of Rigid Objects from Noisy Observations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Ramesh C. Jain,et al.  Structure from motion-a critical analysis of methods , 1991, IEEE Trans. Syst. Man Cybern..

[26]  Paul A. Beardsley,et al.  Affine and Projective Structure from Motion , 1992, BMVC.

[27]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[28]  Olivier Faugeras,et al.  3D Dynamic Scene Analysis , 1992 .

[29]  Jitendra Malik,et al.  A Computational Framework for Determining Stereo Correspondence from a Set of Linear Spatial Filters , 1991, ECCV.

[30]  Minas E. Spetsakis.  A linear algorithm for point and line-based structure from motion , 1992, CVGIP Image Underst..

[31]  P. Beardsley,et al.  Affine and Projective Structure from Motion , 1992 .

[32]  Andrew Blake,et al.  Planar Region Detection and Motion Recovery , 1992, BMVC.

[33]  Søren I. Olsen,et al.  Epipolar Line Estimation , 1992, ECCV.

[34]  Kostas Daniilidis,et al.  Zur Fehlerempfindlichkeit in der Ermittlung von Objektbeschreibungen und relativen Bewegungen aus monokularen Bildfolgen , 1992 .

[35]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[36]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[37]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[38]  Quang-Tuan Luong.  Matrice fondamentale et autocalibration en vision par ordinateur , 1992 .

[39]  T. Viéville,et al.  Using pseudo Kalman-filters in the presence of constraints application to sensing behaviors , 1992 .

[40]  Eamon B. Barrett,et al.  Some invariant linear methods in photogrammetry and model-matching , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Jitendra Malik,et al.  Computational framework for determining stereo correspondence from a set of linear spatial filters , 1992, Image Vis. Comput..

[42]  R. Hartley.  Cheirality Invariants , 1993 .

[43]  Patrick Gros,et al.  3D projective invariants from two images , 1993, Optics & Photonics.

[44]  Olivier D. Faugeras,et al.  Relative 3D positioning and 3D convex hull computation from a weakly calibrated stereo pair , 1993, 1993 (4th) International Conference on Computer Vision.

[45]  O. Faugeras,et al.  On determining the fundamental matrix : analysis of different methods and experimental results , 1993 .

[46]  Andrew Blake,et al.  Planar region detection and motion recovery , 1993, Image Vis. Comput..

[47]  Amnon Shashua,et al.  Projective depth: A geometric invariant for 3D reconstruction from two perspective/orthographic views and for visual recognition , 1993, 1993 (4th) International Conference on Computer Vision.

[48]  Laurent Moll,et al.  Real time correlation-based stereo: algorithm, implementations and applications , 1993 .

[49]  Rajiv Gupta,et al.  Computing matched-epipolar projections , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Olivier D. Faugeras,et al.  Determining the fundamental matrix with planes: instability and new algorithms , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Amnon Shashua,et al.  On Geomatric and Algebraic Aspects of 3D Affine and Projective Structures from Perspective 2D Views , 1993, Applications of Invariance in Computer Vision.

[52]  Kenichi Kanatani,et al.  Geometric computation for machine vision , 1993 .

[53]  Roberto Cipolla,et al.  Robust Egomotion Estimation from Affine Motion Parallax , 1994, ECCV.

[54]  Martial Hebert,et al.  Deriving Orientation Cues from Stereo Images , 1994, ECCV.

[55]  Jean Ponce,et al.  Analytical Methods for Uncalibrated Stereo and Motion Reconstruction , 1994, ECCV.

[56]  Philip H. S. Torr,et al.  Stochastic Motion Clustering , 1994, ECCV.

[57]  Paul A. Beardsley,et al.  Navigation using Affine Structure from Motion , 1994, ECCV.

[58]  Ian D. Reid,et al.  Recursive Affine Structure and Motion from Image Sequences , 1994, ECCV.

[59]  Olivier D. Faugeras,et al.  What can two images tell us about a third one? , 1994, ECCV.

[60]  Richard I. Hartley,et al.  Projective reconstruction from line correspondences , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Richard I. Hartley,et al.  An algorithm for self calibration from several views , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Andrew Zisserman,et al.  Motion From Point Matches Using Affine Epipolar Geometry , 1994, ECCV.

[63]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[64]  Nassir Navab,et al.  Relative affine structure: theory and application to 3D reconstruction from perspective views , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Thierry Viéville,et al.  Canonic Representations for the Geometries of Multiple Projective Views , 1994, ECCV.

[66]  Olivier D. Faugeras,et al.  Computing differential properties of 3-D shapes from stereoscopic images without 3-D models , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[67]  Rachid Deriche,et al.  Robust Recovery of the Epipolar Geometry for an Uncalibrated Stereo Rig , 1994, ECCV.

[68]  S. P. Mudur,et al.  Three-Dimensional Computer Vision: A Geometric Viewpoint , 1995 .

[69]  Jitendra Malik,et al.  Rigid body segmentation and shape description from dense optical flow under weak perspective , 1995, Proceedings of IEEE International Conference on Computer Vision.

[70]  G. Dullerud.  A Computational Framework , 1996 .

[71]  Thierry Viéville,et al.  Canonical Representations for the Geometries of Multiple Projective Views , 1996, Comput. Vis. Image Underst..

[72]  Olivier D. Faugeras,et al.  Camera Calibration, Scene Motion and Structure recovery from point correspondences and fundamental m , 1997 .

[73]  J. Buchar,et al.  The calibration problem , 1998 .

[74]  A. J. Batista-Leyva,et al.  On the interpretation of , 2004 .

[75]  Olivier D. Faugeras,et al.  Motion of points and lines in the uncalibrated case , 2004, International Journal of Computer Vision.

[76]  Berthold K. P. Horn.  Relative orientation , 1987, International Journal of Computer Vision.

[77]  Olivier D. Faugeras,et al.  A theory of self-calibration of a moving camera , 1992, International Journal of Computer Vision.

[78]  Long Quan,et al.  Affine shape representation from motion through reference points , 2004, Journal of Mathematical Imaging and Vision.

[79]  Berthold K. P. Horn,et al.  Direct methods for recovering motion , 1988, International Journal of Computer Vision.