Variational Methods for Multimodal Image Matching

Matching images of different modalities can be achieved by the maximization of suitable statistical similarity measures within a given class of geometric transformations. Handling complex, nonrigid deformations in this context turns out to be particularly difficult and has attracted much attention in the last few years. The thrust of this paper is that many of the existing methods for nonrigid monomodal registration that use simple criteria for comparing the intensities (e.g. SSD) can be extended to the multimodal case where more complex intensity similarity measures are necessary. To this end, we perform a formal computation of the variational gradient of a hierarchy of statistical similarity measures, and use the results to generalize a recently proposed and very effective optical flow algorithm (L. Alvarez, J. Weickert, and J. Sánchez, 2000, Technical Report, and IJCV 39(1):41–56) to the case of multimodal image registrationOur method readily extends to the case of locally computed similarity measures, thus providing the flexibility to cope with spatial non-stationarities in the way the intensities in the two images are related. The well posedness of the resulting equations is proved in a complementary work (O.D. Faugeras and G. Hermosillo, 2001, Technical Report 4235, INRIA) using well established techniques in functional analysis. We briefly describe our numerical implementation of these equations and show results on real and synthetic data.

[1]  Michael J. Black Robust incremental optical flow , 1992 .

[2]  Jürgen Weese,et al.  A comparison of similarity measures for use in 2-D-3-D medical image registration , 1998, IEEE Transactions on Medical Imaging.

[3]  X. Pennec,et al.  3D non-rigid registration by gradient descent on a Gaussian-windowed similarity measure using convolutions , 2000, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis. MMBIA-2000 (Cat. No.PR00737).

[4]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[5]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[6]  Rachid Deriche,et al.  Fast algorithms for low-level vision , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[7]  A. Verri,et al.  Constraints for the computation of optical flow , 1989, [1989] Proceedings. Workshop on Visual Motion.

[8]  A. Roche Recalage d'images médicales par inférence statistique , 2001 .

[9]  Paolo Nesi,et al.  Variational approach to optical flow estimation managing discontinuities , 1993, Image Vis. Comput..

[10]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[11]  Laurent Moll,et al.  Real time correlation-based stereo: algorithm, implementations and applications , 1993 .

[12]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[13]  Yali Amit,et al.  A Nonlinear Variational Problem for Image Matching , 1994, SIAM J. Sci. Comput..

[14]  Paul A. Viola Alignment by maximisation of mutual information , 1993 .

[15]  Joachim Weickert,et al.  Variational Optic Flow Computation with a Spatio-Temporal Smoothness Constraint , 2001, Journal of Mathematical Imaging and Vision.

[16]  Marco Mattavelli,et al.  Motion estimation relaxing the constancy brightness constraint , 1994, Proceedings of 1st International Conference on Image Processing.

[17]  A. Verri,et al.  Differential techniques for optical flow , 1990 .

[18]  Tomaso A. Poggio,et al.  Motion Field and Optical Flow: Qualitative Properties , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Rachid Deriche,et al.  Optical-Flow Estimation while Preserving Its Discontinuities: A Variational Approach , 1995, ACCV.

[20]  Takeyoshi Dohi,et al.  Multimodality Deformable Registration of Pre- and Intraoperative Images for MRI-guided Brain Surgery , 1998, MICCAI.

[21]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[22]  Nicholas Ayache,et al.  Regularization in Image Non-Rigid Registration: I. Trade-off between Smoothness and Intensity Similarity , 2001 .

[23]  Hans-Hellmut Nagel,et al.  On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results , 1987, Artif. Intell..

[24]  Luc Van Gool,et al.  Determination of Optical Flow and its Discontinuities using Non-Linear Diffusion , 1994, ECCV.

[25]  Andrea J. van Doorn,et al.  Blur and Disorder , 2000, J. Vis. Commun. Image Represent..

[26]  W. Eric L. Grimson,et al.  Multi-modal Volume Registration Using Joint Intensity Distributions , 1998, MICCAI.

[27]  Rachid Deriche,et al.  Computing Optical Flow via Variational Techniques , 1999, SIAM J. Appl. Math..

[28]  G. Christensen,et al.  k 3D Deformable Magnetic Resonance Textbook Based on Elasticity , 1994 .

[29]  Michael I. Miller,et al.  Group Actions, Homeomorphisms, and Matching: A General Framework , 2004, International Journal of Computer Vision.

[30]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[31]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[32]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Patrick Bouthemy,et al.  Computation and analysis of image motion: A synopsis of current problems and methods , 1996, International Journal of Computer Vision.

[34]  Joachim Weickert,et al.  Reliable Estimation of Dense Optical Flow Fields with Large Displacements , 2000, International Journal of Computer Vision.

[35]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[37]  G. Christensen,et al.  Consistent nonlinear elastic image registration , 2001, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA 2001).

[38]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[39]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[40]  S. P. Mudur,et al.  Three-Dimensional Computer Vision: A Geometric Viewpoint , 1995 .

[41]  Max A. Viergever,et al.  General multimodal elastic registration based on mutual information , 1998, Medical Imaging.

[42]  Paul Suetens,et al.  Non-rigid Multimodal Image Registration Using Mutual Information , 1998, MICCAI.

[43]  Gerardo Hermosillo Valadez Variational Methods for Multimodal Image Matching , 2002 .

[44]  乔花玲,et al.  关于Semigroups of Linear Operators and Applications to Partial Differential Equations的两个注解 , 2003 .

[45]  Christoph Schnörr,et al.  Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class , 1991, International Journal of Computer Vision.

[46]  Joachim Weickert,et al.  A Theoretical Framework for Convex Regularizers in PDE-Based Computation of Image Motion , 2001, International Journal of Computer Vision.

[47]  Chuck Meyer,et al.  Evaluation of Control Point Selection in Automatic, Mutual Information Driven, 3D Warping , 1998, MICCAI.

[48]  Frédéric Guichard,et al.  Accurate estimation of discontinuous optical flow by minimizing divergence related functionals , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[49]  T. Netsch,et al.  Towards real-time multi-modality 3-D medical image registration , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[50]  J. Koenderink The structure of images , 2004, Biological Cybernetics.

[51]  J. Mazziotta,et al.  MRI‐PET Registration with Automated Algorithm , 1993, Journal of computer assisted tomography.

[52]  Guy Marchal,et al.  Multi-modality image registration by maximization of mutual information , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[53]  Paul A. Viola,et al.  Multi-modal volume registration by maximization of mutual information , 1996, Medical Image Anal..

[54]  Denis Bosq,et al.  Nonparametric statistics for stochastic processes , 1996 .

[55]  Jerry L. Prince,et al.  On div-curl regularization for motion estimation in 3-D volumetric imaging , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[56]  S. Brendle,et al.  Calculus of Variations , 1927, Nature.

[57]  I. Du,et al.  Direct Methods , 1998 .

[58]  M. Bertero,et al.  Ill-posed problems in early vision , 1988, Proc. IEEE.

[59]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Nicholas Ayache,et al.  Understanding the "Demon's Algorithm": 3D Non-rigid Registration by Gradient Descent , 1999, MICCAI.

[61]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[62]  Françoise Veillon One pass computation of morphological and geometrical properties of objects in digital pictures , 1979 .

[63]  I. Cohen Nonlinear Variational Method for Optical Flow Computation , 2006 .

[64]  David Rey,et al.  Symmetrization of the Non-rigid Registration Problem Using Inversion-Invariant Energies: Application to Multiple Sclerosis , 2000, MICCAI.

[65]  Nicholas Ayache,et al.  Multimodal Elastic Matching of Brain Images , 2000, ECCV.

[66]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[67]  Bob Fisher,et al.  From surfaces to objects , 1989 .

[68]  Hans-Hellmut Nagel,et al.  Direct Estimation of Optical Flow and of Its Derivatives , 1992 .

[69]  Nicholas Ayache,et al.  Unifying maximum likelihood approaches in medical image registration , 2000, Int. J. Imaging Syst. Technol..

[70]  Michael A. Penna,et al.  Projective geometry and its applications to computer graphics , 1986 .

[71]  Joachim Weickert,et al.  On Discontinuity-Preserving Optic Flow , 1998 .

[72]  Michael Unser,et al.  Unwarping of unidirectionally distorted EPI images , 2000, IEEE Transactions on Medical Imaging.

[73]  Lawrence H. Staib,et al.  Physical model-based non-rigid registration incorporating statistical shape information , 2000, Medical Image Anal..

[74]  N. Ayache,et al.  Multimodal Image Registration by Maximization of the Correlation Ratio , 1998 .

[75]  P. Lions,et al.  Axioms and fundamental equations of image processing , 1993 .

[76]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[77]  Patrick Pérez,et al.  Dense/parametric estimation of fluid flows , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[78]  O. Faugeras,et al.  A variational approach to multi-modal image matching , 2001, Proceedings IEEE Workshop on Variational and Level Set Methods in Computer Vision.

[79]  Joachim Weickert,et al.  A Scale-Space Approach to Nonlocal Optical Flow Calculations , 1999, Scale-Space.

[80]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations, 2nd Edition , 2010, Applied mathematical sciences.

[81]  Jean-Philippe Thirion,et al.  Image matching as a diffusion process: an analogy with Maxwell's demons , 1998, Medical Image Anal..

[82]  Francisco Santana Recursivity and PDE's in Image Processing , 2000 .

[83]  Daniel Rueckert,et al.  Non-rigid Registration of Breast MR Images Using Mutual Information , 1998, MICCAI.

[84]  Sébastien Ourselin,et al.  Block Matching: A General Framework to Improve Robustness of Rigid Registration of Medical Images , 2000, MICCAI.

[85]  O. Faugeras,et al.  Well-posedness of eight problems of multi-modal statistical image-matching , 2002, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[86]  Alain Trouvé,et al.  Diffeomorphisms Groups and Pattern Matching in Image Analysis , 1998, International Journal of Computer Vision.

[87]  Rachid Deriche,et al.  Dense Disparity Map Estimation Respecting Image Discontinuities: A PDE and Scale-Space BasedApproach , 2002, MVA.

[88]  Michael I. Miller,et al.  Deformable templates using large deformation kinematics , 1996, IEEE Trans. Image Process..

[89]  Patrick Bouthemy,et al.  Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[90]  Dorin Comaniciu,et al.  The Variable Bandwidth Mean Shift and Data-Driven Scale Selection , 2001, ICCV.

[91]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[92]  Nicholas Ayache,et al.  The Correlation Ratio as a New Similarity Measure for Multimodal Image Registration , 1998, MICCAI.

[93]  Hans-Hellmut Nagel,et al.  Optical Flow Estimation: Advances and Comparisons , 1994, ECCV.

[94]  G. Aubert,et al.  A mathematical study of the relaxed optical flow problem in the space BV (&Ω) , 1999 .

[95]  Olivier D. Faugeras,et al.  Flows of diffeomorphisms for multimodal image registration , 2002, Proceedings IEEE International Symposium on Biomedical Imaging.

[96]  G. M.,et al.  Partial Differential Equations I , 2023, Applied Mathematical Sciences.

[97]  Patrick Pérez,et al.  A multigrid approach for hierarchical motion estimation , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).