A Unifying and Rigorous Shape from Shading Method Adapted to Realistic Data and Applications

We propose a new method for the Lambertian Shape From Shading (SFS) problem based on the notion of Crandall-Lions viscosity solution. This method has the advantage of requiring the knowledge of the solution (the surface to be reconstructed) only on some part of the boundary and/or of the singular set (the set of the points at maximal intensity). Moreover it unifies in an unique mathematical formulation the works of Rouy et al. [34, 50], Falcone et al. [21], Prados et al. [46, 48, 49], based on the notion of viscosity solutions and the work of Dupuis and Oliensis [17] dealing with classical solutions and value functions. Also, it allows to generalize their results to the “perspective SFS” problem recently simultaneously introduced in [13,46,55].While the theoretical part has been developed in [44], in this paper we give some stability results and we describe numerical schemes for the SFS based on this method. We construct provably convergent and robust algorithms. Finally, we apply our SFS method to real images and we suggest some real-life applications.

[1]  P. Lions,et al.  Hamilton-Jacobi equations with state constraints , 1990 .

[2]  Alfred M. Bruckstein,et al.  Global Shape from Shading , 1996, Comput. Vis. Image Underst..

[3]  Pierre Gurdjos,et al.  Towards shape from shading under realistic photographic conditions , 2004, ICPR 2004.

[4]  Duncan Fyfe Gillies,et al.  Automated Endoscope Navigation and Advisory System from medical imaging , 1999 .

[5]  J. OLIENSIS Shape from shading as a partially well-constrained problem , 1991, CVGIP Image Underst..

[6]  Emmanuel Prados Application of the theory of the viscosity solutions to the Shape From Shading problem , 2004 .

[7]  Hideo Saito,et al.  A divide-and-conquer strategy in shape from shading problem , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[9]  P. Danielsson Euclidean distance mapping , 1980 .

[10]  Maurizio Falcone,et al.  An Algorithm for the Global Solution of the Shape-from-Shading Model , 1997, ICIAP.

[11]  Quinn Y. J. Smithwick,et al.  Depth enhancement using a scanning fiber optical endoscope , 2002, SPIE BiOS.

[12]  Yehezkel Yeshurun,et al.  Reconstruction of medical images by perspective shape-from-shading , 2004, ICPR 2004.

[13]  Emmanuel Prados,et al.  Une approche du "Shape From Shading" par solutions de viscosité , 2001 .

[14]  Henri Maître,et al.  On convergence in the methods of Strat and of Smith for Shape from Shading , 1996, International Journal of Computer Vision.

[15]  Hung-Tat Tsui,et al.  Global Shape from Shading for an Endoscope Image , 1999, MICCAI.

[16]  P. Lions,et al.  Shape-from-shading, viscosity solutions and edges , 1993 .

[17]  W. Brent Seales,et al.  Document restoration using 3D shape: a general deskewing algorithm for arbitrarily warped documents , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[19]  Alfred M. Bruckstein,et al.  Shape from Shading , 2020, Computer Vision.

[20]  Reinhard Klette,et al.  Shape from Shading and Photometric Stereo Methods , 1998 .

[21]  Berthold K. P. Horn Obtaining shape from shading information , 1989 .

[22]  John Oliensis,et al.  A global algorithm for shape from shading , 1993, 1993 (4th) International Conference on Computer Vision.

[23]  Duncan Fyfe Gillies,et al.  Automated endoscopic navigation and advisory system from medical image , 1999, Medical Imaging.

[24]  Olivier Faugeras,et al.  A mathematical and algorithmic study of the Lambertian SFS problem for orthographic and pinhole cameras , 2003 .

[25]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  G. Barles Solutions de viscosité des équations de Hamilton-Jacobi , 1994 .

[27]  Fabio Camilli,et al.  An approximation scheme for the maximal solution of the shape-from-shading model , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[28]  Olivier D. Faugeras,et al.  Dense image matching with global and local statistical criteria: a variational approach , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  P. Dupuis,et al.  Direct method for reconstructing shape from shading , 1991, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Olivier D. Faugeras,et al.  A Generic and Provably Convergent Shape-from-Shading Method for Orthographic and Pinhole Cameras , 2005, International Journal of Computer Vision.

[31]  Takashi Matsuyama,et al.  Shape from shading with interreflections under proximal light source-3D shape reconstruction of unfolded book surface from a scanner image , 1995, Proceedings of IEEE International Conference on Computer Vision.

[32]  Yehezkel Yeshurun,et al.  Reconstruction of medical images by perspective shape-from-shading , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[33]  Aly A. Farag,et al.  A system for human jaw modeling using intra-oral images , 1998, Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vol.20 Biomedical Engineering Towards the Year 2000 and Beyond (Cat. No.98CH36286).

[34]  Olivier D. Faugeras,et al.  Unifying Approaches and Removing Unrealistic Assumptions in Shape from Shading: Mathematics Can Help , 2004, ECCV.

[35]  F. Camilli,et al.  Nonconvex degenerate Hamilton–Jacobi equations , 2002 .

[36]  Olivier D. Faugeras,et al.  Shape from Shading and Viscosity Solutions , 2002, ECCV.

[37]  John Oliensis,et al.  Uniqueness in shape from shading , 1991, International Journal of Computer Vision.

[38]  Daniel N. Ostrov Solutions of Hamilton–Jacobi Equations and Scalar Conservation Laws with Discontinuous Space–Time Dependence , 2002 .

[39]  Alfred M. Bruckstein,et al.  Tracking Level Sets by Level Sets: A Method for Solving the Shape from Shading Problem , 1995, Comput. Vis. Image Underst..

[40]  Rachid Deriche,et al.  Vector-valued image regularization with PDE's: a common framework for different applications , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[41]  Fabio Camilli,et al.  Maximal subsolutions for a class of degenerate Hamilton-Jacobi problems , 1999 .

[42]  John Oliensis,et al.  Shape from Shading: Provably Convergent Algorithms and Uniqueness Results , 1994, ECCV.

[43]  H. Ishii,et al.  Uniqueness results for a class of hamilton-jacobi equations with singular coefficients , 1995 .

[44]  Olivier Faugeras,et al.  Shape from Shading and Vis osity , 2002 .

[45]  Jean-Denis Durou,et al.  A Survey of Numerical Methods for Shape from Shading , 2004 .

[46]  H. Soner Optimal control with state-space constraint I , 1986 .

[47]  J. Oliensis,et al.  Shape from shading as a partially well-constrained problem , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[48]  Edwin R. Hancock,et al.  Facial Pose using Shape-from-Shading , 1999, BMVC.

[49]  Paul Dupuis,et al.  Direct method for reconstructing shape from shading , 1991, Optics & Photonics.

[50]  Yehezkel Yeshurun,et al.  A new perspective [on] shape-from-shading , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[51]  P. Dupuis,et al.  An Optimal Control Formulation and Related Numerical Methods for a Problem in Shape Reconstruction , 1994 .

[52]  Rama Chellappa,et al.  Illumination-insensitive face recognition using symmetric shape-from-shading , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[53]  Edwin R. Hancock,et al.  Face Recognition Using Shape-from-shading , 2002, BMVC.

[54]  Olivier Faugeras,et al.  A viscosity method for Shape-from-Shading without boundary data , 2004 .

[55]  Olivier D. Faugeras,et al.  "Perspective shape from shading" and viscosity solutions , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[56]  Carlos Henrique Quartucci Forster,et al.  Towards 3D reconstruction of endoscope images using shape from shading , 2000, Proceedings 13th Brazilian Symposium on Computer Graphics and Image Processing (Cat. No.PR00878).

[57]  Qiang Ji,et al.  Digital imaging colposcopy: corrected area measurements using shape-from-shading , 1998, IEEE Transactions on Medical Imaging.

[58]  E. Rouy,et al.  A viscosity solutions approach to shape-from-shading , 1992 .

[59]  O. Faugeras,et al.  A VISCOSITY SOLUTION METHOD FOR SHAPE-FROM-SHADING WITHOUT IMAGE BOUNDARY DATA , 2006 .

[60]  Takayuki Okatani,et al.  Shape Reconstruction from an Endoscope Image by Shape from Shading Technique for a Point Light Source at the Projection Center , 1997, Comput. Vis. Image Underst..

[61]  Gary W. Meyer,et al.  Wavelength selection for synthetic image generation , 1988, Comput. Vis. Graph. Image Process..

[62]  William T. Freeman,et al.  Learning Local Evidence for Shading and Reflectance , 2001, ICCV.

[63]  P. Lions Generalized Solutions of Hamilton-Jacobi Equations , 1982 .

[64]  Maurizio Falcone,et al.  A scheme for the shape-from-shading model with “black shadows” , 2003 .

[65]  Ron Kimmel,et al.  Optimal Algorithm for Shape from Shading and Path Planning , 2001, Journal of Mathematical Imaging and Vision.

[66]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1991 .

[67]  W. Freeman,et al.  Learning local evidence for shading and reflectance , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[68]  Pierre Gurdjos,et al.  Towards shape from shading under realistic photographic conditions , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..