A Generic and Provably Convergent Shape-from-Shading Method for Orthographic and Pinhole Cameras

We describe a mathematical and algorithmic study of the Lambertian “Shape-From-Shading” problem for orthographic and pinhole cameras. Our approach is based upon the notion of viscosity solutions of Hamilton-Jacobi equations. This approach provides a mathematical framework in which we can show that the problem is well-posed (we prove the existence of a solution and we characterize all the solutions). Our contribution is threefold. First, we model the camera both as orthographic and as perspective (pinhole), whereas most authors assume an orthographic projection (see Horn and Brooks (1989) for a survey of the SFS problem up to 1989 and Zhang et al. (1999), Kozera (1998), Durou et al. (2004) for more recent ones); thus we extend the applicability of shape from shading methods to more realistic acquisition models. In particular it extends the work of Prados et al. (2002a) and Rouy and Tourin (1992). We provide some novel mathematical formulations of this problem yielding new partial differential equations. Results about the existence and uniqueness of their solutions are also obtained. Second, by introducing a “generic” Hamiltonian, we define a general framework allowing to deal with both models (orthographic and perspective), thereby simplifying the formalization of the problem. Thanks to this unification, each algorithm we propose can compute numerical solutions corresponding to all the modeling. Third, our work allows us to come up with two new generic algorithms for computing numerical approximations of the “continuous solution of the “Shape-From-Shading” problem as well as a proof of their convergence toward that solution. Moreover, our two generic algorithms are able to deal with discontinuous images as well as images containing black shadows.

[1]  Paul Dupuis,et al.  Direct method for reconstructing shape from shading , 1991, Optics & Photonics.

[2]  Yehezkel Yeshurun,et al.  A new perspective [on] shape-from-shading , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[3]  Takayuki Okatani,et al.  Reconstructing shape from shading with a point light source at the projection center: shape reconstruction from an endoscope image , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[4]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[5]  P. Danielsson Euclidean distance mapping , 1980 .

[6]  Reinhard Klette,et al.  Shape from Shading and Photometric Stereo Methods , 1998 .

[7]  J. Oliensis,et al.  Shape from shading as a partially well-constrained problem , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  H. Ishii Hamilton-Jacobi Equations with Discontinuous Hamiltonians on Arbitrary Open Sets , 1985 .

[9]  Shiu Yin Yuen,et al.  FAST MARCHING METHOD FOR SHAPE FROM SHADING UNDER PERSPECTIVE PROJECTION , 2002 .

[10]  Alfred M. Bruckstein,et al.  Global Shape from Shading , 1996, Comput. Vis. Image Underst..

[11]  Fabio Camilli A CHARACTERIZATION OF THE VALUE FUNCTION FOR A CLASS OF DEGENERATE CONTROL PROBLEMS , 2001 .

[12]  Berthold K. P. Horn Obtaining shape from shading information , 1989 .

[13]  Jean-Denis Durou,et al.  Ambiguous Shape from Shading with Critical Points , 2004, Journal of Mathematical Imaging and Vision.

[14]  Michael A. Penna,et al.  Local and semi-local shape from shading for a single perspective image of a smooth object , 1989, Comput. Vis. Graph. Image Process..

[15]  Maurizio Falcone,et al.  Numerical methods for viscosity solutions and applications , 2006 .

[16]  Olivier D. Faugeras,et al.  "Perspective shape from shading" and viscosity solutions , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Maurizio Falcone,et al.  An Algorithm for the Global Solution of the Shape-from-Shading Model , 1997, ICIAP.

[18]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Dimitris N. Metaxas,et al.  Coupled lighting direction and shape estimation from single images , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[20]  Richard Szeliski,et al.  Impossible Shaded Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Ron Kimmel,et al.  Optimal Algorithm for Shape from Shading and Path Planning , 2001, Journal of Mathematical Imaging and Vision.

[22]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1991 .

[23]  M. Chial,et al.  in simple , 2003 .

[24]  Jean-Denis Durou,et al.  A Survey of Numerical Methods for Shape from Shading , 2004 .

[25]  Pierpaolo Soravia Optimal control with discontinuous running cost: Eikonal equation and shape-from-shading , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[26]  Daniel N. Ostrov,et al.  Numerical Shape-From-Shading for Discontinuous Photographic Images , 2001, International Journal of Computer Vision.

[27]  Hitoshi Ishii,et al.  A boundary value problem of the Dirichlet type for Hamilton-Jacobi equations , 1989 .

[28]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[29]  Alfred M. Bruckstein,et al.  Shape from Shading , 2020, Computer Vision.

[30]  H. Ishii,et al.  Uniqueness results for a class of hamilton-jacobi equations with singular coefficients , 1995 .

[31]  Yehezkel Yeshurun,et al.  Shape-from-Shading Under Perspective Projection , 2005, International Journal of Computer Vision.

[32]  Alfred M. Bruckstein,et al.  Tracking Level Sets by Level Sets: A Method for Solving the Shape from Shading Problem , 1995, Comput. Vis. Image Underst..

[33]  Stanley Osher,et al.  Level Set Methods , 2003 .

[34]  P. Lions,et al.  Shape-from-shading, viscosity solutions and edges , 1993 .

[35]  Daniel N. Ostrov Extending viscosity solutions to Eikonal equations with discontinuous spatial dependence , 2000 .

[36]  Emmanuel Prados,et al.  Une approche du "Shape From Shading" par solutions de viscosité , 2001 .

[37]  Henri Maître,et al.  On convergence in the methods of Strat and of Smith for Shape from Shading , 1996, International Journal of Computer Vision.

[38]  H. Ishii A simple, direct proof of uniqueness for solutions of the hamilton-jacobi equations of eikonal type , 1987 .

[39]  Bertrand Kerautret,et al.  A robust discrete Approach for Shape from Shading and Photometric Stereo , 2004, ICCVG.

[40]  Olivier Faugeras,et al.  A mathematical and algorithmic study of the Lambertian SFS problem for orthographic and pinhole cameras , 2003 .

[41]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Kyoung Mu Lee,et al.  Shape from Shading with Perspective Projection , 1994 .

[43]  E. Rouy,et al.  A viscosity solutions approach to shape-from-shading , 1992 .

[44]  Takayuki Okatani,et al.  Shape Reconstruction from an Endoscope Image by Shape from Shading Technique for a Point Light Source at the Projection Center , 1997, Comput. Vis. Image Underst..

[45]  H. Farmer A new perspective. , 1988, The Journal of the Florida Medical Association.

[46]  Yehezkel Yeshurun,et al.  Perspective shape-from-shading by fast marching , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[47]  Ryszard Kozera An Overview of the shape-from-shading problem , 1998 .

[48]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.

[49]  Fabio Camilli,et al.  Hamilton-Jacobi equations with measurable dependence on the state variable , 2003 .

[50]  Michael A. Penna,et al.  A Shape From Shading Analysis for a Single Perspective Image of a Polyhedron , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Fabio Camilli,et al.  Maximal subsolutions for a class of degenerate Hamilton-Jacobi problems , 1999 .

[52]  P. Dupuis,et al.  An Optimal Control Formulation and Related Numerical Methods for a Problem in Shape Reconstruction , 1994 .

[53]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[54]  Wojciech Chojnacki,et al.  Shading without shape , 1992 .

[55]  Maurizio Falcone,et al.  A scheme for the shape-from-shading model with “black shadows” , 2003 .

[56]  G. Barles Solutions de viscosité des équations de Hamilton-Jacobi , 1994 .

[57]  Fabio Camilli,et al.  An approximation scheme for the maximal solution of the shape-from-shading model , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[58]  Olivier D. Faugeras,et al.  Shape from Shading and Viscosity Solutions , 2002, ECCV.

[59]  P. Lions Generalized Solutions of Hamilton-Jacobi Equations , 1982 .

[60]  John Oliensis,et al.  A global algorithm for shape from shading , 1993, 1993 (4th) International Conference on Computer Vision.

[61]  Anna R. Bruss The Eikonal equation: some results applicable to computer vision , 1982 .

[62]  Ryszard Kozera,et al.  Uniqueness in Shape from Shading Revisited , 1997, Journal of Mathematical Imaging and Vision.

[63]  Clésio Luiz Tozzi,et al.  Shape from shading with perspective projection and camera calibration , 1996, Comput. Graph..