Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images

The human visual system is proficient in perceiving three-dimensional shape from the shading patterns in a two-dimensional image. How it does this is not well understood and continues to be a question of fundamental and practical interest. In this paper we present a new quantitative approach to shape-from-shading that may provide some answers. We suggest that the brain, through evolution or prior experience, has discovered that objects can be classified into lower-dimensional object-classes as to their shape. Extraction of shape from shading is then equivalent to the much simpler problem of parameter estimation in a low-dimensional space. We carry out this proposal for an important class of three-dimensional (3D) objects: human heads. From an ensemble of several hundred laser-scanned 3D heads, we use principal component analysis to derive a low-dimensional parameterization of head shape space. An algorithm for solving shape-from-shading using this representation is presented. It works well even on real images where it is able to recover the 3D surface for a given person, maintaining facial detail and identity, from a single 2D image of his face. This algorithm has applications in face recognition and animation.

[1]  E. Anding.  Lambert's Photometrie , .

[2]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[3]  R. Gregory.  The intelligent eye , 1970 .

[4]  E. Simmons.  Diffuse reflectance spectroscopy: a comparison of the theories. , 1975, Applied optics.

[5]  Robert B. Lawson,et al.  Human stereopsis : a psychophysical analysis , 1976 .

[6]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[7]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[8]  Robert J. Woodham,et al.  Analysing Images of Curved Surfaces , 1981, Artif. Intell..

[9]  A. Pentland.  Finding the illuminant direction , 1982 .

[10]  E. Mingolla,et al.  Perception of surface curvature and direction of illumination from patterns of shading. , 1983, Journal of experimental psychology. Human perception and performance.

[11]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Azriel Rosenfeld,et al.  Improved Methods of Estimating Shape from Shading Using the Light Source Coordinate System , 1985, Artif. Intell..

[13]  Michael J. Brooks,et al.  Shape and Source from Shading , 1985, IJCAI.

[14]  William H. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[15]  L Sirovich,et al.  Low-dimensional Procedure for the Characterization of Human Faces , 1986 .

[16]  L. Sirovich.  Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[17]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[18]  V. S. Ramachandran,et al.  Perception of shape from shading , 1988, Nature.

[19]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[20]  Vilayanur S. Ramachandran,et al.  Theories of Perception. , 1951 .

[21]  George Wolberg,et al.  Digital image warping , 1990 .

[22]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Rama Chellappa,et al.  Estimation of illuminant direction, albedo, and shape from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Yoshio Ohno,et al.  Computer Graphics : Principles and Practice, 2nd edition, J.D. Foley, A.van Dam, S.K. Feiner, J.F. Hughes, Addison-Wesley, 1990 , 1991 .

[25]  Rama Chellappa,et al.  Estimation of Illuminant Direction, Albedo, and Shape from Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  J. Oliensis,et al.  Shape from shading as a partially well-constrained problem , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  J. Koenderink,et al.  Surface perception in pictures , 1992, Perception & psychophysics.

[28]  Lawrence Sirovich,et al.  Management and Analysis of Large Scientific Datasets , 1992 .

[29]  T. Sejnowski,et al.  A critique of pure vision , 1993 .

[30]  Shimon Edelman,et al.  Representation of Similarity in Three-Dimensional Object Discrimination , 1995, Neural Computation.

[31]  S. Edelman.  Representation of Similarity in 3D Object Discrimination , 1995 .

[32]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[33]  Shimon Edelman,et al.  Receptive field spaces and class-based generalization from a single view in face recognition , 1995 .

[34]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..