Learning shape from shading by a multilayer network

The multilayer feedforward network has often been used for learning a nonlinear mapping based on a set of examples of the input-output data. In this paper, we present a novel use of the network, in which the example data are not explicitly given. We consider the problem of shape from shading in computer vision, where the input (image coordinates) and the output (surface depth) satisfy only a known differential equation. We use the feedforward network as a parametric representation of the object surface and reformulate the shape from shading problem as the minimization of an error function over the network weights. The stochastic gradient and conjugate gradient methods are used for the minimization. Boundary conditions for either surface depth or surface normal (or both) can be imposed by adjusting the same network at different levels. It is further shown that the light source direction can be estimated, based on an initial guess, by integrating the source estimation with the surface estimation. Extensions of the method to a wider class of problems are discussed. The efficiency of the method is verified by examples of both synthetic and real images.

[1]  Azriel Rosenfeld,et al.  Improved Methods of Estimating Shape from Shading Using the Light Source Coordinate System , 1985, Artif. Intell..

[2]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[3]  William H. Press,et al.  Book-Review - Numerical Recipes in Pascal - the Art of Scientific Computing , 1989 .

[4]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[5]  A. Pentland Finding the illuminant direction , 1982 .

[6]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[7]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Guo-Quing Hirzinger Gerh. Wei Learning Shape from Shading by Neural Networks , 1994 .

[9]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[10]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[11]  Douglas H. Norrie,et al.  An introduction to finite element analysis , 1978 .

[12]  R. K. Livesley Finite Elements: An Introduction for Engineers , 1983 .

[13]  Aaron F. Bobick,et al.  The direct computation of height from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[15]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .

[16]  Michael J. Brooks,et al.  The variational approach to shape from shading , 1986, Comput. Vis. Graph. Image Process..

[17]  Berthold K. P. Horn Obtaining shape from shading information , 1989 .

[18]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[19]  Rama Chellappa,et al.  Estimation of Illuminant Direction, Albedo, and Shape from Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Richard Szeliski,et al.  Fast shape from shading , 1990, CVGIP Image Underst..

[21]  William H. Press,et al.  Numerical Recipes in C The Art of Scientific Computing , 1995 .

[22]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  M.D. Levine,et al.  Where and Why Local Shading Analysis Works , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[25]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[26]  Michael J. Brooks,et al.  Shape and Source from Shading , 1985, IJCAI.

[27]  C.-C. Jay Kuo,et al.  Shape from Shading with a Linear Triangular Element Surface Model , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  J. OLIENSIS Shape from shading as a partially well-constrained problem , 1991, CVGIP Image Underst..