Generalized Deformable Models, Statistical Physics, and Matching Problems

We describe how to formulate matching and combinatorial problems of vision and neural network theory by generalizing elastic and deformable templates models to include binary matching elements. Techniques from statistical physics, which can be interpreted as computing marginal probability distributions, are then used to analyze these models and are shown to (1) relate them to existing theories and (2) give insight into the relations between, and relative effectivenesses of, existing theories. In particular we exploit the power of statistical techniques to put global constraints on the set of allowable states of the binary matching elements. The binary elements can then be removed analytically before minimization. This is demonstrated to be preferable to existing methods of imposing such constraints by adding bias terms in the energy functions. We give applications to winner-take-all networks, correspondence for stereo and long-range motion, the traveling salesman problem, deformable template matching, learning, content addressable memories, and models of brain development. The biological plausibility of these networks is briefly discussed.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[3]  Martin A. Fischler,et al.  The Representation and Matching of Pictorial Structures , 1973, IEEE Transactions on Computers.

[4]  A.H. Haddad,et al.  Applied optimal estimation , 1976, Proceedings of the IEEE.

[5]  C. W. Groetsch,et al.  Regularization of Ill-Posed Problems. , 1978 .

[6]  S. Ullman The Interpretation of Visual Motion , 1979 .

[7]  D. Burr A dynamic model for image registration , 1981 .

[8]  David J. Burr,et al.  Elastic Matching of Line Drawings , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Andrew Blake,et al.  The least-disturbance principle and weak constraints , 1983, Pattern Recognit. Lett..

[11]  V. S. Ramachandran,et al.  Extrapolation of motion path in human visual perception , 1983, Vision Research.

[12]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[13]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  G. Blasdel,et al.  Voltage-sensitive dyes reveal a modular organization in monkey striate cortex , 1986, Nature.

[16]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[17]  Geoffrey E. Hinton,et al.  Separating Figure from Ground with a Parallel Network , 1986, Perception.

[18]  V. Ramachandran,et al.  Visual inertia in apparent motion , 1987, Vision Research.

[19]  A. Pentland Recognition by Parts , 1987 .

[20]  Carsten Peterson,et al.  A Mean Field Theory Learning Algorithm for Neural Networks , 1987, Complex Syst..

[21]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.

[22]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[23]  R. Brockett,et al.  Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[24]  G. Tesauro A plausible neural circuit for classical conditioning without synaptic plasticity. , 1988, Proceedings of the National Academy of Sciences of the United States of America.

[25]  E. Gardner The space of interactions in neural network models , 1988 .

[26]  Carsten Peterson,et al.  A New Method for Mapping Optimization Problems Onto Neural Networks , 1989, Int. J. Neural Syst..

[27]  Andrew Blake,et al.  Comparison of the Efficiency of Deterministic and Stochastic Algorithms for Visual Reconstruction , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Geoffrey E. Hinton Deterministic Boltzmann Learning Performs Steepest Descent in Weight-Space , 1989, Neural Computation.

[29]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[30]  Richard Szeliski,et al.  An Analysis of the Elastic Net Approach to the Traveling Salesman Problem , 1989, Neural Computation.

[31]  Alan L. Yuille,et al.  A Winner-Take-All Mechanism Based on Presynaptic Inhibition Feedback , 1989, Neural Computation.

[32]  R. Brockett Least squares matching problems , 1989 .

[33]  D. Geiger,et al.  Parallel and deterministic algorithms from MRFs (Markov Random Fields): Surface reconstruction and integration. Memorandum report , 1989 .

[34]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[35]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[36]  Carsten Peterson,et al.  Parallel Distributed Approaches to Combinatorial Optimization: Benchmark Studies on Traveling Salesman Problem , 1990, Neural Computation.

[37]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[38]  Petar D. Simic,et al.  Statistical mechanics as the underlying theory of ‘elastic’ and ‘neural’ optimisations , 1990 .

[39]  Alan L. Yuille,et al.  Deformable Templates for Feature Extraction from Medical Images , 1990, ECCV.

[40]  Petar D. Simic Constrained Nets for Graph Matching and Other Quadratic Assignment Problems , 1991, Neural Comput..

[41]  Martin W. Simmen Parameter Sensitivity of the Elastic Net Approach to the Traveling Salesman Problem , 1991, Neural Computation.

[42]  T. Bayes An essay towards solving a problem in the doctrine of chances , 2003 .

[43]  A. Yuille,et al.  Track finding with deformable templates — the elastic arms approach , 1992 .

[44]  Jung-Hsien Chiang,et al.  Training neural pattern classifiers with a mean field theory learning algorithm , 1992 .