Neural networks in geophysical applications

Neural networks are increasingly popular in geophysics. Because they are universal approximators, these tools can approximate any continuous function with an arbitrary precision. Hence, they may yield important contributions to finding solutions to a variety of geophysical applications. However, knowledge of many methods and techniques recently developed to increase the performance and to facilitate the use of neural networks does not seem to be widespread in the geophysical community. Therefore, the power of these tools has not yet been explored to their full extent. In this paper, techniques are described for faster training, better overall performance, i.e., generalization, and the automatic estimation of network size and architecture.

[1]  Ishwar K. Sethi.  Neural implementation of tree classifiers , 1995, IEEE Trans. Syst. Man Cybern..

[2]  Michael R. Berthold,et al.  Boosting the Performance of RBF Networks with Dynamic Decay Adjustment , 1994, NIPS.

[3]  H. Akaike.  Statistical predictor identification , 1970 .

[4]  Christian Jutten,et al.  DWINA: Depth and Width Incremental Neural Algorithm , 1996 .

[5]  Yves Grandvalet,et al.  Comments on "Noise injection into inputs in back propagation learning" , 1995, IEEE Trans. Syst. Man Cybern..

[6]  Christopher M. Bishop,et al.  Training with Noise is Equivalent to Tikhonov Regularization , 1995, Neural Computation.

[7]  S. Treitel,et al.  A REVIEW OF LEAST-SQUARES INVERSION AND ITS APPLICATION TO GEOPHYSICAL PROBLEMS* , 1984 .

[8]  D.R. Hush,et al.  Progress in supervised neural networks , 1993, IEEE Signal Processing Magazine.

[9]  K. V. Paulson,et al.  Magnetotelluric inversion using regularized Hopfield neural networks , 1997 .

[10]  S. Grossberg,et al.  ART 2: self-organization of stable category recognition codes for analog input patterns. , 1987, Applied optics.

[11]  Roel Snieder,et al.  What is noise , 1998 .

[12]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[13]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[14]  Marie Cottrell,et al.  Neural modeling for time series: A statistical stepwise method for weight elimination , 1995, IEEE Trans. Neural Networks.

[15]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[16]  Anna Maria Fanelli,et al.  A Method of Pruning Layered Feed-Forward Neural Networks , 1993, IWANN.

[17]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[18]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[19]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[20]  Peter Schmidt,et al.  The Theory and Practice of Econometrics , 1985 .

[21]  Luigi Occhipinti,et al.  Estimation of seismic waveform governing parameters with neural networks , 1996 .

[22]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[23]  Peter M. Williams,et al.  Bayesian Regularization and Pruning Using a Laplace Prior , 1995, Neural Computation.

[24]  H. A. van der Vorst,et al.  Numerical solution of large, sparse linear algebraic systems arising from tomographic problems , 1987 .

[25]  Lorien Y. Pratt,et al.  Comparing Biases for Minimal Network Construction with Back-Propagation , 1988, NIPS.

[26]  Mary M. Poulton,et al.  Location of subsurface targets in geophysical data using neural networks , 1992 .

[27]  David S. Touretzky,et al.  Advances in neural information processing systems 2 , 1989 .

[28]  Christian Jutten,et al.  Combining sigmoids and radial basis functions in evolutive neural architectures , 1996, ESANN.

[29]  Kanter,et al.  Eigenvalues of covariance matrices: Application to neural-network learning. , 1991, Physical review letters.

[30]  Christian Jutten,et al.  Simultaneous Estimation of Signal and Noise in the Constructive Neural Networks , 1998, ICSC Symposium on Neural Computation.

[31]  A. Tarantola.  Inverse problem theory : methods for data fitting and model parameter estimation , 1987 .

[32]  P. GALLINARI,et al.  On the relations between discriminant analysis and multilayer perceptrons , 1991, Neural Networks.

[33]  D. E. Rumelhart,et al.  Learning internal representations by back-propagating errors , 1986 .

[34]  Stephen Grossberg,et al.  Art 2: Self-Organization Of Stable Category Recognition Codes For Analog Input Patterns , 1988, Other Conferences.

[35]  Don R. Hush,et al.  Error surfaces for multilayer perceptrons , 1992, IEEE Trans. Syst. Man Cybern..

[36]  Colin MacBeth,et al.  Split shear-wave analysis using an artificial neural network ? , 1994 .

[37]  Michael C. Mozer,et al.  Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment , 1988, NIPS.

[38]  H. Akaike.  Statistical predictor identification , 1970 .

[39]  Farid U. Dowla,et al.  Seismic discrimination with artificial neural networks: Preliminary results with regional spectral data , 1990 .

[40]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[41]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[42]  G. Wahba,et al.  A completely automatic french curve: fitting spline functions by cross validation , 1975 .

[43]  Jenq-Neng Hwang,et al.  Regression modeling in back-propagation and projection pursuit learning , 1994, IEEE Trans. Neural Networks.

[44]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[45]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[46]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[47]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[49]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[50]  Ehud D. Karnin,et al.  A simple procedure for pruning back-propagation trained neural networks , 1990, IEEE Trans. Neural Networks.

[51]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[52]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[53]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[54]  Jocelyn Sietsma,et al.  Creating artificial neural networks that generalize , 1991, Neural Networks.

[55]  Harry Wechsler,et al.  From Statistics to Neural Networks , 1994, NATO ASI Series.

[56]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[57]  R. Chentouf.  Construction de reseaux de neurones multicouches pour l'approximation , 1997 .

[58]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[59]  Min I. Chung.  Neural network as generalized transversal filters , 1988, Neural Networks.

[60]  John Moody,et al.  Prediction Risk and Architecture Selection for Neural Networks , 1994 .

[61]  Michael E. Murat,et al.  AUTOMATED FIRST ARRIVAL PICKING: A NEURAL NETWORK APPROACH1 , 1992 .

[62]  Harry Wechsler,et al.  From Statistics to Neural Networks: Theory and Pattern Recognition Applications , 1996 .

[63]  James T. Kwok,et al.  Constructive algorithms for structure learning in feedforward neural networks for regression problems , 1997, IEEE Trans. Neural Networks.

[64]  Thierry Denoeux,et al.  Initializing back propagation networks with prototypes , 1993, Neural Networks.

[65]  Mirko van der Baan,et al.  Recognition and reconstruction of coherent energy with application to deep seismic reflection data , 2000 .

[66]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[67]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[68]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[69]  Timothy Masters,et al.  Advanced algorithms for neural networks: a C++ sourcebook , 1995 .

[70]  Jerry M. Mendel,et al.  Adaptive minimum prediction-error deconvolution and source wavelet estimation using Hopfield neural networks , 1992 .

[71]  Tsoung-Chao Lee,et al.  The Theory and Practice of Econometrics , 1981 .

[72]  Ishwar K. Sethi,et al.  Entropy nets: from decision trees to neural networks , 1990, Proc. IEEE.

[73]  Yih-Fang Huang,et al.  Bounds on the number of hidden neurons in multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[74]  Zehui Huang,et al.  Permeability prediction with artificial neural network modeling in the Venture gas field, offshore eastern Canada , 1996 .

[75]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[76]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[77]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[78]  G. Judge,et al.  The Theory and Practice of Econometrics , 1981 .

[79]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[80]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[81]  Michael D. McCormack,et al.  FIRST-BREAK REFRACTION EVENT PICKING AND SEISMIC DATA TRACE EDITING USING NEURAL NETWORKS , 1993 .

[82]  Bernard Widrow,et al.  Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[83]  Albert Tarantola,et al.  Neural networks and inversion of seismic data , 1994 .

[84]  Luís B. Almeida.  Neural Preprocessing Methods , 1994 .

[85]  G. Romeo.  Seismic signals detection and classification using artiricial neural networks , 1994 .

[86]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[87]  Kiyotoshi Matsuoka,et al.  Noise injection into inputs in back-propagation learning , 1992, IEEE Trans. Syst. Man Cybern..

[88]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[89]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[90]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[91]  Mrinal K. Sen,et al.  Automatic NMO correction and velocity estimation by a feedforward neural network , 1998 .

[92]  Mrinal K. Sen,et al.  Hopfield neural networks, and mean field annealing for seismic deconvolution and multiple attenuation , 1997 .

[93]  John A. Richards,et al.  Remote Sensing Digital Image Analysis: An Introduction , 1999 .