Multimodal approach to feature extraction for image and signal learning problems

We present ZEUS, an algorithm for extracting features from images and time series signals. ZEIS is designed to solve a variety of machine learning problems including time series forecasting, signal classification, image and pixel classification of multispectral and panchromatic imagery. An evolutionary approach is used to extract features from a near-infinite space of possible combinations of nonlinear operators. Each problem type (i.e. signal or image, regression or classification, multiclass or binary) has its own set of primitive operators. We employ fairly generic operators, but note that the choice of which operators to use provides an opportunity to consult with a domain expert. Each feature is produced from a composition of some subset of these primitive operators. The fitness for an evolved set of features is given by the performance of a back-end classifier (or regressor) on training data. We demonstrate our multimodal approach to feature extraction on a variety of problems in remote sensing. The performance of this algorithm will be compared to standard approaches, and the relative benefit of various aspects of the algorithm will be investigated.

[1]  Simon J. Perkins,et al.  Support vector machines for broad-area feature classification in remotely sensed images , 2001, SPIE Defense + Commercial Sensing.

[2]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[3]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[4]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[5]  Neal R. Harvey,et al.  Automated simultaneous multiple feature classification of MTI data , 2002, SPIE Defense + Commercial Sensing.

[6]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[7]  Neal R. Harvey,et al.  Evolving retrieval algorithms with a genetic programming scheme , 1999, Optics & Photonics.

[8]  Melanie Mitchell,et al.  Investigation of image feature extraction by a genetic algorithm , 1999, Optics + Photonics.

[9]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[10]  Neal R. Harvey,et al.  Comparison of GENIE and conventional supervised classifiers for multispectral image feature extraction , 2002, IEEE Trans. Geosci. Remote. Sens..

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1995, COLT '90.

[13]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[14]  Simon J. Perkins,et al.  Genetic Algorithms and Support Vector Machines for Time Series Classification , 2002, Optics + Photonics.

[15]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[16]  Don R. Hush,et al.  Polynomial-Time Decomposition Algorithms for Support Vector Machines , 2003, Machine Learning.

[17]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[18]  Don R. Hush,et al.  Weighted Order Statistic Classifiers with Large Rank-Order Margin , 2003, ICML.

[19]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .