An Evolutionary Multiobjective Approach to Sparse Reconstruction

This paper addresses the problem of finding sparse solutions to linear systems. Although this problem involves two competing cost function terms (measurement error and a sparsity-inducing term), previous approaches combine these into a single cost term and solve the problem using conventional numerical optimization methods. In contrast, the main contribution of this paper is to use a multiobjective approach. The paper begins by investigating the sparse reconstruction problem, and presents data to show that knee regions do exist on the Pareto front (PF) for this problem and that optimal solutions can be found in these knee regions. Another contribution of the paper, a new soft-thresholding evolutionary multiobjective algorithm (StEMO), is then presented, which uses a soft-thresholding technique to incorporate two additional heuristics: one with greater chance to increase speed of convergence toward the PF, and another with higher probability to improve the spread of solutions along the PF, enabling an optimal solution to be found in the knee region. Experiments are presented, which show that StEMO significantly outperforms five other well known techniques that are commonly used for sparse reconstruction. Practical applications are also demonstrated to fundamental problems of recovering signals and images from noisy data.

[1]  J. D. Schaffer,et al.  Real-Coded Genetic Algorithms and Interval-Schemata , 1992, FOGA.

[2]  L. J. Eshelman,et al.  chapter Real-Coded Genetic Algorithms and Interval-Schemata , 1993 .

[3]  Zhifeng Zhang,et al.  Adaptive Nonlinear Approximations , 1994 .

[4]  Amara Lynn Graps,et al.  An introduction to wavelets , 1995 .

[5]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[6]  A. Neubauer,et al.  A theoretical analysis of the non-uniform mutation operator for the modified genetic algorithm , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[7]  Vladimir N. Temlyakov,et al.  The best m-term approximation and greedy algorithms , 1998, Adv. Comput. Math..

[8]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[9]  Indraneel Das On characterizing the “knee” of the Pareto curve based on Normal-Boundary Intersection , 1999 .

[10]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[11]  U. Aickelin,et al.  The Application of Bayesian Optimization and Classifier Systems in Nurse Scheduling , 2004, PPSN.

[12]  Xin Yao,et al.  Digital filter design using multiple pareto fronts , 2004, Soft Comput..

[13]  Kalyanmoy Deb,et al.  Finding Knees in Multi-objective Optimization , 2004, PPSN.

[14]  Kim-Fung Man,et al.  Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction , 2005, Fuzzy Sets Syst..

[15]  Dmitry M. Malioutov,et al.  Homotopy continuation for sparse signal representation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[16]  Kalyanmoy Deb,et al.  Improved Pruning of Non-Dominated Solutions Based on Crowding Distance for Bi-Objective Optimization Problems , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[17]  Yaochu Jin,et al.  Multi-Objective Machine Learning , 2006, Studies in Computational Intelligence.

[18]  Yong Wang,et al.  A Multiobjective Optimization-Based Evolutionary Algorithm for Constrained Optimization , 2006, IEEE Transactions on Evolutionary Computation.

[19]  Xin Yao,et al.  Evolving hybrid ensembles of learning machines for better generalisation , 2006, Neurocomputing.

[20]  Joel A. Tropp,et al.  Sparse Approximation Via Iterative Thresholding , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[21]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[22]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[23]  Rayan Saab,et al.  Sparco: A Testing Framework for Sparse Reconstruction , 2007 .

[24]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[25]  Yaochu Jin,et al.  Single/Multi-objective Inverse Robust Evolutionary Design Methodology in the Presence of Uncertainty , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[26]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[27]  Bernhard Sendhoff,et al.  Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[28]  Yaakov Tsaig,et al.  Fast Solution of $\ell _{1}$ -Norm Minimization Problems When the Solution May Be Sparse , 2008, IEEE Transactions on Information Theory.

[29]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[30]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[31]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[32]  E. Candès,et al.  Near-ideal model selection by ℓ1 minimization , 2008, 0801.0345.

[33]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[34]  Lily Rachmawati,et al.  Multiobjective Evolutionary Algorithm With Controllable Focus on the Knees of the Pareto Front , 2009, IEEE Transactions on Evolutionary Computation.

[35]  Allen Y. Yang,et al.  Fast ℓ1-minimization algorithms and an application in robust face recognition: A review , 2010, 2010 IEEE International Conference on Image Processing.

[36]  Maoguo Gong,et al.  ADAPTIVE RANKS CLONE AND k‐NEAREST NEIGHBOR LIST–BASED IMMUNE MULTI‐OBJECTIVE OPTIMIZATION , 2010, Comput. Intell..

[37]  Mike E. Davies,et al.  Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance , 2010, IEEE Journal of Selected Topics in Signal Processing.

[38]  Huanhuan Chen,et al.  Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[39]  Xin Yao,et al.  Multi-Objective Approaches to Optimal Testing Resource Allocation in Modular Software Systems , 2010, IEEE Transactions on Reliability.

[40]  Fang Liu,et al.  Compressive Sensing SAR Image Reconstruction Based on Bayesian Framework and Evolutionary Computation , 2011, IEEE Transactions on Image Processing.

[41]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[42]  Xin Yao,et al.  Software Module Clustering as a Multi-Objective Search Problem , 2011, IEEE Transactions on Software Engineering.

[43]  K. Deb,et al.  Understanding knee points in bicriteria problems and their implications as preferred solution principles , 2011 .

[44]  Zongben Xu,et al.  $L_{1/2}$ Regularization: A Thresholding Representation Theory and a Fast Solver , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Fang Liu,et al.  A novel selection evolutionary strategy for constrained optimization , 2013, Inf. Sci..

[46]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.