A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization

Hyperspectral remote sensing images (HSIs) usually have high spectral resolution and low spatial resolution. Conversely, multispectral images (MSIs) usually have low spectral and high spatial resolutions. The problem of inferring images that combine the high spectral and high spatial resolutions of HSIs and MSIs, respectively, is a data fusion problem that has been the focus of recent active research due to the increasing availability of HSIs and MSIs retrieved from the same geographical area. We formulate this problem as the minimization of a convex objective function containing two quadratic data-fitting terms and an edge-preserving regularizer. The data-fitting terms account for blur, different resolutions, and additive noise. The regularizer, a form of vector total variation, promotes piecewise-smooth solutions with discontinuities aligned across the hyperspectral bands. The downsampling operator accounting for the different spatial resolutions, the nonquadratic and nonsmooth nature of the regularizer, and the very large size of the HSI to be estimated lead to a hard optimization problem. We deal with these difficulties by exploiting the fact that HSIs generally “live” in a low-dimensional subspace and by tailoring the split augmented Lagrangian shrinkage algorithm (SALSA), which is an instance of the alternating direction method of multipliers (ADMM), to this optimization problem, by means of a convenient variable splitting. The spatial blur and the spectral linear operators linked, respectively, with the HSI and MSI acquisition processes are also estimated, and we obtain an effective algorithm that outperforms the state of the art, as illustrated in a series of experiments with simulated and real-life data.

[1]  Mário A. T. Figueiredo,et al.  Deconvolving Images With Unknown Boundaries Using the Alternating Direction Method of Multipliers , 2012, IEEE Transactions on Image Processing.

[2]  Fionn Murtagh,et al.  Image restoration with noise suppression using the wavelet transform , 1994 .

[3]  Yücel Altunbasak,et al.  Super-resolution reconstruction of hyperspectral images , 2004, IEEE Transactions on Image Processing.

[4]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[5]  Aggelos K. Katsaggelos,et al.  A survey of classical methods and new trends in pansharpening of multispectral images , 2011, EURASIP J. Adv. Signal Process..

[6]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[7]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2013, J. Optim. Theory Appl..

[9]  Amandine Robin,et al.  Determining the Intrinsic Dimension of a Hyperspectral Image Using Random Matrix Theory , 2013, IEEE Transactions on Image Processing.

[10]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[11]  Guangjian Yan,et al.  Improved Methods for Spectral Calibration of On-Orbit Imaging Spectrometers , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Jocelyn Chanussot,et al.  A New Pansharpening Method Based on Spatial and Spectral Sparsity Priors , 2014, IEEE Transactions on Image Processing.

[13]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[14]  Liangpei Zhang,et al.  Hyperspectral Image Denoising Employing a Spectral–Spatial Adaptive Total Variation Model , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Quan Pan,et al.  Hyperspectral imagery super-resolution by sparse representation and spectral regularization , 2011, EURASIP J. Adv. Signal Process..

[16]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[17]  Bruno A. Olshausen,et al.  Learning Sparse Codes for Hyperspectral Imagery , 2011, IEEE Journal of Selected Topics in Signal Processing.

[18]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  K. Egiazarian,et al.  Blind image deconvolution , 2007 .

[20]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[21]  T. Chan,et al.  Fast dual minimization of the vectorial total variation norm and applications to color image processing , 2008 .

[22]  Jean-Luc Starck,et al.  Deconvolution and Blind Deconvolution in Astronomy , 2007 .

[23]  Yifan Zhang,et al.  Noise-Resistant Wavelet-Based Bayesian Fusion of Multispectral and Hyperspectral Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[25]  Kaihua Zhang,et al.  Spatio-spectral fusion of satellite images based on dictionary-pair learning , 2014, Inf. Fusion.

[26]  Russell C. Hardie,et al.  MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor , 2004, IEEE Transactions on Image Processing.

[27]  Zongben Xu,et al.  Spatial and Spectral Image Fusion Using Sparse Matrix Factorization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[28]  José M. Bioucas-Dias,et al.  Hyperspectral Subspace Identification , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Jeffrey A. Fessler,et al.  Accelerated Edge-Preserving Image Restoration Without Boundary Artifacts , 2013, IEEE Transactions on Image Processing.

[30]  Michael K. Ng,et al.  Deblurring and Sparse Unmixing for Hyperspectral Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Peyman Milanfar,et al.  Robust Multichannel Blind Deconvolution via Fast Alternating Minimization , 2012, IEEE Transactions on Image Processing.

[32]  Jocelyn Chanussot,et al.  Fusion of hyperspectral and panchromatic images using multiresolution analysis and nonlinear PCA band reduction , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[33]  Michael E. Schaepman,et al.  Unmixing-Based Landsat TM and MERIS FR Data Fusion , 2008, IEEE Geoscience and Remote Sensing Letters.

[34]  Henry Leung,et al.  A Maximum Likelihood Approach to Joint Image Registration and Fusion , 2011, IEEE Transactions on Image Processing.

[35]  Jocelyn Chanussot,et al.  Pansharpening of Hyperspectral images using spatial distortion optimization , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[36]  H. Kramer Observation of the Earth and Its Environment , 1994 .

[37]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[38]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Jocelyn Chanussot,et al.  Hyperspectral image superresolution: An edge-preserving convex formulation , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[40]  Jean-Yves Tourneret,et al.  Bayesian fusion of hyperspectral and multispectral images , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[41]  John B. Greer,et al.  Sparse Demixing of Hyperspectral Images , 2012, IEEE Transactions on Image Processing.

[42]  Subhasis Chaudhuri,et al.  Hyperspectral Image Fusion , 2013, Springer New York.

[43]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[44]  Yifan Zhang,et al.  A Bayesian Restoration Approach for Hyperspectral Images , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[45]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[46]  Multispectral and Hyperspectral Pansharpening : A Critical Examination and New Developments , 2014 .

[47]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[48]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[49]  Te-Ming Tu,et al.  A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[50]  Lawrence Ong,et al.  The Earth Observing One (EO-1) Satellite Mission: Over a Decade in Space , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[51]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[52]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[53]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[54]  Junfeng Yang,et al.  ALTERNATING DIRECTION ALGORITHMS FOR TOTAL VARIATION DECONVOLUTION IN IMAGE RECONSTRUCTION , 2009 .

[55]  Naoto Yokoya,et al.  Cross-Calibration for Data Fusion of EO-1/Hyperion and Terra/ASTER , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[56]  Li Xu,et al.  Two-Phase Kernel Estimation for Robust Motion Deblurring , 2010, ECCV.

[57]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[58]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[59]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.