A Spectral-Temporal Patch-Based Missing Area Reconstruction for Time-Series Images

Clouds, cloud shadows (CCS), and numerous other factors will cause a missing data problem in passive remote sensing images. A well-known reconstruction method is the selection of a similar pixel (with an additional clear reference image) from the remaining clear part of an image to replace the missing pixel. Due to the merit of filling the missing value using a pixel acquired on the same image with the same sensor and the same date, this method is suitable for time-series applications when a time-series profile-based similar measure is utilized for selecting the similar pixel. Since the similar pixel is independently selected, the improper reference pixel or various accuracies obtained by different land covers causes the problem of salt-and-pepper noise in the reconstructed part of an image. To overcome these problems, this paper presents a spectral–temporal patch (STP)-based missing area reconstruction method for time-series images. First, the STP, the pixels of which have similar spectral and temporal evolution characteristics, is extracted using multi-temporal image segmentation. However, some STP have Missing Observations (STPMO) in the time series, which should be reconstructed. Next, for an STPMO, the most similar STP is selected as the reference STP; then, the mean and standard deviation of the STPMO is predicted using a linear regression method with the reference STP. Finally, the textural information, which is denoted by the spatial configuration of color or intensities of neighboring pixels, is extracted from the clear temporal-adjacent STP and “injected” into the missing area to obtain synthetic cloud-free images. We performed an STP-based missing area reconstruction experiment in Jiangzhou, Chongzuo, Guangxi with time-series images acquired by wide field view (WFV) onboard Chinese Gao Fen 1 on 12 different dates. The results indicate that the proposed method can effectively recover the missing information without salt-and-pepper noise in the reconstructed area; also, the reconstructed part of the image is consistent with the clear part without a false edge. The results confirm that the spectral information from the remaining clear part of the same image and textural information from the temporal-adjacent image can create seamless time-series images.

[1]  Farid Melgani,et al.  Missing-Area Reconstruction in Multispectral Images Under a Compressive Sensing Perspective , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Chao-Hung Lin,et al.  Cloud Removal From Multitemporal Satellite Images Using Information Cloning , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Farid Melgani,et al.  Contextual reconstruction of cloud-contaminated multitemporal multispectral images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Lin Yan,et al.  Improved time series land cover classification by missing-observation-adaptive nonlinear dimensionality reduction , 2015 .

[5]  Y. J. Kaufman,et al.  The effect of subpixel clouds on remote sensing , 1987 .

[6]  Shaowen Wang,et al.  A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach , 2018, Remote Sensing of Environment.

[7]  Lammert Kooistra,et al.  Reconstructing land use history from Landsat time-series: Case study of a swidden agriculture system in Brazil , 2016, Int. J. Appl. Earth Obs. Geoinformation.

[8]  G. L. Schmidt,et al.  A multi‐scale segmentation approach to filling gaps in Landsat ETM+ SLC‐off images , 2007 .

[9]  Lin Yan,et al.  Large-Area Gap Filling of Landsat Reflectance Time Series by Spectral-Angle-Mapper Based Spatio-Temporal Similarity (SAMSTS) , 2018, Remote. Sens..

[10]  Feng Gao,et al.  A Modified Neighborhood Similar Pixel Interpolator Approach for Removing Thick Clouds in Landsat Images , 2012, IEEE Geoscience and Remote Sensing Letters.

[11]  Min Zhang,et al.  Scale parameter selection by spatial statistics for GeOBIA: Using mean-shift based multi-scale segmentation as an example , 2015 .

[12]  Shmuel Peleg,et al.  Seamless image stitching by minimizing false edges , 2006, IEEE Transactions on Image Processing.

[13]  Tim R. McVicar,et al.  Determining temporal windows for crop discrimination with remote sensing: a case study in south-eastern Australia , 2004 .

[14]  E. Helmer,et al.  A comparison of radiometric normalization methods when filling cloud gaps in Landsat imagery , 2007 .

[15]  Liangpei Zhang,et al.  Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF model , 2014 .

[16]  Bin Chen,et al.  Spatially and Temporally Weighted Regression: A Novel Method to Produce Continuous Cloud-Free Landsat Imagery , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Christine Fernandez-Maloigne,et al.  A Bandelet-Based Inpainting Technique for Clouds Removal From Remotely Sensed Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Pierre Soille,et al.  Morphological image compositing , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Guifeng Zhang,et al.  An Edge Embedded Marker-Based Watershed Algorithm for High Spatial Resolution Remote Sensing Image Segmentation , 2010, IEEE Transactions on Image Processing.

[21]  E. Helmer,et al.  Cloud-Free Satellite Image Mosaics with Regression Trees and Histogram Matching. , 2005 .

[22]  Patrick Bogaert,et al.  Forest change detection by statistical object-based method , 2006 .

[23]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[24]  Rasim Latifovic,et al.  Reconstruction of Landsat time series in the presence of irregular and sparse observations: Development and assessment in north-eastern Alberta, Canada , 2018 .

[25]  Lonesome Malambo,et al.  A Multitemporal Profile-Based Interpolation Method for Gap Filling Nonstationary Data , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Yuliya Tarabalka,et al.  Best Merge Region-Growing Segmentation With Integrated Nonadjacent Region Object Aggregation , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[28]  S. Goward,et al.  An automated approach for reconstructing recent forest disturbance history using dense Landsat time series stacks , 2010 .

[29]  Dirk Pflugmacher,et al.  Mapping the timing of cropland abandonment and recultivation in northern Kazakhstan using annual Landsat time series , 2018 .

[30]  Yanfeng Gu,et al.  Multitemporal Landsat Missing Data Recovery Based on Tempo-Spectral Angle Model , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Weidong Li,et al.  Restoration of clouded pixels in multispectral remotely sensed imagery with cokriging , 2009 .

[32]  Clement Atzberger,et al.  Smoothing and gap-filling of high resolution multi-spectral time series: Example of Landsat data , 2016, Int. J. Appl. Earth Obs. Geoinformation.

[33]  B. Holben.  Characteristics of maximum-value composite images from temporal AVHRR data , 1986 .

[34]  Julien Michel,et al.  Stable Mean-Shift Algorithm and Its Application to the Segmentation of Arbitrarily Large Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Valerie A. Thomas,et al.  Fitting the Multitemporal Curve: A Fourier Series Approach to the Missing Data Problem in Remote Sensing Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[36]  J. Cihlar,et al.  An image transform to characterize and compensate for spatial variations in thin cloud contamination of Landsat images , 2002 .