Deep snow: synthesizing remote sensing imagery with generative adversarial nets

In this work we demonstrate that generative adversarial networks (GANs) can be used to generate realistic pervasive changes in RGB remote sensing imagery, even in an unpaired training setting. We investigate some transformation quality metrics based on deep embedding of the generated and real images which enable visualization and understanding of the training dynamics of the GAN, and provide a useful measure in terms of quantifying how distinguishable the generated images are from real images. We also identify some artifacts introduced by the GAN in the generated images, which are likely to contribute to the differences seen between the real and generated samples in the deep embedding feature space even in cases where the real and generated samples appear perceptually similar.

[1]  Michael Schmitt,et al.  A Conditional Generative Adversarial Network to Fuse Sar And Multispectral Optical Data For Cloud Removal From Sentinel-2 Images , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[2]  Parvaneh Saeedi,et al.  Cloudmaskgan: A Content-Aware Unpaired Image-To-Image Translation Algorithm for Remote Sensing Imagery , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[3]  James Theiler,et al.  Proposed Framework for Anomalous Change Detection , 2006 .

[4]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[5]  Itaru Kitahara,et al.  SAR2OPT: Image Alignment Between Multi-Modal Images Using Generative Adversarial Networks , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[6]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[7]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[8]  Michael S. Warren,et al.  Visual search over billions of aerial and satellite images , 2019, Comput. Vis. Image Underst..

[9]  David W. Messinger,et al.  Algorithms, Technologies, and Applications for Multispectral and Hyperspectral Imagery XXV , 2019 .

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Qian Du,et al.  Multi-Modal Change Detection, Application to the Detection of Flooded Areas: Outcome of the 2009–2010 Data Fusion Contest , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  James Theiler,et al.  Multi-sensor anomalous change detection at scale , 2019, Defense + Commercial Sensing.

[13]  Corentin Henry,et al.  SAR-to-Optical Image Translation Based on Conditional Generative Adversarial Networks - Optimization, Opportunities and Limits , 2019, Remote. Sens..

[14]  Shan Zhong,et al.  SAR Image Colorization Using Multidomain Cycle-Consistency Generative Adversarial Network , 2021, IEEE Geoscience and Remote Sensing Letters.

[15]  Redha Touati,et al.  Multimodal Change Detection in Remote Sensing Images Using an Unsupervised Pixel Pairwise-Based Markov Random Field Model , 2020, IEEE Transactions on Image Processing.

[16]  Xiao Xiang Zhu,et al.  The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion , 2018, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[17]  James Theiler,et al.  Cycle-Consistent Adversarial Networks for Realistic Pervasive Change Generation in Remote Sensing Imagery , 2020, 2020 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI).

[18]  D. Dowson,et al.  The Fréchet distance between multivariate normal distributions , 1982 .

[19]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[21]  Xiao Xiang Zhu,et al.  SEN12MS - A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion , 2019, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[22]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).