Volume Preserving Image Segmentation with Entropic Regularization Optimal Transport and Its Applications in Deep Learning

Image segmentation with a volume constraint is an important prior for many real applications. In this work, we present a novel volume preserving image segmentation algorithm, which is based on the framework of entropic regularized optimal transport theory. The classical Total Variation (TV) regularizer and volume preserving are integrated into a regularized optimal transport model, and the volume and classification constraints can be regarded as two measures preserving constraints in the optimal transport problem. By studying the dual problem, we develop a simple and efficient dual algorithm for our model. Moreover, to be different from many variational based image segmentation algorithms, the proposed algorithm can be directly unrolled to a new Volume Preserving and TV regularized softmax (VPTV-softmax) layer for semantic segmentation in the popular Deep Convolution Neural Network (DCNN). The experiment results show that our proposed model is very competitive and can improve the performance of many semantic segmentation nets such as the popular U-net.

[1]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transportation , 2013, NIPS 2013.

[2]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[3]  R. B. Potts.  Some generalized order-disorder transformations , 1952, Mathematical Proceedings of the Cambridge Philosophical Society.

[4]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[5]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[6]  Xavier Bresson,et al.  Local Histogram Based Segmentation Using the Wasserstein Distance , 2009, International Journal of Computer Vision.

[7]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[8]  Jack Xin,et al.  Diffusion-Generated Motion by Mean Curvature for Filaments , 2001, J. Nonlinear Sci..

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  Julien Rabin,et al.  Convex Histogram-Based Joint Image Segmentation with Regularized Optimal Transport Cost , 2016, Journal of Mathematical Imaging and Vision.

[11]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[12]  Xin Zheng,et al.  Fast and robust segmentation of white blood cell images by self-supervised learning. , 2018, Micron.

[13]  Nicolas Papadakis,et al.  A Convex Formulation for Global Histogram Based Binary Segmentation , 2013, EMMCVPR.

[14]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[16]  Xue-Cheng Tai,et al.  A fast segmentation method based on constraint optimization and its applications: Intensity inhomogeneity and texture segmentation , 2011, Pattern Recognit..

[17]  Mohamed-Jalal Fadili,et al.  Wasserstein active contours , 2012, 2012 19th IEEE International Conference on Image Processing.

[18]  Roberto Cominetti,et al.  Asymptotic analysis of the exponential penalty trajectory in linear programming , 1994, Math. Program..

[19]  Xue-Cheng Tai,et al.  A Weighted Dictionary Learning Model for Denoising Images Corrupted by Mixed Noise , 2013, IEEE Transactions on Image Processing.

[20]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[21]  S. Osher,et al.  Motion of multiple junctions: a level set approach , 1994 .

[22]  Jun Liu,et al.  A Regularized Convolutional Neural Network for Semantic Image Segmentation , 2019, Analysis and Applications.

[23]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[24]  Xue-Cheng Tai,et al.  Global Minimization for Continuous Multiphase Partitioning Problems Using a Dual Approach , 2011, International Journal of Computer Vision.

[25]  Marc Teboulle,et al.  A Unified Continuous Optimization Framework for Center-Based Clustering Methods , 2007, J. Mach. Learn. Res..

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[27]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[28]  Gabriel Peyré,et al.  A Smoothed Dual Approach for Variational Wasserstein Problems , 2015, SIAM J. Imaging Sci..

[29]  Jun Liu,et al.  Expectation-maximization algorithm with total variation regularization for vector-valued image segmentation , 2012, J. Vis. Commun. Image Represent..

[30]  Haohan Li,et al.  An efficient iterative thresholding method for image segmentation , 2016, J. Comput. Phys..

[31]  Felix Otto,et al.  Threshold Dynamics for Networks with Arbitrary Surface Tensions , 2015 .

[32]  Julien Rabin,et al.  Convex Color Image Segmentation with Optimal Transport Distances , 2015, SSVM.

[33]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[34]  Xue-Cheng Tai,et al.  Simultaneous Denoising and Illumination Correction via Local Data-Fidelity and Nonlocal Regularization , 2011, SSVM.

[35]  Julian Besag,et al.  Digital Image Processing: Towards Bayesian image analysis , 1989 .

[36]  J. Lorenz,et al.  On the scaling of multidimensional matrices , 1989 .

[37]  Haim H. Permuter,et al.  Gaussian mixture models of texture and colour for image database retrieval , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[38]  Selim Esedoglu,et al.  Auction dynamics: A volume constrained MBO scheme , 2018, J. Comput. Phys..