Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation

Many material and biological samples in scientific imaging are characterized by nonlocal repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a two-dimensional image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and minimize sample damage caused by the electron beam. In this paper, we present an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the nonlocal redundancy in images. We adapt a framework, termed plug-and-play priors, to solve these imaging problems in a regularized inversion setting. The power of the plug-and-play approach is that it allows a wide array of modern denoising algorithms to be used as a “prior model” for tomography and image interpolation. We also present sufficient mathematical conditions that ensure convergence of the plug-and-play approach, and we use these insights to design a new nonlocal means denoising algorithm. Finally, we demonstrate that the algorithm produces higher quality reconstructions on both simulated and real electron microscope data, along with improved convergence properties compared to other methods.

[1]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[2]  Y. Katznelson An Introduction to Harmonic Analysis: Interpolation of Linear Operators , 1968 .

[3]  David Yevick,et al.  Calculus of Vector Functions , 2014 .

[4]  Jeff Simmons,et al.  Three-dimensional analysis of secondary γ′ precipitates in René-88 DT and UMF-20 superalloys , 2009 .

[5]  Wolfgang Baumeister,et al.  Three-Dimensional Structure of Herpes Simplex Virus from Cryo-Electron Tomography , 2003, Science.

[6]  Euhanna Ghadimi,et al.  Optimal Parameter Selection for the Alternating Direction Method of Multipliers (ADMM): Quadratic Problems , 2013, IEEE Transactions on Automatic Control.

[7]  Alexander Wong,et al.  A nonlocal-means approach to exemplar-based inpainting , 2008, 2008 15th IEEE International Conference on Image Processing.

[8]  Tieyong Zeng,et al.  A Universal Variational Framework for Sparsity-Based Image Inpainting , 2014, IEEE Transactions on Image Processing.

[9]  Guillermo Sapiro,et al.  Compressive Sensing by Learning a Gaussian Mixture Model From Measurements , 2015, IEEE Transactions on Image Processing.

[10]  Huixi Zhao,et al.  A unified Markov random field/marked point process image model and its application to computational materials , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[11]  Charles A. Bouman,et al.  Submitted to Ieee Transactions on Image Processing 1 a Model Based Iterative Reconstruction Algorithm for High Angle Annular Dark Field -scanning Transmission Electron Microscope (haadf-stem) Tomography , 2022 .

[12]  L. Carin,et al.  The potential for Bayesian compressive sensing to significantly reduce electron dose in high-resolution STEM images. , 2014, Microscopy.

[13]  Christine Guillemot,et al.  Image Inpainting : Overview and Recent Advances , 2014, IEEE Signal Processing Magazine.

[14]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[15]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[16]  Benjamin Recht,et al.  Sharp Time–Data Tradeoffs for Linear Inverse Problems , 2015, IEEE Transactions on Information Theory.

[17]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[18]  Chengchun Hao Introduction to Harmonic Analysis , 2016 .

[19]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[20]  D. Hoaglin,et al.  ON THE ROBUST ESTIMATION OF ECONOMETRIC MODELS , 1997 .

[21]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[22]  Mathews Jacob,et al.  Nonlocal Regularization of Inverse Problems: A Unified Variational Framework , 2013, IEEE Transactions on Image Processing.

[23]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical , 2013, IEEE Signal Processing Magazine.

[24]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[25]  Hyrum S. Anderson,et al.  Sparse imaging for fast electron microscopy , 2013, Electronic Imaging.

[26]  Michael Elad,et al.  Poisson Inverse Problems by the Plug-and-Play scheme , 2015, J. Vis. Commun. Image Represent..

[27]  Marc De Graef,et al.  Introduction to Conventional Transmission Electron Microscopy , 2003 .

[28]  Charles A. Bouman,et al.  Model-Based Iterative Reconstruction for Bright-Field Electron Tomography , 2013, IEEE Transactions on Computational Imaging.

[29]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[30]  S. Morris COWLES FOUNDATION FOR RESEARCH IN ECONOMICS , 2001 .

[31]  Ehsan Samei,et al.  Radiation Dose Reduction in Abdominal Computed Tomography During the Late Hepatic Arterial Phase Using a Model-Based Iterative Reconstruction Algorithm: How Low Can We Go? , 2012, Investigative radiology.

[32]  Charles A. Bouman,et al.  Advanced prior modeling for 3D bright field electron tomography , 2015, Electronic Imaging.

[33]  O. Perron Zur Theorie der Matrices , 1907 .

[34]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[35]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[36]  Vito Pascazio,et al.  Bayesian Regularization in Nonlinear Imaging: Reconstructions From Experimental Data in Nonlinearized Microwave Tomography , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[37]  M. Graef Introduction to Conventional Transmission Electron Microscopy: List of symbols , 2003 .

[38]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[39]  Peyman Milanfar,et al.  Motion deblurring with graph Laplacian regularization , 2015, Electronic Imaging.

[40]  José M. Bioucas-Dias,et al.  Image restoration and reconstruction using variable splitting and class-adapted image priors , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[41]  Z. Wang Introduction to Conventional Transmission Electron Microscopy , 2003 .

[42]  M. Slater Lagrange Multipliers Revisited , 2014 .

[43]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[44]  Peyman Milanfar,et al.  Symmetrizing Smoothing Filters , 2013, SIAM J. Imaging Sci..

[45]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[46]  Abraham J. Koster,et al.  Electron tomography in life science , 2009, Seminars in Cell & Developmental Biology.

[47]  Charles A. Bouman,et al.  A general framework for nonlinear multigrid inversion , 2005, IEEE Transactions on Image Processing.

[48]  Stephen P. Boyd,et al.  An ADMM Algorithm for a Class of Total Variation Regularized Estimation Problems , 2012, 1203.1828.

[49]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[50]  Shane Z. Sullivan,et al.  High frame-rate multichannel beam-scanning microscopy based on Lissajous trajectories. , 2014, Optics express.

[51]  Kari Pulli,et al.  FlexISP: a flexible camera image processing framework , 2014, ACM Trans. Graph..

[52]  Simon R. Cherry,et al.  Fast gradient-based methods for Bayesian reconstruction of transmission and emission PET images , 1994, IEEE Trans. Medical Imaging.

[53]  Ali Mohammad-Djafari,et al.  Joint estimation of parameters and hyperparameters in a Bayesian approach of solving inverse problems , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[54]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[55]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[56]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[57]  J. Moreau Proximité et dualité dans un espace hilbertien , 1965 .

[58]  Jianhua Ma,et al.  Nonlocal Prior Bayesian Tomographic Reconstruction , 2008, Journal of Mathematical Imaging and Vision.

[59]  Lawrence F. Drummy,et al.  ZnO Nanorod−Thermoplastic Polyurethane Nanocomposites: Morphology and Shape Memory Performance , 2009 .

[60]  Ken D. Sauer,et al.  A generalized Gaussian image model for edge-preserving MAP estimation , 1993, IEEE Trans. Image Process..

[61]  Peyman Milanfar,et al.  Patch-Based Near-Optimal Image Denoising , 2012, IEEE Transactions on Image Processing.

[62]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[63]  Karen O. Egiazarian,et al.  Compressed Sensing Image Reconstruction Via Recursive Spatially Adaptive Filtering , 2007, 2007 IEEE International Conference on Image Processing.

[64]  Jean-Baptiste Thibault,et al.  A three-dimensional statistical approach to improved image quality for multislice helical CT. , 2007, Medical physics.

[65]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[66]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[67]  P. Midgley,et al.  Electron tomography and holography in materials science. , 2009, Nature materials.

[68]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[69]  Peyman Milanfar,et al.  Global Image Denoising , 2014, IEEE Transactions on Image Processing.

[70]  Lei Zhang,et al.  Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling , 2013, IEEE Transactions on Image Processing.

[71]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..