Forward-backward filtering and penalized least-Squares optimization: A Unified framework

Abstract The paper proposes a framework for unification of the penalized least-squares optimization (PLSO) and forward-backward filtering scheme. It provides a mathematical proof that forward-backward filtering (zero-phase IIR filters) can be presented as instances of PLSO. On the basis of this result, the paper then represents a unifying approach to the design and implementation of forward-backward filtering and PLSO algorithms in the time and frequency domain. A new block-wise matrix formulation is also presented for implementing the PLSO and forward-backward filtering algorithms. The approach presented in this paper is particularly suited for understanding the task of zero-phase filters in the time domain and analyzing PLSO algorithms in the frequency domain. In this paper, we show that the task of a zero-phase digital Butterworth filter in the time domain is to fit the signal with impulse train and penalties on the derivatives of the fitted model. For a zero-phase digital Chebyshev filter, a linear combination of derivatives of the model is used in the penalty term.

[1]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[2]  Daniele Perrone,et al.  A Clearer Picture of Total Variation Blind Deconvolution , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[4]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[5]  Stphane Mallat,et al.  A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way , 2008 .

[6]  Laurent Condat,et al.  Discrete Total Variation: New Definition and Minimization , 2017, SIAM J. Imaging Sci..

[7]  Zuowei Shen,et al.  Dictionary Learning for Sparse Coding: Algorithms and Convergence Analysis , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  James O. Ramsay,et al.  Functional Data Analysis , 2005 .

[9]  Thierry Blu,et al.  Generalized smoothing splines and the optimal discretization of the Wiener filter , 2005, IEEE Transactions on Signal Processing.

[10]  Edmund Taylor Whittaker.  On a New Method of Graduation , 1922, Proceedings of the Edinburgh Mathematical Society.

[11]  Erich Schröger,et al.  Digital filter design for electrophysiological data – a practical approach , 2015, Journal of Neuroscience Methods.

[12]  Laurent Condat,et al.  A Direct Algorithm for 1-D Total Variation Denoising , 2013, IEEE Signal Processing Letters.

[13]  Alan V. Oppenheim,et al.  Discrete-Time Signal Pro-cessing , 1989 .

[14]  Ivo Senjanović.  Harmonic acceleration method for dynamic structural analysis , 1984 .

[15]  Graham C. Goodwin,et al.  Digital control and estimation : a unified approach , 1990 .

[16]  Les Thede,et al.  Practical Analog And Digital Filter Design , 2004 .

[17]  Javier F. Pena,et al.  Optimization Methods in Finance: Stochastic programming: theory and algorithms , 2006 .

[18]  Manya V. Afonso,et al.  Blind Inpainting Using $\ell _{0}$ and Total Variation Regularization , 2015, IEEE Transactions on Image Processing.

[19]  Stephen P. Boyd,et al.  Linear controller design: limits of performance , 1991 .

[20]  Arman Kheirati Roonizi.  A New Approach to ARMAX Signals Smoothing: Application to Variable-Q ARMA Filter Design , 2019, IEEE Transactions on Signal Processing.

[21]  R. Shiller.  A DISTRIBUTED LAG ESTIMATOR DERIVED FROM SMOOTHNESS PRIORS , 1973 .

[22]  Zhaoran Wang,et al.  OPTIMAL COMPUTATIONAL AND STATISTICAL RATES OF CONVERGENCE FOR SPARSE NONCONVEX LEARNING PROBLEMS. , 2013, Annals of statistics.

[23]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[24]  Valeria Villani,et al.  Baseline wander removal for bioelectrical signals by quadratic variation reduction , 2014, Signal Process..

[25]  Guy Gilboa,et al.  Spectral Total-Variation Local Scale Signatures for Image Manipulation and Fusion , 2019, IEEE Transactions on Image Processing.

[26]  Mathews Jacob,et al.  Generalized Higher Degree Total Variation (HDTV) Regularization , 2014, IEEE Transactions on Image Processing.

[27]  Steven W. Smith,et al.  The Scientist and Engineer's Guide to Digital Signal Processing , 1997 .

[28]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications , 1949 .

[29]  Tony F. Chan,et al.  High-Order Total Variation-Based Image Restoration , 2000, SIAM J. Sci. Comput..

[30]  M. Dahleh,et al.  Control of Uncertain Systems: A Linear Programming Approach , 1995 .

[31]  Michael A. West,et al.  Time Series: Modeling, Computation, and Inference , 2010 .

[32]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[33]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[34]  Gérard Cornuéjols,et al.  Optimization Methods in Finance: Stochastic programming: theory and algorithms , 2006 .

[35]  Christian Jutten,et al.  Improved smoothness priors using bilinear transform , 2020, Signal Process..

[36]  G. Kitagawa,et al.  A Smoothness Priors–State Space Modeling of Time Series with Trend and Seasonality , 1984 .

[37]  David Zhang,et al.  A Survey of Sparse Representation: Algorithms and Applications , 2015, IEEE Access.

[38]  Shun-ichi Amari,et al.  $\ell _{p}$ -Regularized Least Squares $(0 and Critical Path , 2016 .

[39]  Hugo Van hamme,et al.  Identification of linear dynamic systems using piecewise constant excitations: Use, misuse and alternatives , 1994, Autom..

[40]  Katsuhiko Ogata,et al.  Discrete-time control systems , 1987 .

[41]  S. A. Dyer,et al.  The bilinear transformation , 2000 .

[42]  Fredrik Gustafsson,et al.  Determining the initial states in forward-backward filtering , 1996, IEEE Trans. Signal Process..

[43]  Michael Unser,et al.  Recursive Regularization Filters: Design, Properties, and Applications , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Reza Sameni,et al.  Online filtering using piecewise smoothness priors: Application to normal and abnormal electrocardiogram denoising , 2017, Signal Process..

[45]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[46]  Julius O. Smith,et al.  Introduction to Digital Filters: with Audio Applications , 2007 .

[47]  Mathews Jacob,et al.  Higher Degree Total Variation (HDTV) Regularization for Image Recovery , 2012, IEEE Transactions on Image Processing.

[48]  Per Christian Hansen,et al.  Analysis of Discrete Ill-Posed Problems by Means of the L-Curve , 1992, SIAM Rev..