Bounds on bilinear inverse forms via Gaussian quadrature with applications

We address quadrature-based approximations of the bilinear inverse form $u^\top A^{-1} u$, where $A$ is a real symmetric positive definite matrix, and analyze properties of the Gauss, Gauss-Radau, and Gauss-Lobatto quadrature. In particular, we establish monotonicity of the bounds given by these quadrature rules, compare the tightness of these bounds, and derive associated convergence rates. To our knowledge, this is the first work to establish these properties of Gauss-type quadrature for computing bilinear inverse forms, thus filling a theoretical gap regarding this classical topic. We illustrate the empirical benefits of our theoretical results by applying quadrature to speed up two Markov Chain sampling procedures for (discrete) determinantal point processes.

[1]  R. Lobatto Lessen over de differentiaal- en integraal-rekening , 1851 .

[2]  R. Radau Étude sur les formules d'approximation qui servent à calculer la valeur numérique d'une intégrale définie. , 1880 .

[3]  J. Sherman,et al.  Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given Matrix , 1950 .

[4]  C. Lanczos An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[5]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[6]  W. Wasow A note on the inversion of matrices by random walks , 1952 .

[7]  T. Broadbent Mathematics for the Physical Sciences , 1959, Nature.

[8]  J. H. Wilkinson A HANDBOOK OF NUMERICAL MATRIX INVERSION AND SOLUTION OF LINEAR EQUATIONS , 1970 .

[9]  Christopher C. Paige,et al.  The computation of eigenvalues and eigenvectors of very large sparse matrices , 1971 .

[10]  H. Wilf Mathematics for the Physical Sciences , 1976 .

[11]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[12]  Michel Minoux,et al.  Accelerated greedy algorithms for maximizing submodular set functions , 1978 .

[13]  B. Parlett,et al.  The Lanczos algorithm with selective orthogonalization , 1979 .

[14]  W. Gautschi A Survey of Gauss-Christoffel Quadrature Formulae , 1981 .

[15]  P. Bonacich Power and Centrality: A Family of Measures , 1987, American Journal of Sociology.

[16]  K. Liu,et al.  Stochastic estimation with Z2 noise , 1993, hep-lat/9308015.

[17]  J. Shewchuk An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[18]  G. Golub,et al.  Some large-scale matrix computation problems , 1996 .

[19]  Michele Benzi,et al.  A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method , 1996, SIAM J. Sci. Comput..

[20]  G. Golub,et al.  Matrices, moments and quadrature II; How to compute the norm of the error in iterative methods , 1997 .

[21]  G. Golub,et al.  Bounds for the Entries of Matrix Functions with Applications to Preconditioning , 1999 .

[22]  C. Brezinski,et al.  Error Estimates for the Solution of Linear Systems , 1999, SIAM J. Sci. Comput..

[23]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[24]  A. Frommer,et al.  Numerical Challenges in Lattice Quantum Chromodynamics , 2000 .

[25]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[26]  J. Stoer,et al.  Introduction to Numerical Analysis , 2002 .

[27]  Gérard Meurant The computation of bounds for the norm of the error in the conjugate gradient algorithm , 2004, Numerical Algorithms.

[28]  Gérard Meurant,et al.  Numerical experiments in computing bounds for the norm of the error in the preconditioned conjugate gradient algorithm , 1999, Numerical Algorithms.

[29]  G. Meurant The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations , 2006 .

[30]  Y. Peres,et al.  Determinantal Processes and Independence , 2005, math/0503110.

[31]  G. Roberts,et al.  Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models , 2007, 0710.4228.

[32]  Gene H. Golub,et al.  Calculation of Gauss quadrature rules , 1967, Milestones in Matrix Computation.

[33]  Gene H. Golub,et al.  Some modified matrix eigenvalue problems , 1973, Milestones in Matrix Computation.

[34]  Jure Leskovec,et al.  Statistical properties of community structure in large social and information networks , 2008, WWW.

[35]  A. Wathen,et al.  APPROXIMATION OF THE SCATTERING AMPLITUDE AND LINEAR SYSTEMS , 2008 .

[36]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[37]  Constantine Bekas,et al.  Low cost high performance uncertainty quantification , 2009, WHPCF '09.

[38]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[39]  Mohamed-Ali Belabbas,et al.  Spectral methods in machine learning and new strategies for very large datasets , 2009, Proceedings of the National Academy of Sciences.

[40]  Christos Boutsidis,et al.  An improved approximation algorithm for the column subset selection problem , 2008, SODA.

[41]  G. Golub,et al.  Matrices, Moments and Quadrature with Applications , 2009 .

[42]  Yousef Saad,et al.  Rational approximation to the Fermi–Dirac function with applications in density functional theory , 2011, Numerical Algorithms.

[43]  Desmond J. Higham,et al.  Network Properties Revealed through Matrix Functions , 2010, SIAM Rev..

[44]  Antonio Iera,et al.  The Internet of Things: A survey , 2010, Comput. Networks.

[45]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[46]  E Weinan,et al.  A Fast Parallel Algorithm for Selected Inversion of Structured Sparse Matrices with Application to 2D Electronic Structure Calculations , 2010, SIAM J. Sci. Comput..

[47]  Carl Friedrich Gauss METHODUS NOVA INTEGRALIUM VALORES PER APPROXIMATIONEM INVENIENDI , 2011 .

[48]  Lexing Ying,et al.  SelInv---An Algorithm for Selected Inversion of a Sparse Symmetric Matrix , 2011, TOMS.

[49]  Ben Taskar,et al.  k-DPPs: Fixed-Size Determinantal Point Processes , 2011, ICML.

[50]  Ben Taskar,et al.  Near-Optimal MAP Inference for Determinantal Point Processes , 2012, NIPS.

[51]  C. Brezinski,et al.  ESTIMATIONS OF THE TRACE OF POWERS OF POSITIVE SELF-ADJOINT OPERATORS BY EXTRAPOLATION OF THE MOMENTS , 2012 .

[52]  Ben Taskar,et al.  Determinantal Point Processes for Machine Learning , 2012, Found. Trends Mach. Learn..

[53]  Joseph Naor,et al.  A Tight Linear Time (1/2)-Approximation for Unconstrained Submodular Maximization , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[54]  Ryan P. Adams,et al.  Priors for Diversity in Generative Latent Variable Models , 2012, NIPS.

[55]  Lothar Reichel,et al.  Network Analysis via Partial Spectral Factorization and Gauss Quadrature , 2013, SIAM J. Sci. Comput..

[56]  Byungkon Kang,et al.  Fast Determinantal Point Process Sampling with Application to Clustering , 2013, NIPS.

[57]  Michele Benzi,et al.  Total communicability as a centrality measure , 2013, J. Complex Networks.

[58]  E. George,et al.  Determinantal Priors for Variable Selection , 2014 .

[59]  Yu Cheng,et al.  Scalable Parallel Factorizations of SDD Matrices and Efficient Sampling for Gaussian Graphical Models , 2014, ArXiv.

[60]  Shang-Hua Teng,et al.  Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems , 2006, SIAM J. Matrix Anal. Appl..

[61]  Devavrat Shah,et al.  Solving Systems of Linear Equations: Locally and Asynchronously , 2014, ArXiv.

[62]  Jan Vondrák,et al.  Optimal approximation for submodular and supermodular optimization with bounded curvature , 2013, SODA.

[63]  Andreas Krause,et al.  Lazier Than Lazy Greedy , 2014, AAAI.

[64]  Nicolas Privault,et al.  Determinantal Point Processes , 2016 .

[65]  Nima Anari,et al.  Monte Carlo Markov Chain Algorithms for Sampling Strongly Rayleigh Distributions and Determinantal Point Processes , 2016, COLT.

[66]  Michael W. Mahoney,et al.  Revisiting the Nystrom Method for Improved Large-scale Machine Learning , 2013, J. Mach. Learn. Res..

[67]  Paraskevi Fika,et al.  Estimation of the bilinear form y⁎f(A)x for Hermitian matrices , 2016 .

[68]  J. Freericks Transport in multilayered nanostructures , 2016 .

[69]  Petra Kaufmann,et al.  Transport In Multilayered Nanostructures The Dynamical Mean Field Theory Approach , 2016 .

[70]  Christos Koukouvinos,et al.  Stochastic estimates for the trace of functions of matrices via Hadamard matrices , 2017, Commun. Stat. Simul. Comput..