Elementary Symmetric Polynomials for Optimal Experimental Design

We revisit the classical problem of optimal experimental design (OED) under a new mathematical model grounded in a geometric motivation. Specifically, we introduce models based on elementary symmetric polynomials; these polynomials capture "partial volumes" and offer a graded interpolation between the widely used A-optimal design and D-optimal design models, obtaining each of them as special cases. We analyze properties of our models, and derive both greedy and convex-relaxation algorithms for computing the associated designs. Our analysis establishes approximation guarantees on these algorithms, while our empirical results substantiate our claims and demonstrate a curious phenomenon concerning our greedy method. Finally, as a byproduct, we obtain new results on the theory of elementary symmetric polynomials that may be of independent interest.

[1]  Guillaume Sagnol,et al.  Optimal design of experiments with application to the inference of traffic matrices in large networks: second order cone programming and submodularity , 2010 .

[2]  Yong Chen,et al.  Robust principal component analysis and outlier detection with ecological data , 2004 .

[3]  Stephen P. Boyd,et al.  Determinant Maximization with Linear Matrix Inequality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[4]  A. S. Lewis,et al.  Derivatives of Spectral Functions , 1996, Math. Oper. Res..

[5]  R. Bhatia Positive Definite Matrices , 2007 .

[6]  John Shawe-Taylor,et al.  The Minimum Volume Covering Ellipsoid Estimation in Kernel-Defined Feature Spaces , 2006, ECML.

[7]  Holger Dette,et al.  Locally D-optimal Designs for Exponential Regression , 2004 .

[8]  P. Rousseeuw,et al.  Minimum volume ellipsoid , 2009 .

[9]  Peng Sun,et al.  Computation of Minimum Volume Covering Ellipsoids , 2002, Oper. Res..

[10]  David A. Cohn,et al.  Neural Network Exploration Using Optimal Experiment Design , 1993, NIPS.

[11]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[12]  Christos Boutsidis,et al.  Faster Subset Selection for Matrices and Applications , 2011, SIAM J. Matrix Anal. Appl..

[13]  Lyle H. Ungar,et al.  A -Optimality for Active Learning of Logistic Regression Classifiers , 2004 .

[14]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[15]  I. G. MacDonald,et al.  Symmetric functions and Hall polynomials , 1979 .

[16]  Michael Jackson,et al.  Optimal Design of Experiments , 1994 .

[17]  Richard Jozsa,et al.  Symmetric polynomials in information theory: entropy and subentropy , 2014, ArXiv.

[18]  E. Barnes An algorithm for separating patterns by ellipsoids , 1982 .

[19]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[20]  Zhong Jin,et al.  Neighborhood preserving D-optimal design for active learning and its application to terrain classification , 2012, Neural Computing and Applications.

[21]  Nikolaos M. Manousakis,et al.  Fisher Information-Based Meter Placement in Distribution Grids via the D-Optimal Experimental Design , 2018, IEEE Transactions on Smart Grid.

[22]  Alan J. Miller,et al.  A Fedorov Exchange Algorithm for D-optimal Design , 1994 .

[23]  J. Kiefer Optimal design: Variation in structure and performance under change of criterion , 1975 .

[24]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[25]  Suvrit Sra,et al.  Conic Geometric Optimization on the Manifold of Positive Definite Matrices , 2013, SIAM J. Optim..

[26]  I-Cheng Yeh,et al.  Modeling of strength of high-performance concrete using artificial neural networks , 1998 .

[27]  Tanvi Jain Derivatives for antisymmetric tensor powers and perturbation bounds , 2011 .

[28]  Quan Pan,et al.  Approximation for maximizing monotone non-decreasing set functions with a greedy method , 2016, J. Comb. Optim..

[29]  P. J. García Nieto,et al.  The use of design of experiments to improve a neural network model in order to predict the thickness of the chromium layer in a hard chromium plating process , 2010, Math. Comput. Model..

[30]  S. Silvey,et al.  An algorithm for optimal designs on a design space , 1978 .

[31]  Avi Wigderson,et al.  Depth-3 arithmetic circuits over fields of characteristic zero , 2002, computational complexity.

[32]  Stratis Ioannidis,et al.  Budget Feasible Mechanisms for Experimental Design , 2013, LATIN.

[33]  W. W. Muir,et al.  Inequalities concerning the inverses of positive definite matrices , 1974, Proceedings of the Edinburgh Mathematical Society.

[34]  G. Elfving Optimum Allocation in Linear Regression Theory , 1952 .

[35]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[36]  Timothy A. Davis,et al.  An Efficient Hybrid Algorithm for the Separable Convex Quadratic Knapsack Problem , 2016, ACM Trans. Math. Softw..

[37]  M. Ghosh,et al.  Design Issues for Generalized Linear Models: A Review , 2006, math/0701088.

[38]  My T. Thai,et al.  Breaking the Bonds of Submodularity: Empirical Estimation of Approximation Ratios for Monotone Non-Submodular Greedy Maximization , 2017, ArXiv.

[39]  Anthony C. Atkinson,et al.  Optimum Experimental Designs, with SAS , 2007 .

[40]  Yaming Yu Monotonic convergence of a general algorithm for computing optimal designs , 2009, 0905.2646.

[41]  Deanna Needell,et al.  Constrained Adaptive Sensing , 2015, IEEE Transactions on Signal Processing.

[42]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[43]  Adams Wei Yu,et al.  On Computationally Tractable Selection of Experiments in Regression Models , 2016 .

[44]  Xiaofei He,et al.  Laplacian Regularized D-Optimal Design for Active Learning and Its Application to Image Retrieval , 2010, IEEE Transactions on Image Processing.

[45]  Heinz H. Bauschke,et al.  Hyperbolic Polynomials and Convex Analysis , 2001, Canadian Journal of Mathematics.

[46]  Roberto Cominetti,et al.  A Newton’s method for the continuous quadratic knapsack problem , 2014, Math. Program. Comput..