Kronecker Determinantal Point Processes

Determinantal Point Processes (DPPs) are probabilistic models over all subsets a ground set of $N$ items. They have recently gained prominence in several applications that rely on "diverse" subsets. However, their applicability to large problems is still limited due to the $\mathcal O(N^3)$ complexity of core tasks such as sampling and learning. We enable efficient sampling and learning for DPPs by introducing KronDPP, a DPP model whose kernel matrix decomposes as a tensor product of multiple smaller kernel matrices. This decomposition immediately enables fast exact sampling. But contrary to what one may expect, leveraging the Kronecker product structure for speeding up DPP learning turns out to be more difficult. We overcome this challenge, and derive batch and stochastic optimization algorithms for efficiently learning the parameters of a KronDPP.

[1]  Suvrit Sra,et al.  Fixed-point algorithms for learning determinantal point processes , 2015, ICML.

[2]  C. Loan,et al.  Approximation with Kronecker Products , 1992 .

[3]  Alexander J. Smola,et al.  Fast Kronecker Inference in Gaussian Processes with non-Gaussian Likelihoods , 2015, ICML.

[4]  Shih-Fu Chang,et al.  Fast Orthogonal Projection Based on Kronecker Product , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Ashwin Arulselvan,et al.  A note on the set union knapsack problem , 2014, Discret. Appl. Math..

[6]  Y. Peres,et al.  Determinantal Processes and Independence , 2005, math/0503110.

[7]  Byungkon Kang,et al.  Fast Determinantal Point Process Sampling with Application to Clustering , 2013, NIPS.

[8]  Edward Y. Chang,et al.  Kronecker Factorization for Speeding up Kernel Machines , 2005, SDM.

[9]  J. Møller,et al.  Determinantal point process models and statistical inference , 2012, 1205.4818.

[10]  Ben Taskar,et al.  Learning the Parameters of Determinantal Point Process Kernels , 2014, ICML.

[11]  Roger B. Grosse,et al.  Optimizing Neural Networks with Kronecker-factored Approximate Curvature , 2015, ICML.

[12]  R. Bhatia Positive Definite Matrices , 2007 .

[13]  Alan L. Yuille,et al.  The Concave-Convex Procedure (CCCP) , 2001, NIPS.

[14]  Ulrich Paquet,et al.  Low-Rank Factorization of Determinantal Point Processes , 2016, AAAI.

[15]  Alex Kulesza,et al.  Diversifying Sparsity Using Variational Determinantal Point Processes , 2014, ArXiv.

[16]  Hui Lin,et al.  Learning Mixtures of Submodular Shells with Application to Document Summarization , 2012, UAI.

[17]  Suvrit Sra,et al.  Efficient Sampling for k-Determinantal Point Processes , 2015, AISTATS.

[18]  Ben Taskar,et al.  Expectation-Maximization for Learning Determinantal Point Processes , 2014, NIPS.

[19]  Ben Taskar,et al.  Determinantal Point Processes for Machine Learning , 2012, Found. Trends Mach. Learn..

[20]  Nicolas Privault,et al.  Determinantal Point Processes , 2016 .

[21]  O. Macchi The coincidence approach to stochastic point processes , 1975, Advances in Applied Probability.

[22]  R. Lyons Determinantal probability measures , 2002, math/0204325.

[23]  Ben Taskar,et al.  k-DPPs: Fixed-Size Determinantal Point Processes , 2011, ICML.

[24]  Kristen Grauman,et al.  Large-Margin Determinantal Point Processes , 2014, UAI.

[25]  Ben Taskar,et al.  Learning Determinantal Point Processes , 2011, UAI.

[26]  Yi-Cheng Zhang,et al.  Solving the apparent diversity-accuracy dilemma of recommender systems , 2008, Proceedings of the National Academy of Sciences.

[27]  Suvrit Sra,et al.  Diversity Networks , 2015, ICLR.

[28]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[29]  Suvrit Sra,et al.  Fast DPP Sampling for Nystrom with Application to Kernel Methods , 2016, ICML.