A learning framework using Green's function and kernel regularization with application to recommender system

Green's function for the Laplace operator represents the propagation of influence of point sources and is the foundation for solving many physics problems. On a graph of pairwise similarities, the Green's function is the inverse of the combinatorial Laplacian; we resolve the zero-mode difficulty by showing its physical origin as the consequence of the Von Neumann boundary condition. We propose to use Green's function to propagate label information for both semi-supervised and unsupervised learning. We also derive this learning framework from the kernel regularization using Reproducing Kernel Hilbert Space theory at strong regularization limit. Green's function provides a well-defined distance metric on a generic weighted graph, either as the effective distance on the network of electric resistors, or the average commute time in random walks. We show that for unsupervised learning this approach is identical to Ratio Cut and Normalized Cut spectral clustering algorithms. Experiments on newsgroups and six UCI datasets illustrate the effectiveness of this approach. Finally, we propose a novel item-based recommender system using Green's function and show its effectiveness.

[1]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[2]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[3]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[4]  John Riedl,et al.  Recommender systems in e-commerce , 1999, EC '99.

[5]  Peter G. Doyle,et al.  Random walks and electric networks , 1987, math/0001057.

[6]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[7]  Bernhard Schölkopf,et al.  Cluster Kernels for Semi-Supervised Learning , 2002, NIPS.

[8]  Golub Gene H. Et.Al Matrix Computations, 3rd Edition , 2007 .

[9]  Bradley N. Miller,et al.  MovieLens unplugged: experiences with an occasionally connected recommender system , 2003, IUI '03.

[10]  Luo Si,et al.  An automatic weighting scheme for collaborative filtering , 2004, SIGIR '04.

[11]  Neil D. Lawrence,et al.  Semi-supervised Learning via Gaussian Processes , 2004, NIPS.

[12]  Kenneth Y. Goldberg,et al.  Eigentaste: A Constant Time Collaborative Filtering Algorithm , 2001, Information Retrieval.

[13]  M. Randic,et al.  Resistance distance , 1993 .

[14]  Thorsten Joachims,et al.  Transductive Learning via Spectral Graph Partitioning , 2003, ICML.

[15]  François Fouss,et al.  The Principal Components Analysis of a Graph, and Its Relationships to Spectral Clustering , 2004, ECML.

[16]  Chris H. Q. Ding,et al.  Unsupervised Learning: Self-aggregation in Scaled Principal Component Space , 2002, PKDD.

[17]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[18]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[19]  John D. Lafferty,et al.  Semi-supervised learning using randomized mincuts , 2004, ICML.

[20]  Bernhard Schölkopf,et al.  Introduction to Semi-Supervised Learning , 2006, Semi-Supervised Learning.

[21]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[22]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[23]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[24]  C. Ding,et al.  Spectral relaxation models and structure analysis for K-way graph clustering and bi-clustering , 2001 .

[25]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[26]  Yoshua Bengio,et al.  Semi-supervised Learning by Entropy Minimization , 2004, CAP.

[27]  Fei Wang,et al.  Recommendation on Item Graphs , 2006, Sixth International Conference on Data Mining (ICDM'06).

[28]  Prabhakar Raghavan,et al.  The electrical resistance of a graph captures its commute and cover times , 1989, STOC '89.

[29]  Jun Wang,et al.  Unifying user-based and item-based collaborative filtering approaches by similarity fusion , 2006, SIGIR.

[30]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[31]  F. Göbel,et al.  Random walks on graphs , 1974 .

[32]  George Karypis,et al.  Item-based top-N recommendation algorithms , 2004, TOIS.

[33]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[34]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Tommi S. Jaakkola,et al.  Partially labeled classification with Markov random walks , 2001, NIPS.

[36]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[37]  John Riedl,et al.  Application of Dimensionality Reduction in Recommender Systems , 2000 .

[38]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[39]  John Riedl,et al.  An algorithmic framework for performing collaborative filtering , 1999, SIGIR '99.

[40]  Michael J. Pazzani,et al.  A personal news agent that talks, learns and explains , 1999, AGENTS '99.

[41]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[42]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[43]  Prabhakar Raghavan,et al.  The electrical resistance of a graph captures its commute and cover times , 2005, computational complexity.

[44]  Xiaojin Zhu,et al.  Semi-Supervised Learning Literature Survey , 2005 .