Rating Network Paths for Locality-Aware Overlay Construction and Routing

This paper investigates the rating of network paths, i.e., acquiring quantized measures of path properties such as round-trip time and available bandwidth. Compared to fine-grained measurements, coarse-grained ratings are appealing in that they are not only informative but also cheap to obtain. Motivated by this insight, we first address the scalable acquisition of path ratings by statistical inference. By observing similarities to recommender systems, we examine the applicability of solutions to a recommender system and show that our inference problem can be solved by a class of matrix factorization techniques. A technical contribution is an active and progressive inference framework that not only improves the accuracy by selectively measuring more informative paths, but also speeds up the convergence for available bandwidth by incorporating its measurement methodology. Then, we investigate the usability of rating-based network measurement and inference in applications. A case study is performed on whether locality awareness can be achieved for overlay networks of Pastry and BitTorrent using inferred ratings. We show that such coarse-grained knowledge can improve the performance of peer selection and that finer granularities do not always lead to larger improvements.

[1]  Hui Zhang,et al.  Predicting Internet network distance with coordinates-based approaches , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[2]  Margo I. Seltzer,et al.  Network Coordinates in the Wild , 2007, NSDI.

[3]  Pierre Geurts,et al.  DMFSGD: A Decentralized Matrix Factorization Algorithm for Network Distance Prediction , 2012, IEEE/ACM Transactions on Networking.

[4]  Pierre Geurts,et al.  Decentralized prediction of end-to-end network performance classes , 2011, CoNEXT '11.

[5]  Michael Vitale,et al.  The Wisdom of Crowds , 2015, Cell.

[6]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[7]  Mark Crovella,et al.  On traffic matrix completion in the internet , 2012, IMC '12.

[8]  Walter Willinger,et al.  Laboratory-based calibration of available bandwidth estimation tools , 2007, Microprocess. Microsystems.

[9]  Robert Tappan Morris,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM '04.

[10]  Mark Crovella,et al.  Network Kriging , 2006, IEEE Journal on Selected Areas in Communications.

[11]  Domonkos Tikk,et al.  Scalable Collaborative Filtering Approaches for Large Recommender Systems , 2009, J. Mach. Learn. Res..

[12]  Manish Jain,et al.  End-to-end available bandwidth: measurement methodology, dynamics, and relation with TCP throughput , 2002, SIGCOMM 2002.

[13]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[14]  Ian Clarke,et al.  Freenet: A Distributed Anonymous Information Storage and Retrieval System , 2000, Workshop on Design Issues in Anonymity and Unobservability.

[15]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[16]  Walter Willinger,et al.  Spatio-temporal compressive sensing and internet traffic matrices , 2009, SIGCOMM '09.

[17]  Fabián E. Bustamante,et al.  Taming the torrent: a practical approach to reducing cross-isp traffic in peer-to-peer systems , 2008, SIGCOMM '08.

[18]  Richard G. Baraniuk,et al.  pathChirp: Efficient available bandwidth estimation for network paths , 2003 .

[19]  Gavin Brown,et al.  Ensemble Learning , 2010, Encyclopedia of Machine Learning and Data Mining.

[20]  Jonathan M. Smith,et al.  IDES: An Internet Distance Estimation Service for Large Networks , 2006, IEEE Journal on Selected Areas in Communications.

[21]  Walter Willinger,et al.  Spatio-Temporal Compressive Sensing and Internet Traffic Matrices (Extended Version) , 2012, IEEE/ACM Transactions on Networking.

[22]  Yin Zhang,et al.  NetQuest: A Flexible Framework for Large-Scale Network Measurement , 2009, IEEE/ACM Transactions on Networking.

[23]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[24]  Mark Crovella,et al.  Inferring visibility: who's (not) talking to whom? , 2012, SIGCOMM '12.

[25]  Kimberly C. Claffy,et al.  Comparison of Public End-to-End Bandwidth Estimation Tools on High-Speed Links , 2005, PAM.

[26]  Pierre Geurts,et al.  Network Distance Prediction Based on Decentralized Matrix Factorization , 2010, Networking.

[27]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[28]  Randy H. Katz,et al.  An algebraic approach to practical and scalable overlay network monitoring , 2004, SIGCOMM '04.

[29]  Balachander Krishnamurthy,et al.  Internet Measurement - Infrastructure, Traffic, and Applications , 2006 .

[30]  Emin Gün Sirer,et al.  Meridian: a lightweight network location service without virtual coordinates , 2005, SIGCOMM '05.

[31]  Jon Crowcroft,et al.  A survey and comparison of peer-to-peer overlay network schemes , 2005, IEEE Communications Surveys & Tutorials.

[32]  Manish Jain,et al.  End-to-end available bandwidth: measurement methodology, dynamics, and relation with TCP throughput , 2003, TNET.

[33]  Vijay K. Gurbani,et al.  Improving Peer Selection in Peer-to-peer Applications: Myths vs. Reality , 2012, RFC.

[34]  Manish Jain,et al.  Ten fallacies and pitfalls on end-to-end available bandwidth estimation , 2004, IMC '04.

[35]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[36]  Zhi-Li Zhang,et al.  Vivisecting YouTube: An active measurement study , 2012, 2012 Proceedings IEEE INFOCOM.

[37]  M. Wu,et al.  Collaborative Filtering via Ensembles of Matrix Factorizations , 2007, KDD 2007.

[38]  Mark Crovella,et al.  Virtual landmarks for the internet , 2003, IMC '03.

[39]  Abraham Silberschatz,et al.  P4p: provider portal for applications , 2008, SIGCOMM '08.

[40]  Zhao Wen-tao,et al.  Efficient available bandwidth estimation for network paths , 2008 .

[41]  James Bennett,et al.  The Netflix Prize , 2007 .

[42]  Sujata Banerjee,et al.  S3: a scalable sensing service for monitoring large networked systems , 2006, INM '06.

[43]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[44]  黄瑞波 Google TV:电视革命 , 2010 .

[45]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[46]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[47]  Nikunj C. Oza,et al.  Online Ensemble Learning , 2000, AAAI/IAAI.