Distributed Detection of Zero-Day Network Traffic Flows

Zero-day (or unknown) traffic brings about challenges for network security and management tasks, in terms of identifying the occurrence of those events in the network in an accurate and timely manner. In this paper, we propose a distributed mechanism to detect such unknown traffic in a timely manner. We compare our distributed scheme with a centralized system, where all the network flow data are used as a whole to perform the detection. We combined supervised and unsupervised learning mechanisms to discover and classify the unknown traffic efficiently, using clustering and Random Forest (RF) based schemes for this purpose. Further, we incorporated the correlation information in the traffic flows to improve the accuracy of detection, by means of using a Bag of Flows (BoFs) based method. Evaluation on real traces reveal that our distributed approach achieves a comparable detection performance to that of a centralized scheme. Further, the distributed scheme that incorporates unknown sample sharing in the framework shows improvement in the zero-day traffic detection performance. Moreover, the classifier used with the combination of BoF and RF shows improved detection accuracy, compared with not using BoFs.

[1]  Jun Zhang,et al.  An Effective Network Traffic Classification Method with Unknown Flow Detection , 2013, IEEE Transactions on Network and Service Management.

[2]  Jie Wu,et al.  Robust Network Traffic Classification , 2015, IEEE/ACM Transactions on Networking.

[3]  Tuomo Sipola,et al.  Adaptive framework for network traffic classification using dimensionality reduction and clustering , 2012, 2012 IV International Congress on Ultra Modern Telecommunications and Control Systems.

[4]  Grenville J. Armitage,et al.  A survey of techniques for internet traffic classification using machine learning , 2008, IEEE Communications Surveys & Tutorials.

[5]  Michalis Faloutsos,et al.  Internet traffic classification demystified: myths, caveats, and the best practices , 2008, CoNEXT '08.

[6]  Dario Rossi,et al.  KISS: Stochastic Packet Inspection Classifier for UDP Traffic , 2010, IEEE/ACM Transactions on Networking.

[7]  Zhen Ling,et al.  TorWard: Discovery of malicious traffic over Tor , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[8]  Leyla Bilge,et al.  Cutting the Gordian Knot: A Look Under the Hood of Ransomware Attacks , 2015, DIMVA.

[9]  Antonio Criminisi,et al.  Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning , 2012, Found. Trends Comput. Graph. Vis..

[10]  Christopher Leckie,et al.  Using Virtual Machine Allocation Policies to Defend against Co-Resident Attacks in Cloud Computing , 2017, IEEE Transactions on Dependable and Secure Computing.

[11]  Xiao Chen,et al.  Zero-Day Traffic Identification , 2013, CSS.

[12]  Marimuthu Palaniswami,et al.  Hyperspherical cluster based distributed anomaly detection in wireless sensor networks , 2014, J. Parallel Distributed Comput..

[13]  Jun Zhang,et al.  Comprehensive analysis of network traffic data , 2018, Concurr. Comput. Pract. Exp..

[14]  H. B. Barlow,et al.  Unsupervised Learning , 1989, Neural Computation.

[15]  Nino Vincenzo Verde,et al.  Analyzing Android Encrypted Network Traffic to Identify User Actions , 2016, IEEE Transactions on Information Forensics and Security.

[16]  Luca Salgarelli,et al.  Support Vector Machines for TCP traffic classification , 2009, Comput. Networks.

[17]  J. Erman,et al.  QRP05-4: Internet Traffic Identification using Machine Learning , 2006, IEEE Globecom 2006.

[18]  Paul A. Watters,et al.  Zero-day Malware Detection based on Supervised Learning Algorithms of API call Signatures , 2011, AusDM.

[19]  Jun Zhang,et al.  Network Traffic Classification Using Correlation Information , 2013, IEEE Transactions on Parallel and Distributed Systems.

[20]  Yang Xiang,et al.  An automatic application signature construction system for unknown traffic , 2010 .

[21]  Marco Mellia,et al.  Mining Unclassified Traffic Using Automatic Clustering Techniques , 2011, TMA.