Network anomaly detection through nonlinear analysis

Nowadays every network is susceptible on a daily basis to a significant number of different threats and attacks both from the inside and outside world. Some attacks only exploit system vulnerabilities and their traffic pattern is undistinguishable from normal behavior, but in many cases the attack mechanisms combine protocol or OS tampering activity with a specific traffic pattern having its own particular characteristics. Since these traffic anomalies are now conceived as a structural part of the overall network traffic, it is more and more important to automatically detect, classify and identify them in order to react promptly and adequately. In this work we present a novel approach to network-based anomaly detection based on the analysis of non-stationary properties and ''hidden'' recurrence patterns occurring in the aggregated IP traffic flows. In the observation of the above transition patterns for detecting anomalous behaviors, we adopted recurrence quantification analysis, a nonlinear technique widely used in many science fields to explore the hidden dynamics and time correlations of statistical time series. Our model demonstrated to be effective for providing a deterministic interpretation of recurrence patterns originated by the complex traffic dynamics observable during the occurrence of ''noisy'' network anomaly phenomena (characterized by measurable variations in the statistical properties of the traffic time series), and hence for developing qualitative and quantitative observations that can be reliably used in detecting such events.

[1]  Vasilios A. Siris,et al.  Application of anomaly detection algorithms for detecting SYN flooding attacks , 2004, GLOBECOM.

[2]  Leonard A. Smith,et al.  Distinguishing between low-dimensional dynamics and randomness in measured time series , 1992 .

[3]  H. Abarbanel,et al.  Determining embedding dimension for phase-space reconstruction using a geometrical construction. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[4]  S. Washington,et al.  Statistical and Econometric Methods for Transportation Data Analysis , 2010 .

[5]  C L Webber,et al.  Dynamical assessment of physiological systems and states using recurrence plot strategies. , 1994, Journal of applied physiology.

[6]  John Mhugh The 1998 Lincoln Laboratory IDS evaluation : A critique , 2000 .

[7]  A. N. Sharkovskiĭ Dynamic systems and turbulence , 1989 .

[8]  M. B. Priestley,et al.  Non-linear and non-stationary time series analysis , 1990 .

[9]  Hsuan-Tien Lin A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .

[10]  A. Giuliani,et al.  Recurrence Quantification Analysis and Principal Components in the Detection of Short Complex Signals , 1997, chao-dyn/9712017.

[11]  H. Kantz A robust method to estimate the maximal Lyapunov exponent of a time series , 1994 .

[12]  Philip K. Chan,et al.  Learning nonstationary models of normal network traffic for detecting novel attacks , 2002, KDD.

[13]  Christopher Leckie,et al.  Adaptive Clustering for Network Intrusion Detection , 2004, PAKDD.

[14]  John McHugh,et al.  The 1998 Lincoln Laboratory IDS Evaluation , 2000, Recent Advances in Intrusion Detection.

[15]  Christopher Leckie,et al.  Unsupervised Anomaly Detection in Network Intrusion Detection Using Clusters , 2005, ACSC.

[16]  Seung-Won Shin,et al.  D-SAT: detecting SYN flooding attack by two-stage statistical approach , 2005, The 2005 Symposium on Applications and the Internet.

[17]  Norio Shiratori,et al.  Self-similar and fractal nature of internet traffic , 2004 .

[18]  Fraser,et al.  Independent coordinates for strange attractors from mutual information. , 1986, Physical review. A, General physics.

[19]  H. T. Kung,et al.  Use of spectral analysis in defense against DoS attacks , 2002, Global Telecommunications Conference, 2002. GLOBECOM '02. IEEE.

[20]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[21]  David Ruelle,et al.  Deterministic chaos: the science and the fiction , 1995 .

[22]  Carlo Noe,et al.  APPLICATION OF NON-LINEAR TIME SERIES ANALYSIS TECHNIQUES TO THE NORDIC SPOT ELECTRICITY MARKET DATA , 2007 .

[23]  Jake D. Brutlag,et al.  Aberrant Behavior Detection in Time Series for Network Monitoring , 2000, LISA.

[24]  Richard Lippmann,et al.  Analysis and Results of the 1999 DARPA Off-Line Intrusion Detection Evaluation , 2000, Recent Advances in Intrusion Detection.

[25]  Misako Takayasu,et al.  Phase transition in a computer network model , 1998 .

[26]  Donald F. Towsley,et al.  Detecting anomalies in network traffic using maximum entropy estimation , 2005, IMC '05.

[27]  Gabriel Maciá-Fernández,et al.  Anomaly-based network intrusion detection: Techniques, systems and challenges , 2009, Comput. Secur..

[28]  Jürgen Kurths,et al.  Recurrence plots for the analysis of complex systems , 2009 .

[29]  J. Zbilut,et al.  Embeddings and delays as derived from quantification of recurrence plots , 1992 .

[30]  D. T. Kaplan,et al.  Direct test for determinism in a time series. , 1992, Physical review letters.

[31]  Salvatore J. Stolfo,et al.  A Geometric Framework for Unsupervised Anomaly Detection , 2002, Applications of Data Mining in Computer Security.

[32]  Philip K. Chan,et al.  Learning Models of Network Traffic for Detecting Novel Attacks , 2002 .

[33]  P. Grassberger,et al.  Characterization of Strange Attractors , 1983 .

[34]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[35]  James P. Crutchfield,et al.  Geometry from a Time Series , 1980 .

[36]  D. Ruelle,et al.  The Claude Bernard Lecture, 1989 - Deterministic chaos: the science and the fiction , 1990, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[37]  Sushil Jajodia,et al.  Detecting Novel Network Intrusions Using Bayes Estimators , 2001, SDM.

[38]  Alberto Dainotti,et al.  Wavelet-based Detection of DoS Attacks. , 2006 .

[39]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[40]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[41]  Matthew V. Mahoney,et al.  Network traffic anomaly detection based on packet bytes , 2003, SAC '03.

[42]  Walter Willinger,et al.  Is Network Traffic Self-Similar or Multifractal? , 1997 .

[43]  kc claffy,et al.  The architecture of CoralReef: an Internet traffic monitoring software suite , 2001 .

[44]  Gitae Kim,et al.  NOMAD: traffic-based network monitoring framework for anomaly detection , 1999, Proceedings IEEE International Symposium on Computers and Communications (Cat. No.PR00250).

[45]  Richard Lippmann,et al.  The 1999 DARPA off-line intrusion detection evaluation , 2000, Comput. Networks.

[46]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[47]  Eleazar Eskin,et al.  A GEOMETRIC FRAMEWORK FOR UNSUPERVISED ANOMALY DETECTION: DETECTING INTRUSIONS IN UNLABELED DATA , 2002 .

[48]  N. Marwan,et al.  Nonlinear analysis of bivariate data with cross recurrence plots , 2002, physics/0201061.

[49]  H. Takayasu,et al.  Dynamic phase transition observed in the Internet traffic flow , 2000 .

[50]  J. D. Farmer,et al.  State space reconstruction in the presence of noise" Physica D , 1991 .

[51]  D. Ruelle,et al.  Ergodic theory of chaos and strange attractors , 1985 .

[52]  Chia-Chen Yen,et al.  Unsupervised Anomaly Detection Using HDG-Clustering Algorithm , 2007, ICONIP.

[53]  Holger Kantz,et al.  Practical implementation of nonlinear time series methods: The TISEAN package. , 1998, Chaos.

[54]  Philip K. Chan,et al.  PHAD: packet header anomaly detection for identifying hostile network traffic , 2001 .

[55]  Masao Masugi,et al.  Multi-fractal analysis of IP-network traffic for assessing time variations in scaling properties , 2007 .

[56]  Alexander G. Tartakovsky,et al.  A novel approach to detection of \denial{of{service" attacks via adaptive sequential and batch{sequential change{point detection methods , 2001 .

[57]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[58]  Robert H. Shumway,et al.  Time Series Analysis and Its Applications (Springer Texts in Statistics) , 2005 .