Adaptive network flow concept drift detection method based on information entropy

Abstract

The invention discloses an adaptive network flow concept drift detection method based on information entropy, and relates to the technical field of data processing. The concept drift problem caused byflow characteristic changes due to time lapse and different network environments is solved. By using sliding window techniques, aiming at new and old data streams, discretizing the flow characteristic attributes into a plurality of branches; comparing the plurality of flow characteristics together; comparing differences of new and old data stream sliding windows by counting information entropiesof each characteristic attribute and each branch; and when specific detection is carried out, the method comprises the following steps: firstly, solving a threshold calculation formula of the size ofa sliding window by utilizing a Hoeffding boundary theory; then, determining the size of a sliding window by utilizing a Tie-buffering method; according to the method, firstly, obtaining a new data flow sliding window and an old data flow sliding window, then calculating entropy values of the new data flow sliding window and the old data flow sliding window through information entropy, finally, jusging whether concept drifting happens to data or not according to threshold values and information entropy difference values of the new window and the old window, thus concept drifting can be effectively detected, classifiers are updated, and good classification performance and generalization capacity are shown.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to an adaptive network flow concept drift detection method based on information entropy. Background technique [0002] With the rapid development of the mobile Internet, new services in web browsing, streaming media, and social networks are constantly emerging. At the same time, the user network security requirements make the proportion of encrypted traffic continue to increase, making traditional traffic classification methods face severe challenges. However, the machine learning classification method based on flow characteristics will cause concept drift due to differences in the distribution of services carried by traffic in different time periods and different regions. [0003] Concept drift refers to the phenomenon in predictive analytics and machine learning where the statistical properties of the target variable change in unforeseen ways over time. Over time, the pre...

AI Technical Summary

Problems solved by technology

[0006] The present invention is based on the problem that the concept drift detection method based on the existing classification accuracy cannot be well applied in network traffic detection due to category imbalance, and the type attribute labels required for evaluating the classification error rate in the traffic identification process need to be manually marked , and difficult to obtain, providing an adaptive network flow concept drift detection method based on information entropy

Images