A Data Stream Anomaly Detection Method Based on Local Vector Dot Product Density

Abstract

The invention discloses a data flow anomaly detection method based on local vector dot product density, which is characterized in that it comprises the following steps: 1) processing the real-time data flow; 2) setting the data set S in the sliding window W m And initialize the parameters n, ε, λ; 3) Obtain the mean value of vector dot product MVP; 4) Determine the data set S in the current sliding window W m 5) Obtain the local vector dot product density LDVP of each data point; 6) Determine the candidate outliers in the current sliding window; 7) Determine the outliers through multiple verification. This method can accurately and effectively detect hidden abnormal points in the current real-time, fast and changeable complex data flow environment on high-dimensional space and data sets with uneven distribution of abnormalities. This method does not require clustering. Under this condition, the anomaly detection of the data set can be efficiently completed, and there are few artificially preset parameters, and it has higher robustness and stronger adaptability in the case of different anomaly proportions and different dimensions.

Description

technical field [0001] The present invention relates to data flow anomaly detection, in particular to a data flow anomaly detection method based on local vector dot product density. Background technique [0002] The rapid development of network technology and the continuous improvement of social informatization have triggered an explosive growth in the amount of information, causing various industries to generate massive, high-speed, dynamic streaming data, such as network intrusion monitoring, business transaction management and analysis, video surveillance , sensor network monitoring, etc. Due to the real-time and unlimited characteristics of dynamic data streams, traditional static data anomaly detection methods can no longer accurately and effectively analyze and process such large-scale and dynamically growing stream data. Therefore, building a real-time and effective anomaly detection method suitable for data streams becomes especially important. [0003] Existing da...

AI Technical Summary

Problems solved by technology

[0004] The above traditional algorithms have problems such as high complexity, many artificially preset parameters, and low effectiveness in a multi-dimensional data environment.

Images