On-line attribute abnormal point detecting method for supporting dynamic update

Abstract

The invention discloses an on-line attribute abnormal point detecting method for supporting dynamic update. The method comprises the following steps of: providing brand-new attribute abnormal point definition by analyzing practical application and user requirement, detecting an abnormal point on the premise of taking an attribute correlation between data points in a data set into consideration, providing more effective abnormal information relative to the traditional definition, and supporting on-line attribute abnormal point detection for dynamically updated stream data by combining practical stream data system application and using a sliding window and an on-line clustering method so as to provide real-time detection result feedback for a user. Aiming at the practical system overload condition in the stream data system application, a method for effectively reducing the load is provided so that the detecting method still can feed back the detection result in real time under the mass stream data update condition, the result error is in a user controlled range and effective balance of the detecting method between the running efficiency and the result precision is achieved.

Description

technical field [0001] The invention relates to data mining, outlier point detection and stream data system management technology, in particular to an online attribute outlier point detection method supporting dynamic update. Background technique [0002] Outlier detection is one of the most important application technical methods in the field of data mining. Different from other common data mining methods, outlier detection is to find relatively isolated and outlier abnormal points and abnormal patterns in massive data. Most of the early data mining systems only regard the abnormal points in the data as noise, and the detection of abnormal points is mainly used to clean the noise. However, with the generation of massive data, abnormal data can often reflect greater value, so the application of abnormal point detection in reality is becoming more and more extensive, such as network intrusion detection, bank credit fraud, etc. [0003] One of the first prerequisites for outl...

AI Technical Summary

Problems solved by technology

Since streaming data systems often need to detect a large amount of dynamic data in real time, and in most cases the actual deployment machine cannot provide corresponding processing and storage capabilities, it is necessary to provide an efficient detection method

[0005] Outlier detection in streaming data systems is a new research field and has important practical value, but there is still a lack of a mature technical solution. Therefore, it is necessary to provide an online attribute outlier detection method that can support dynamic updates. Under the premise of accuracy, it can process massive dynamic data in streaming data systems through approximate methods

Images