Exception detection method based on data flow concept drift

Abstract

The invention provides an exception detection method based on data flow concept drift, belongs to the field of data mining and exception detection, and aims to provide an exception detection method based on data flow concept drift which can detect concept drift in time. The method comprises the steps of S1, obtaining the real data collected by a to-be-detected system at different moments to form areal data stream, and establishing a current prediction model of the to-be-detected system according to the real data stream; S2, predicting the data of the next time period through a prediction model to obtain a prediction data stream; S3, calculating a similarity data set between the real data flow and the prediction data flow; S4, judging whether concept drift occurs or not according to the similarity data set and a current concept drift threshold value of the to-be-detected system; S5, if not, repeating the steps S2 to S4; S6, if yes, updating the prediction model, the concept drift threshold value and the exception detection threshold value, and repeating the steps S2 to S6 according to the updated prediction model and the concept drift threshold value.

Description

technical field [0001] The invention relates to the technical field of data mining and anomaly detection, in particular to an anomaly detection method based on data flow concept drift. Background technique [0002] In systems that collect data by time, such as actual production, there are often data that do not conform to normal data or existing data change patterns, and these data are so-called abnormal data. There are many methods for anomaly data detection, but they rarely consider the phenomenon of concept drift in data flow. Concept drift means that the concept contained in the data flow has changed, for example, with the improvement of the process flow, the aging of the machine, the update of the equipment and the emergence of unknown working conditions, the target concept of the data flow has changed, making the old data and The concepts contained in the new data are no longer consistent. Once the concept drifts in the data stream, the diagnostic performance of the ...

AI Technical Summary

Problems solved by technology

Once the concept drifts in the data stream, the diagnostic performance of the original anomaly diagnosis model generated based on the old concept will decline under the new data, which is prone to misjudgment of abnormal data and affects the normal use of users. Therefore, it is necessary to check whether the concept has drifted. Make timely detections to adjust the anomaly detection model under the new concept

Images