Abnormal data point identification method in time sequence data on the basis of global information

Abstract

The invention discloses an abnormal data point identification method in time sequence data on the basis of global information. The method comprises the following steps that: obtaining original time sequence data and the observation value of each data point; calculating the observation value change speed of each data point and the speed change rate of each data point; according to the speed changerate of each data point, calculating the average speed change rate of the original time sequence data, carrying out statistics on the discrete probability distribution of the speed change rate of eachdata point, and carrying out fitting on the discrete probability distribution to obtain a probability density function; and according to the value change speed constraint of the time sequence data and the speed change rate of the data point, detecting an abnormal data point. The average speed change rate of the time sequence data and the probability distribution of the speed change rate are takenas global information, the integral characteristics of the time sequence data are fully reflected, and the data point which violates the integral characteristics can be searched to effectively identify the abnormal data point so as to identify the abnormal data point. By use of the method, the unexpected abnormal data points in the time sequence data can be accurately identified.

Description

technical field [0001] The invention belongs to the field of data cleaning, and more specifically relates to a method for identifying abnormal data points in time series data based on global information. Background technique [0002] With the development of information technology, data is generated and used all the time. Data from all walks of life is constantly growing, and data has become a very important role in people's lives. Due to the widespread use of various sensors, more and more time-series data are collected and applied in daily life, such as air temperature data and GPS trajectory data. Because these data contain rich information, time series data mining has become a hot research topic today. However, at the same time, dirty data also widely exists in time series data, and low-quality time series data has brought a huge impact on data mining and analysis. There is no doubt that by cleaning the time series data, thereby improving the data quality of the time s...

AI Technical Summary

Problems solved by technology

However, such methods only smooth all the data points in the sequence, and it is difficult to effectively judge the position of the abnormal point

[0004] In summary, when identifying sudden abnormal data points in time series data, it is difficult to effectively and accurately determine the location of abnormal data points, which affects the identification of abnormal data points

Images