Time series data cleaning method and system

Abstract

The invention relates to a time series data cleaning method and system. The method comprises the following steps: 1) collecting one piece of original data, wherein the original data comprises multiple pieces of original time series data; 2) carrying out random sampling and estimation on the original time series data to obtain the multiple pieces of estimation data, completing vacancy points generated by the random sampling, and obtaining multiple pieces of completion estimation data; 3) classifying all completion estimation data according to sampling time points to obtain multiple groups of time classification data, and sorting all groups of time classification data according to the magnitude to obtain multiple groups of sorting arrays; 4) processing each group of sorting array to obtain one piece of corresponding mean data, wherein multiple groups of sorting arrays correspond to multiple pieces of mean data, and the mean data forms a mean series; 5) outputting the mean series, wherein the mean series is data for clearing aberrant points and high-frequency noise. Integrated data cleaning is used for processing vacancy values, removing aberrant points and smoothening noise data.

Description

technical field [0001] The invention relates to a time series data cleaning method and system, belonging to the technical field of data processing. Background technique [0002] Data analysis is currently a hot topic in the field of artificial intelligence and database research. The first step in the data analysis process is data preprocessing. Data preprocessing can effectively improve data quality and provide more targeted available data for the data mining core. Not only can save a lot of time and space, but also the mining results can play a better role in decision-making and prediction. [0003] According to statistics, in the entire process of time series data analysis, data preprocessing accounts for 60% of the entire workload, which shows its importance. The reason is that the original time series data often has certain wild points and high-frequency noise, because the data in the real world is often incomplete, noisy and inconsistent, and the real data trend is los...

AI Technical Summary

Problems solved by technology

[0014] The technical problem to be solved by the present invention is to provide a method for removing wild points and high-frequency noise in the original data for the next step, aiming at the shortcomings of the current method that can only process the default value, wild points and noise data separately. For data analysis, time series data cleaning method and system based on Kalman filter and statistical average

Images