Method for filling station meteorological data missing value based on svd (singular value decomposition) algorithm

Abstract

The invention discloses a method for filling station meteorological data missing value based on an svd (singular value decomposition) algorithm, which mainly solves the problem of influence to filling accuracy of missing value caused by using single model to fill different attributes of data in the prior art. The method comprises the following implementing steps of (1) receiving data; (2) preprocessing raw data; (3) selecting a training set and a testing set; (4) training parameters; (5) filling the missing value; (6) outputting the filled station meteorological data. The method has the advantage that by singly extracting each meteorological attribute, a data file is formed, the svd algorithm training is respectively performed, and the station meteorological data after missing value filling is obtained, so that the robustness and the accuracy of missing data filling are improved.

Description

technical field [0001] The invention belongs to the technical field of computers, and further relates to a method for filling the missing value of station meteorological data based on a singular value decomposition (Singular Value Decomposition) algorithm in the technical field of data processing. The present invention can be applied to data missing scenarios caused by mechanical and human factors in weather and meteorological stations. Considering the potential relationship between attribute features in meteorological station data, the optimal calculation model is obtained by using the singular value decomposition svd algorithm for training, so as to be more accurate fill in missing values. Background technique [0002] Missing data means that the value of one or some attributes in the existing data set is incomplete, which is caused by mechanical failure or human subjective error. Each type of missing will have different impacts on statistical analysis. How to effectively...

AI Technical Summary

Problems solved by technology

However, the disadvantage of this method is that it does not perform differential training and filling on the data of different attributes. When abnormal conditions such as missing data and errors occur, it simply calls the previous data at the current invalid time point. If If the data at the previous moment is also missing or wrong, the upward retrospective method is used for filling processing. This method is too simple, and there is no obvious correlation between the data at the previous moment and this moment, resulting in low accuracy of the filled data. Subsequent studies have introduced errors

However, the disadvantage of this method is that the range of data blocks extracted when judging whether it is abnormal is only 1 to 3, too few data blocks are selected, and the missing data is only the average value of the data blocks that do not include missing observation points. To replace, which is based on the assumption of complete random missing, the accuracy of data filling is not high

Images