Multi-algorithm fusion-based missing value interpolation method

Abstract

The invention provides a multi-algorithm fusion-based missing value interpolation method. The method comprises the following steps of: S1, carrying out hierarchical clustering on all the pieces of data; S2, judging whether classes with missing values are records with missing values or not, and dividing the records into a missing data group and a complete data group; S3, randomly dividing data in the complete data group into a training set and a test set, and predicting the test set by using n existing interpolation methods so as to construct a certain quantity of sample sets; S4, interpolatingthe classes with missing values by using a neural network model so as to obtain a final interpolation value; S6, judging whether classes with missing values still exist or not, if the judging resultis positive, executing the step S2, and otherwise, executing step S7; and S7, ending the operation. According to the method, missing values are obtained by using a plurality of existing methods, so that the defects of artificially and subjectively selecting missing value interpolation methods are solved and the missing values can be interpolated objectively and effectively.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a missing value interpolation method based on fusion of multiple algorithms. Background technique [0002] In many studies that need to collect data, missing data is very common, and the reasons for missing data are various, mainly mechanical and human. The former is the failure of data collection or preservation due to mechanical reasons, such as data storage failure, memory damage, etc.; the latter is the lack of data caused by human subjective mistakes, historical limitations or intentional concealment, such as the interviewee in a questionnaire survey. The personnel refused to disclose the answers to the relevant questions, or the questions answered were invalid, and the data entry personnel made mistakes and omitted data. Before the investigation, it is extremely necessary to plan well and pay attention to avoiding missing data for some important data. However, for ...

AI Technical Summary

Problems solved by technology

Without a comprehensive understanding of data distribution and missing conditions, it is impossible to know which method has a better imputation effect on missing values. At this time, people often choose one of many imputation methods based on past experience or at will. It is difficult to control the rationality of the imputation of missing values, especially for some key variables, the imputation values ​​obtained by different imputation methods may be very different, and the results and research conclusions obtained may be different. will be completely different

Images