Data processing method and system and electronic equipment

Abstract

The invention provides a data processing method, and the method comprises the steps: obtaining a to-be-processed training data set which comprises training data; generating target training data corresponding to the training data according to the target code corresponding to the training data and the acquisition time point; determining to-be-processed training data contained in the to-be-processed training data set, and generating an increment time point corresponding to the to-be-processed training data according to the acquisition time point of the to-be-processed training data and a corresponding time processing rule; generating incremental training data corresponding to the to-be-processed training data according to the incremental time point, the to-be-processed training data and the corresponding target code; generating a target training data set according to the incremental training data and the target training data so as to train the to-be-trained model by using the target training data set; and achieving data enhancement based on the newly added tag. The method and system can be applied to structured data, and the training effect on the model can be improved based on the obtained target training data.

Description

technical field [0001] The invention relates to the field of structured data, in particular to a data processing method, system and electronic equipment. Background technique [0002] Data augmentation has been widely used in the field of unstructured data, but in the field of structured data, there are not many effective methods for data augmentation. The fundamental difficulty lies in the inability to verify the accuracy of the newly added data labels. [0003] In the prior art, there are some data augmentation methods applied to structured data. There is a way to do down sampling (up sampling) for types with rich labels or up sampling (down sampling) for types with few labels, but this method only changes the weight of the existing data and cannot provide true data for the model. Therefore, the effect of data enhancement is not good; another method is to make small disturbances to the characteristics of existing data to form new data, but this method is not suitable for ...

AI Technical Summary

Problems solved by technology

There is a way to do down sampling (up sampling) for types with rich labels or up sampling (down sampling) for types with few labels, but this method only changes the weight of the existing data and cannot provide true data for the model. Therefore, the effect of data enhancement is not good; another method is to make small disturbances to the characteristics of existing data to form new data, but this method is not suitable for more sensitive features, but it cannot be used for insensitive features. Have a significant impact, resulting in poor actual results; there is another way to shuffle the order of features, while the labels remain the same, but it is only applicable to specific scenarios

Images