Progressive supervised deep learning neural network training method, system, medium and device

Abstract

The invention discloses a progressive supervision deep learning neural network training method, system, medium and equipment. The method includes: establishing an initial deep learning neural network; using the classifier layer of the initial deep learning neural network as the current training layer to be supervised ; Use the labeled training input data as the input data of the initial deep learning neural network, and perform top-down supervised training on the current unsupervised training layer; when the supervised training fails, the current unsupervised training layer and its previous hidden layer As the current to-be-supervised training layer, perform top-down supervised training on the current to-be-supervised training layer; when the supervised training is successful, output the current deep learning neural network. The invention corrects unnecessary characteristic details not eliminated during unsupervised training through supervised training, so as to achieve the effect of sufficient fitting but not overfitting, thereby improving the training success rate of sample data and test data.

Description

technical field [0001] The invention relates to a deep learning neural network training method, in particular to a progressively supervised deep learning neural network training method, system, medium and equipment, belonging to the field of neural network training. Background technique [0002] Existing deep learning technology can obtain output labels through input data (such as obtaining the ID card number of the person through the avatar, or obtaining the ID card number of the person through voice). Supervised training (such as avatars with ID number labels, and speech with ID number labels). [0003] However, the top-down supervised training of existing deep learning techniques either only adjusts the network weights between the output layer and the hidden layer, or adjusts the network weights of all layers. When there are more categories of top-level concepts than labels, if only the weights of the classifier network between the output layer and the hidden layer are a...

AI Technical Summary

Problems solved by technology

[0003] However, the top-down supervised training of existing deep learning techniques either only adjusts the network weights between the output layer and the hidden layer, or adjusts the network weights of all layers

When there are more categories of top-level concepts than labels, if only the weights of the classifier network between the output layer and the hidden layer are adjusted, and if the network structure of the classifier is relatively simple, the result of repeatedly adjusting the network parameters of the classifier is often in line with This output label cannot match that output label, which means that a sufficient fit cannot be achieved

If the network structure of the classifier is designed to be very complex, such as using a hierarchically complex BP neural network as the classifier, then there will be over-fitting, which will cause some key features to be discarded during the fitting, so that the sample The classification result is completely correct, but it will be found to be wrong when it is applied

[0004] It can be seen that only supervised training on the level between the output layer and the hidden layer will either fail to fully fit or overfit, which will cause deep learning to fail when applied.

If the network weights of all layers are adjusted, the cognitive weights and generation weights in the hidden layer will be destroyed, so that the concepts and scenes obtained after adjustment are no longer completely derived from the characteristics and scenes of the input data, but for the output labels The features and scenes that are distorted by the needs of the system will also appear overfitting, so that the classification result is completely correct for the sample, but it will be found to be wrong when it is applied.

Images