Convolutional neural network initialization method based on pre-training model filter extraction

Abstract

The invention provides a convolutional neural network (CNN) initialization method based on pre-training model filter extraction and relates to the technical field of video processing. The method extracts a filter parameter in a pre-training model by using a minimum entropy loss and a minimum reconstruction error in order to initialize a target task network model and achieve the small and medium-sized network initialization that meets practical application problems. By using the minimum entropy loss and the minimum linear reconstruction method, the method extracts the filter parameter from thepre-training model to initialize the target task network model, does not require that the target task network structure is consistent with the pre-training network structure, flexibly design the network structure of the target task according to practical applications so as to meet the memory overhead and calculation speed requirements in practical application problems.

Description

technical field [0001] The invention relates to the technical field of video processing, in particular to a method for initializing a convolutional neural network. Background technique [0002] Convolutional Neural Network (CNN) realizes the approximation of the objective function by learning a multi-layer nonlinear network structure and using a simple network structure, and then can learn the feature representation of the sample data from the original data sample set. Thanks to massive amounts of data, deep convolutional neural networks are one of the most important breakthroughs in artificial intelligence and machine learning in recent years, and they have achieved great success in image analysis, speech recognition, and natural language processing. [0003] Since there is often only a small amount of data in practical application problems, the trained CNN model has overfitting, its model generalization ability is weak, and its performance on target tasks is poor. An effe...

AI Technical Summary

Problems solved by technology

However, this method of initialization using pre-trained models has the following limitations

First of all, using the pre-training model requires that the network structure of the pre-training model is consistent with the network structure of the target task, such as the number of filters in the convolutional layer, the size of the filter, and the step size, which makes the network model unable to flexibly design the network structure according to the target task

Secondly, since the network structure of the pre-training model is usually large, the content overhead and calculation speed of the target task in the actual application problem are high, so the large-scale network structure model cannot be adapted to the target task in the actual application problem.

Images