Deep network model compression training method based on generative adversarial neural network

Abstract

The invention discloses a deep network model compression training method based on a generative adversarial neural network. The method comprises the steps: randomly selecting a picture from a trained data set to serve as input of an original network and an initial compressed network; taking the feature extractor of the initial compressed network as a G network; calculating the JS divergence of thefeature indexing of the feature map extracted by the feature extractor of the original network and the initial compressed network as a structural loss function; constructing a comprehensive loss function based on the structural loss function and a self-loss function of the initial compressed network; respectively inputting the feature maps extracted by the original network feature extractor and the initial compressed network feature extractor into a D network, updating the D network or the compressed network according to the judgment result of the D network, and finally enabling the feature map output by the G network to be infinitely close to the feature map of the original network, wherein the compressed network is updated based on the comprehensive loss function, so that the predictionprecision and robustness of the model are improved.

Description

technical field [0001] The present invention relates to the technical field of deep network model compression, and more specifically relates to a deep network model compression training method based on generating an adversarial neural network. Background technique [0002] With the rapid development of deep learning technology, deep neural networks have achieved leapfrog breakthroughs in the fields of computer vision, speech recognition, and natural processing. However, deep learning algorithms have not been widely used in the fields of industry, manufacturing, aerospace and navigation. One of the reasons is that the model of the deep learning network is huge and the amount of calculation is huge. The weight file of a CNN network can easily be hundreds of megabytes, such as AlexNet With 61M parameters and 249MB memory, the memory capacity of complex VGG16 and VGG19 has exceeded 500MB, which means larger storage capacity and more floating-point operations are required. Due t...

AI Technical Summary

Problems solved by technology

[0005] Whether it is pruning or quantization of the network, the accuracy and robustness of the deep network model will be more or less reduced. As the compression degree of the deep network model continues to increase, the loss of accuracy and robustness will drop sharply.

Images