General miniaturization method of deep neural network

Abstract

The invention discloses a general miniaturization method of a deep neural network. The method comprises the following steps: performing feature reconstruction on an initial deep neural network to forma new small network; compressing the number of feature channels and the number of group convolutions of the deep neural network to reduce the number of output feature channels and the number of groupconvolutions, wherein the feature reconstruction uses a feature layering approach, a predetermined compression ratio is set, and the low-level features of the deep neural network are first reconstructed and then the high-level features of the deep neural network are reconstructed at the same predetermined compression ratio. The general miniaturization method reduces storage space, simplifies thenetwork, maintains good stability and effectiveness, and is widely applied to deep neural network models deployed on embedded and mobile computing platforms.

Description

technical field [0001] The invention belongs to the technical field of deep learning, in particular to a notification miniaturization method of a deep neural network. Background technique [0002] In recent years, the rapid development of deep learning has made leapfrog progress in the performance of algorithms in a series of fields such as computer vision and natural language processing. Deep learning algorithms have been widely used in academia, but they have not been widely used in industry. One of the reasons is that the model of deep learning network is huge and the amount of calculation is huge. The weight file of a convolutional neural network is often hundreds of megabytes. , In a scientific research environment, a huge GPU cluster can be used to increase the training speed and reduce the running time, but if it is to be put into an actual product, the capacity of hundreds of megabytes is unacceptable. At present, the performance of smartphones is already very super...

AI Technical Summary

Problems solved by technology

[0006] 3. Compact network: Designing a more compact network structure is actually to reduce the parameters of each layer of the network. However, even if there is a large amount of training data using this method, when the model is small to a certain extent, it will also bring performance degradation.

Images