Compression method and system for deep convolutional neural network

Abstract

The invention relates to a compression method and system for a deep convolutional neural network, and the method comprises the steps: an unimportant filter in a to-be-compressed deep convolutional neural network is determined according to a filter importance selection mode and/or a model compression rate; progressive sparse constraints are applied to the unimportant filters and added into a loss function of network training as regular terms, and an optimized loss function is obtained; according to the regular term, a threshold iterative algorithm and a back propagation algorithm are adopted for combined solving, and updating parameters of the to-be-compressed deep convolutional neural network are obtained; a convolutional neural network model with a filter sparse form based is obtained onthe optimization loss function and the updating parameters; and the convolutional neural network model is pruned with the filter sparse form by using a structured pruning algorithm to obtain a compressed convolutional neural network model with relatively high network precision.

Description

technical field [0001] The invention relates to the technical fields of convolutional neural networks and artificial intelligence, in particular to a compression method and system for deep convolutional neural networks. Background technique [0002] With the continuous development of deep neural networks in the field of artificial intelligence, such as computer vision, speech recognition, and natural language processing, researchers from all walks of life have achieved more intelligent results by deploying artificial intelligence-related algorithms into actual products. Effect, and then promote the arrival of a new round of artificial intelligence research climax. [0003] However, when deep neural networks are deployed in practical applications, they require huge storage space and high computing resource consumption, making it difficult to apply them in some mobile terminals and embedded devices. Therefore, deep neural network model compression and acceleration methods hav...

AI Technical Summary

Problems solved by technology

[0005] However, the usual structured pruning method has the following disadvantages: 1) When removing redundancy, unimportant connections are directly removed, resulting in a large drop in network accuracy; 2) The pruning method based on sparse constraints, in the entire In the process of model training, a fixed penalty is always added to all connections, so that important connections are affected in the process of learning sparseness, which leads to a decrease in model accuracy

Images