Neural network model compression method, system and device and medium

Abstract

The invention provides a compression method and system for a neural network model, electronic equipment and a readable medium. The method comprises the steps: presetting a pruning strategy of each network layer, and the pruning strategy comprises a pruning rate, a pruning threshold and a filter weight value; in the current compression period, selecting a pruning filter to be pruned from each network layer according to a pruning strategy, and setting the value of each pruning filter to be zero; setting the value of the pruning filter to be a non-zero value through fine adjustment so as to update the neural network model and obtain a current neural network model; and determining whether the pruning strategy of the next compression period needs to be adjusted or not according to the current meta-attribute value output by the current neural network model. Allowing the pruned filter to update without reducing the number of feature maps of the network; an extra model fine adjustment stage isnot needed any more, and the network training time is shortened; according to the parameter statistical distribution of the current model, the pruning strategy most suitable for the current model parameters is selected, and the model training effect is improved.

Description

technical field [0001] The invention belongs to the technical field of neural network compression, and in particular relates to a model compression method, in particular to a model compression method based on meta-learning and soft pruning. Background technique [0002] The mainstream methods for compressing and accelerating neural networks can be divided into five types: 1) parameter pruning; 2) parameter sharing; 3) low-rank decomposition; 4) compact convolution Core design (designing compact convolutional filters); 5) knowledge distillation. Parameter pruning mainly removes redundant parameters by designing criteria for judging whether parameters are important or not. Parameter sharing mainly explores the redundancy of model parameters, and uses technologies such as Hash or quantization to compress weights. Low-rank decomposition utilizes matrix or tensor decomposition techniques to estimate and decompose the original convolution kernels in deep models. The design of t...

AI Technical Summary

Problems solved by technology

[0010] Once the compression strategy is selected, the selected strategy will remain constant throughout the compression debugging process and cannot be adjusted according to changes in the situation, such as changes in the probability distribution of filter parameters or changes in the architecture of the deep learning model

Images