Hierarchical pruning method based on layer recovery sensitivity

Abstract

The invention provides a hierarchical pruning method based on layer recovery sensitivity. The method comprises the steps of S1, performing uniform pruning on each layer of a target neural network model; S2, performing layer recovery on the target neural network model after uniform pruning to obtain the performance contribution degree of each layer to the model; S3, grading each layer in the targetneural network model according to contribution degrees, and setting a pruning proportion for each stage; and S4, pruning the original target neural network model according to the set pruning proportion. The contribution degree of each layer to the model performance is judged more visually, efficiently and simply. Moreover, the oscillation problem caused by randomness during model parameter initialization can be greatly reduced, the model calculation amount is greatly reduced, the hardware requirement is reduced, the calculation speed is increased, the calculation energy consumption is reduced, and the equipment real-time performance is improved.

Description

Technical field [0001] The invention belongs to the field of artificial intelligence, and is particularly suitable for the compression of deep neural network models. Background technique [0002] Recently, deep neural networks have made tremendous developments in various fields. In order to pursue good model performance, various research institutions gradually design network models with more weights and deeper structures, which will inevitably cause redundancy. Although the model performance has been greatly improved, it is difficult to run such a huge network on mobile edge devices with limited resources. Therefore, it is of great significance to the study of deep neural network model compression. Pruning the existing network is one of the mainstream methods of model compression. [0003] The methods of deep neural network model compression mainly include the following: 1. Pruning: tailoring the existing network structure; 2. Knowledge distillation: using the structure informati...

AI Technical Summary

Problems solved by technology

Instead of a single weight on the filter, the pruning of a single weight is unstructured, which will cause unstructured sparsity, and requires a dedicated operating environment to achieve acceleration and compression

Therefore, the current mainstream research methods are mostly structured pruning.

Images