Principal component analysis (PCA)-based neural network pruning method

Abstract

The invention discloses a neural network pruning method based on PCA, and relates to the technical field of pruning, and the method comprises the steps: setting a percentage parameter of reserved information; extracting convolution kernel parameters of each layer from top to bottom; analyzing and transforming convolution kernel parameters by using PCA, and calculating the number and parameters of convolution kernels to be reserved according to percentage parameters; reconstructing a neural network model through the reserved parameters of each layer; and performing fine adjustment on the model by using the training data to obtain a pruned model. According to the new neural network pruning method provided by the invention, main components of the model are reserved, redundant information of a convolutional layer is removed, and parameters and calculation amount are reduced, so the model can be deployed on a low-power-consumption platform, and cost overhead is reduced.

Description

technical field [0001] The invention relates to the technical field of pruning, in particular to a PCA-based neural network pruning method. Background technique [0002] Neural networks are more and more widely used in the fields of image recognition, speech recognition, and machine translation. However, high-precision models often have a large amount of parameters and calculations, which greatly limits the application of neural networks. Therefore, model compression comes into being. pregnancy. The early model compression algorithm retained the important convolution kernel of each network layer through the absolute value of the parameters or some statistical information of the feature map, but ignored the information of the discarded network layer, resulting in a decrease in accuracy. [0003] Therefore, those skilled in the art are committed to developing a PCA-based neural network compression method. By setting the percentage of information retained in each layer, the re...

AI Technical Summary

Problems solved by technology

[0002] Neural networks are more and more widely used in the fields of image recognition, speech recognition and machine translation. However, high-precision models often have a large amount of parameters and calculations, which greatly limits the application of neural networks. Therefore, model compression comes into being. pregnancy

The early model compression algorithm retained the important convolution kernel of each network layer through the absolute value of the parameters or some statistical information of the feature map, but ignored the information of the discarded network layer, resulting in a decrease in accuracy

Images