Neural network acceleration and compression method based on trace norm constraints

Abstract

The invention discloses a neural network acceleration and compression method based on trace norm constraints, which comprises the following steps: forward propagation of the neural network is carriedout, and whether to converge is judged; a task-related loss function and trace norm constraints are separated, and neural network back propagation based on the loss function is carried out; trace normconstraints are carried out on each layer of parameter matrix after updating of the back propagation; the parameter matrix after the trace norm constraints is updated to the network; and after the training process of the neural network converges, the final parameter matrix is subjected to singular value decomposition, and the neural network is compressed and reconstructed. Through improving the low rank characteristics of model parameters during a model training stage, good acceleration and compression effects are obtained, the applicability is wide, the precision is not influenced, and the needs of reducing storage space and improving the processing speed in practical applications can be met.

Description

technical field [0001] The invention relates to an acceleration and compression method, in particular to a neural network acceleration and compression method based on trace norm constraints, and belongs to the technical field of neural networks. Background technique [0002] A neural network is a feedforward network composed of artificial neurons. A neural network consists of various types of layers, including convolutional layers, fully connected layers, pooling layers, and activation layers. When the input data passes through each layer, it is operated with the parameters of each layer to obtain the output of the neural network. Neural networks are widely used in many computer vision tasks such as target detection, image classification, semantic segmentation, and feature point positioning. When the neural network has tens of thousands of neuron connections, the physical storage requirements of the neural network parameters will be very high. Applications on devices with ...

AI Technical Summary

Problems solved by technology

However, this type of method only designs acceleration and compression methods for the convolutional layer, and has limited versatility for other layers in the neural network, and losing low-frequency information will also bring a relatively large loss of accuracy.

[0004] Although the above methods can accelerate and compress to a certain extent, their scope of application is limited, and they all passively compress the trained model, without considering the compression ability of the model from the training stage, reducing the compressed precision loss

Images