Neural network quantification method based on parameter norms

Abstract

The invention provides a neural network quantification method based on parameter norms. The method comprises the steps that for a given pre-trained neural network parameter model, a quantification center is divided by performing statistical analysis on values of parameters of needed quantification layers; according to the selected quantification center, quantification loss of the parameters of each corresponding quantification layer is calculated; the quantification loss and classification error loss of the neural network training parameter model are added to serve as total loss, back propagation optimization is performed, and meanwhile the quantification center is updated during optimization; and after training is over, quantification operation is performed on the corresponding layers according to the quantification center, and a compression model after quantification is obtained. Through the method, a weight center can be divided, and the neural network model is quantified by exerting simple quantification loss and using an optimizer the same as that in a traditional method, so that the compression model of the original model is obtained, and the network storage volume and operation complexity are reduced.

Description

technical field [0001] The invention relates to the field of neural networks, in particular to a parameter norm-based neural network quantification method. Background technique [0002] As early as the end of the last century, Yann LeCun and others have used neural networks to successfully identify handwritten postal codes on mail. In recent years, different neural network structures have emerged one after another, and have achieved good results far exceeding traditional algorithms. Huge breakthroughs have been made in many fields such as computer vision, speech processing, and recommendation systems, and they are also widely used in industries such as the Internet, smart devices, and security equipment. has been widely used. [0003] In order to make the neural network achieve better results, the network parameters are supervised optimization and learning based on a large-scale labeled data set during the training process. At the same time, in order to learn data more com...

AI Technical Summary

Problems solved by technology

Therefore, the results after quantization often have a large decrease in accuracy compared with the original model, which is difficult to use in actual application scenarios.

Images