Neural network model compression method and device

Abstract

The invention discloses a neural network model compression method and a neural network model compression device. The neural network model compression method comprises the steps of: determining a model parameter set of each neuron layer for each neuron layer in a neural network model, wherein the model parameter set includes a plurality of model parameters; carrying out first transformation on the plurality of model parameters to generate a plurality of intermediate parameters; quantizing the plurality of intermediate parameters according to a preset quantization step size to obtain a plurality of quantization parameters; selecting a plurality of sampling quantization points from the plurality of quantization parameters according to a preset quantization bit number; generating quantized values of the plurality of model parameters according to values of the plurality of quantization parameters and the plurality of sampling quantization points; and compressing and storing the plurality of model parameters according to the quantized values. The neural network model compression method can better maintain the effect of the model, greatly reduces the size of the neural network model, and reduces the occupation of computing resources, especially memory resources.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a neural network model compression method and device. Background technique [0002] At present, with the increase of training data in deep neural network, in order to better learn the characteristics of training data to improve the effect, the parameters used to represent the model increase rapidly, so the consumption of computing resources also increases rapidly, which will restrict the deep neural network. Network application scenarios, such as mobile phones and other devices with limited computing resources. [0003] In related technologies, the size of the model is usually reduced by quantizing and compressing the neural network model. At present, there are usually the following two compression methods for neural network models: 1) Half-precision-based compression methods, the principle of which is to compress the floating-point numbers represented by 32bit (bits) in the n...

AI Technical Summary

Problems solved by technology

[0004] However, the existing problems are: the above-mentioned compression method based on half-precision only compresses the floating-point number to 16 bits, which makes the compressed size limited; the above-mentioned quantization method based on random sampling, due to the randomness of the sampling, makes the sampling The point will fall near the peak, and the sampling of important and large-value elements may be insufficient; in the above-mentioned linear-based quantization method, linear quantization treats large weights and small weights on average, and does not sample more in places where data distribution is dense , leading to large loss of model data and poor compression effect

Images