Low bit width bias cross-border processing method in quantitative reasoning process

Abstract

The method aims at eliminating abnormal values existing when bias are combined in the low-bit quantization process of the model, and it is ensured that the model reasoning precision and the training network framework are kept consistent. The invention provides a low bit wide bias border crossing processing method in a quantitative reasoning process, and the method comprises deep learning convolution floating point type float operation, and comprises the following steps: S1, inputting floating point type data; s2, convolution is carried out, and weight merging is carried out; s3, carrying out batch standardization, combining related parameters in a quantization process, independently processing a gamma value of a batchnorm channel, and carrying out a derivation formula on the batchnorm in a specific quantization process as follows: assuming quantization calculation of an ith layer: limiting the gamma value to 0.1 or more in an actual training process, and ensuring that the bias does not cross a boundary; and S4, outputting a floating point type result.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to a low-bit-width bias out-of-bounds processing method in a quantization inference process. Background technique [0002] In the prior art, as the number of layers stacked in the convolutional neural network model increases, the number of weight parameters of the network model also increases. The main purpose of low-bit quantization of the model is to reduce the model storage volume and feature output. Save the bit width, which can speed up the operation. However, in many low-bit models, in order to ensure the operation speed during the model quantization inference process, the feature and bias operation results are stored in an int16-bit width, so that the bias out-of-bounds situation will inevitably occur. [0003] In particular, when the float 32bit is quantized to low bits, the network model needs to be operated on int16 because of the need to ensure the operation spe...

AI Technical Summary

Problems solved by technology

However, in many low-bit models, in order to ensure the calculation speed in the process of model quantization and reasoning, the feature and bias calculation results are stored in int16-bit width, so that the bias will inevitably appear out of bounds

[0003] In particular, when quantizing float 32bit to a low bit, due to the need to ensure the calculation speed after quantization, the network model needs to be calculated on int16. Due to the addition of corresponding quantization operation methods, when merging quantization parameters, the bias is susceptible to various Influenced by factors, the bias is prone to cross-border situations, which in turn affects the quantitative reasoning accuracy of the model, causing reasoning errors in the model during the reasoning process

Images