Method and device for quantizing neural network on processing unit

Abstract

The present disclosure describes a method, an electronic device and a computing device for quantizing a neural network on a processing unit, where the computing device may be included in a combined processing device, which may also include a universal interconnection interface and other processing devices. And the computing device interacts with other processing devices to jointly complete the computing operation specified by the user. The combined processing device can further comprise a storage device, and the storage device is connected with the computing device and the other processing devices and used for storing data of the computing device and the other processing devices.

Description

technical field [0001] The present disclosure relates to the field of computers, and more particularly, to a method of data quantification in neural networks. Background technique [0002] Int8 quantization can significantly reduce the space occupied by the neural network model and the bandwidth occupied by the neural network model when it runs. At present, the quantization process of convolutional neural network is carried out on the CPU: the network to be quantized is run on the CPU, and the required parameters are obtained by modifying the operator function to be quantized during the network operation process, and executed when the network is running. quantification process. Finally, the quantized convolutional neural network can be run on the MLU platform for application. However, the above quantization process has the following four disadvantages: 1. Since the CPU is very slow in processing the convolutional neural network, when the samples used in quantization are to...

AI Technical Summary

Problems solved by technology

However, the above-mentioned quantization process has the following four disadvantages: 1. Due to the slow speed of the CPU processing the convolutional neural network, when the samples used in the quantization are too large, the time spent on the entire quantization will increase significantly; The large operator spliced ​​by several cnml operators and the operator implemented by Bang c language (generally defined by the user, not supported by the official framework) cannot run on the CPU, resulting in the failure of the entire quantization; 3. For multiple cnml For a large operator spliced ​​by operators, the operator to be quantified may be one of the cnml operators, and the relevant parameters cannot be obtained during the network operation process, which leads to the inability to quantify the spliced ​​operator; 4. Destruction of the framework structure, and Subsequent framework changes and quantitative tools also need to be modified, which increases maintenance costs

Images