Method and apparatus for quantizing and compressing neural network with adjustable quantization bit width

Abstract

The invention relates to the technical field of neural networks, and specifically provides a method and apparatus for quantifying and compressing a convolutional neural network. The invention aims to solve the existing problem of large loss of network performance caused by an existing method for quantifying and compressing a neural network. The method of the invention comprises the steps of obtaining a weight tensor and an input eigen tensor of an original convolutional neural network; performing fixed-point quantization on the weight tensor and the input eigen tensor based on a preset quantization bit width; and replacing the original weight tensor and the input eigen tensor with the obtained weight fixed-point representation tensor and the input feature fixed-point representation tensor to obtain a new convolutional neural network after quantization and compression of the original convolutional neural network. The method of the invention can flexibly adjust the bit width according to different task requirements and can realize quantization and compression of the convolutional neural network without adjusting the algorithm structure and the network structure so as to reduce the occupation of memory and storage resources. The invention further provides a storage apparatus and a processing apparatus, which have the above beneficial effects.

Description

technical field [0001] The invention belongs to the technical field of neural networks, and specifically provides a method and device for quantizing and compressing convolutional neural networks. Background technique [0002] In recent years, with the development of convolutional neural network in the field of target detection and recognition, its detection accuracy has reached the commercial level. At the same time, the rapid development of portable devices (such as mobile terminals, smart devices) has allowed researchers to see the future Opportunities for combining convolutional neural networks with portable devices. However, target recognition based on convolutional neural networks often relies on high-performance GPU (Graphics Processing Unit, Graphics Processing Unit) equipment, which requires a huge amount of computation and consumes a large amount of memory. If it is run on a smart phone or an embedded device The convolutional neural network model will quickly consu...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the above-mentioned problems in the prior art, that is, in order to solve the problem that the performance of the convolutional neural network is greatly lost when the prior art quantizes and compresses the convolutional neural network, an aspect of the present invention provides a convolutional neural network. Methods for quantization and compression of product neural networks, including:

Images