Neural network acceleration method and device

Abstract

Embodiments of the present invention provide a neural network acceleration method and device, wherein the method includes: for any layer in the neural network, according to the quantization mode of the layer, block the feature map input to the layer to obtain several For block data, set the same index value for pixels in different channels but at the same position in the block data; calculate the sparsity of the block data, discard the block data that is all 0, and for the remaining block data, according to the remaining block data The data sparsity and the preset threshold determine the corresponding sparsity type, and perform sparse coding on the remaining block data according to the sparsity type. The implementation of the invention ensures that the number of indexes at each pixel position does not increase exponentially due to different quantization modes, and solves the problem of neural network coding in which multiple sparsities and multiple quantization bit widths are mixed.

Description

technical field [0001] The present invention relates to the technical field of accelerator design, and more specifically, to an acceleration method of a neural network. Background technique [0002] Since the current neural network has an activation function (ReLU), it can cause a large amount of feature data (featuremap) to be sparse, and using methods such as pruning to train the neural network can also cause a large amount of weight data (weight data) to be sparse. Effectively exploiting these sparsities can greatly improve the processing efficiency of neural network accelerators. At the same time, when the hardware processes the neural network, fixed-point processing will bring a great energy boost compared to floating-point processing. Using fixed-point processing for neural networks has become common practice for energy-efficient accelerators. At present, many literatures have focused on the sparsity of neural networks and the utilization of fixed-point quantization....

AI Technical Summary

Problems solved by technology

Current sparse neural network accelerators cannot handle neural networks with different quantizations simultaneously

In fact, the main technical problem is that the sparse network processing requires the index value of the data, and the neural network storage of different quantization modes will cause the index value to be unable to be stored efficiently.

Images