Multi-precision neural network computing device and method based on data stream architecture

Abstract

The embodiment of the invention provides a multi-precision neural network computing device based on a data stream architecture, which comprises a microcontroller and a PE array connected with the microcontroller, and each PE of the PE array is configured with a plurality of low-precision computing components with original precision and precision lower than the original precision. The computing component with lower precision is provided with more parallel multiply accumulators to make full use of the on-chip network bandwidth, and each low-precision computing component in each PE is provided with sufficient registers to avoid data overflow; and the microcontroller is configured to respond to an acceleration request for a specific convolutional neural network, control a calculation component with original precision or low precision matched with the precision of the specific convolutional neural network in the PE array to execute operation in corresponding convolution operation and store an intermediate calculation result to a corresponding register. Therefore, the convolutional neural networks with different precisions can be accelerated, the calculation time delay and energy consumption are reduced, and the user experience is improved.

Description

technical field [0001] The present invention relates to the field of computers, in particular to the field of acceleration devices or accelerators for neural network model calculations, and more specifically, to a multi-precision neural network calculation device and method based on a data flow architecture. Background technique [0002] Deep Neural Network (DNN) has shown significant advantages in many application fields, from computer vision to natural language processing and speech recognition. The support of powerful hardware makes it easier to train DNN models. With the diversification and complexity of applications, The DNN model is also complex, its depth is getting deeper and its parameters are getting larger and larger, and the complex DNN model is more expressive in capturing the nonlinear relationship between features and output, thus showing good result accuracy. [0003] Although these neural networks are very powerful, a large number of weights will occupy a la...

AI Technical Summary

Problems solved by technology

However, on the existing accelerators, since there are no relevant computing components to support low-precision calculations, and there is no corresponding instruction support, these low-precision calculations cannot be supported

Therefore, low-precision network calculations cannot be performed to reduce the power consumption of the accelerator

If the original precision operation unit is used to perform low-precision calculations, the power consumption is wasted and there is no performance improvement

In addition, the results of low-precision calculations are usually saved with a wide bit width to avoid data overflow. In addition, the use of SIMD operations in existing acceleration devices has high data bandwidth (on-chip network bandwidth). When the designed low-precision components are unreasonable, Not only will the operating bandwidth be wasted, but it will also cause data overflow, which makes it challenging to support multiple low-precision component designs

Images