Convolution operation structure for reducing data migration and power consumption of deep neural network

Abstract

The invention discloses a convolution operation structure for reducing data migration and power consumption of a deep neural network. The convolution operation structure comprises a multiplier and anadder. The input end of the multiplying unit is connected with the multiplexer MUX1 and the multiplexer MUX2 respectively; the output end of the multiplier and the output end of the multiplexer MUX1 are connected with the input end of the adder through the multiplexer MUX3; the input end of the adder is also connected with the input end of a multiplexer MUX4; wherein the output ends of the multiplexer MUX1, the multiplexer MUX2, the multiplier, the multiplexer MUX3, the multiplexer MUX4 and the adder are respectively connected with the register reg1, the register reg2, the adder and the register reg3; the output end of the adder is connected with the register reg2, and the output end of the register reg2 is connected with the input end of the multiplexer MUX4 and used for achieving multiply-accumulate operation of convolution operation. The convolution operation structure is suitable for all current convolutional neural network models, effectively reduces the dynamic power consumptionof global calculation on the premise of meeting the data parallelism degree to the maximum extent, is simple in control structure, and is very high in universality.

Description

technical field [0001] The invention belongs to the technical field of integrated circuit design and special hardware accelerator, and in particular relates to a convolution operation structure for reducing data migration and power consumption of a deep neural network. Background technique [0002] In recent years, with the important breakthroughs in speech and image recognition, deep neural networks (DNNs) have become the basis of many modern artificial intelligence applications. The superior performance of DNNs benefits from its ability to extract high-level features from large amounts of data to obtain effective representations of similar data. A common form of DNNs is Convolutional Neural Networks (CNNs), the main part of which is composed of multiple convolutional layers, each of which is a higher-dimensional abstraction of the input feature map (ifmap). Currently, CNNs The number of convolutional layers has evolved from the original 2-layer LeNet to the current 53-lay...

AI Technical Summary

Problems solved by technology

This structure fixes the weights in the registers in the PE, avoiding the delay and power consumption overhead caused by weight access, but this solution requires frequent partial and migration between PEs, and the entire PE array can only Output a convolution result

(2) Partial and fixed system is adopted, which is different from the flow migration of results with fixed weights. This system requires that each convolution operation result is fixed to be output by a PE unit, and its characteristic data can be reused horizontally and vertically adjacent to the PE unit. This system It mainly reduces the power consumption of the multiplication part and the result flowing between different PEs. However, the proposed structure does not achieve the maximum reuse of feature data, and does not consider the reusability of weights at the PE level.

(3) Adopt a row-fixed system, which requires each PE to read the input feature map and a row of data of the convolution kernel. Unlike the previous two systems, which calculate the output feature map data one by one, here each cycle PE will generate output features in parallel. The intermediate results of multiple data, but due to the large difference in the size of the feature maps of different applications, and the architecture requires the complete feature map information to be read in by row, so the storage structure in the PE array is less flexible and adaptable

[0004] The purpose of the above-mentioned different hardware acceleration engine systems is to reduce repeated data access and try to compress the huge power consumption overhead caused by multiplication and accumulation in theoretical calculations. However, they all only mine reusability from factors such as weights and input feature maps. Failed to further improve computing performance

Images