Sparse Neural Network Architecture and Its Implementation

Abstract

The invention discloses a sparse neural network architecture and a realization method thereof. The sparse neural network architecture comprises an external memory controller, a weight cache, an inputcache, and output cache, an input cache controller and a computing array, wherein the computing array comprises multiple computing units, each row of reconfigurable computing units in the computing array share partial input in the input cache, and a partial weight, shared by each column of reconfigurable computing units, in the weight cache is computed; the input cache controller performs sparse operation on input of the input cache, and a zero value in the input is removed; and the external memory controller stores data of the computing array before and after processing. Through the sparse neural network architecture and the realization method thereof, invalid computing performed when the input is zero can be reduced and even eliminated, computed quantities among all the computing units are balanced, the hardware resource utilization rate is increased, and meanwhile shortest computing delay is guaranteed.

Description

technical field [0001] The present invention relates to neural network deep learning technology, in particular to a sparse neural network architecture and its implementation method. Background technique [0002] In recent years, excellent hardware architectures for deep learning have also emerged. For example, Nvidia dominates the current deep learning market with its large-scale parallel GPU and dedicated GPU programming framework CUDA. More and more companies have developed hardware accelerators for deep learning, such as Google's Tensor Processing Unit (TPU / Tensor Processing Unit), Intel's Xeon Phi Knight's Landing, and Qualcomm's Neural Network Processor (NNU / Neural Network Processor). Teradeep is now using FPGAs (Field Programmable Gate Arrays) because they are 10 times more energy efficient than GPUs. FPGAs are more flexible, scalable, and have higher performance-per-watt ratios. These hardware structures have good performance for dense deep neural networks, but are...

AI Technical Summary

Problems solved by technology

[0004] Second, due to the application of the activation function will bring a large sparsity

[0013] Third, some compression algorithms of neural networks that are currently very popular reduce the amount of calculation through pruning and quantization, which will also bring about the sparsity of the network.

Images