A Deep Network Compression Method Based on Dimension Adaptive Tucker Decomposition

Abstract

The invention discloses a deep network compression method of Tucker decomposition based on dimension adaptive adjustment, which includes a dimension adaptive adjustment process and a dimension adaptive weight tensor decomposition process, and generates a new tensor by adapting to adjust the size of each dimension of the tensor Arbitrary order tensors, and then realize tensor decomposition through learnable kernel tensors and transfer matrices, thereby achieving the purpose of network optimization and compression. Compared with the existing low-rank compression method, the present invention has a larger compression factor for network parameters while maintaining network performance, and can obtain a higher compression factor; at the same time, it does not need to store the guide position of non-zero elements , does not need to record the index, and can use the storage space more effectively.

Description

technical field [0001] The invention relates to the technical field of deep learning network optimization compression, in particular to a deep network compression method based on dimension adaptive Tucker decomposition. Background technique [0002] Due to the rapid development of image processor (GPU) computing power in recent years and the increasing amount of data that people can obtain, deep convolutional networks have achieved significant results in the fields of computer vision and natural language processing. According to the development history of the neural network structure (documents [1]~[5]), there are two major trends in the current network structure: the number of layers of the network is getting deeper and deeper, and the convolutional layer is becoming more and more important. While these deeper networks can achieve good results on some problems, their computational cost and storage cost are not a small problem. [0003] Due to the limitations of certain con...

AI Technical Summary

Problems solved by technology

Since the parameters are sparse, the problem with this method is that it needs to record the coordinate guidance of non-zero elements, which will waste a large part of storage space

The problem with this method is that tensor decomposition can only solve fixed-order tensors, and the form of decomposition is fixed. Tensor decomposition is mainly achieved by adding more layers; although this method has achieved good results in network acceleration effect, but due to the limitation of decomposition, the original tensor decomposition method cannot solve the problem of network compression very well

Images