Deep neural network compression method, system and device based on multi-group tensor decomposition and storage medium

Abstract

The invention provides a deep neural network compression method, system and device based on multi-group tensor decomposition and a storage medium. The method comprises the steps that a neural networkstructure is built; wherein the first convolution layer and the last full connection layer of the neural network structure do not use TT decomposition, and the weight matrix of the remaining network structures is expressed through a TT format; operation is directly carried out on the core tensor on the full connection layer, and the convolution layer needs to be finally restored to the size of theoriginal weight matrix for convolution; multi-TT decomposition is carried out on a convolution layer; and adding a sparse value on the basis of TT decomposition to form a new compressed network structure. The beneficial effects of the deep neural network compression method, system and device are that: the tensor column model is adopted to reconstruct the original weight matrix into several high-dimensional tensor compression models, then a new network structure is established on the basis of decomposition, parameters are reduced, experiments show that the robustness of the compression modelsis increased along with the increase of the number of modes in the depth model, and the compression method can reach a good parameter ratio.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a deep neural network compression method, system, device and storage medium based on the decomposition of multiple sets of tensors. Background technique [0002] Although the deep neural network has been successfully and widely used in practical applications, its complex structure and many parameters cause waste of resources and increase of training time. When applying deep neural networks to specific devices such as smartphones, wearables, and embedded devices, these devices have certain limitations in terms of model size, power consumption, etc. The enormous limitations of these devices make them difficult to apply to deep neural networks, thereby prompting researchers to discover the inherent redundancy in parameter and feature maps in deep models. By eliminating redundancy, resources can be saved without compromising the capacity and performance of most deep ...

AI Technical Summary

Problems solved by technology

[0005] In summary, deep neural networks usually have the characteristics of high computational complexity and large storage capacity, so it is difficult to apply them in mobile devices.

Images