Compression method for deep recurrent neural network based on SVD and pruning

Abstract

The invention relates to a compression method for a deep recurrent neural network based on SVD and pruning. The compression method comprises the steps of S1, performing SVD decomposition on a recurrent neural network needing to be compressed; s2, training the network after SVD decomposition again, and removing redundant connection step by step; s3, respectively performing K-means clustering on theweights of the remaining connections of each layer of the network; s4, training the clustered network again; and S5, encoding and storing the network weight. According to the invention, SVD and redundant network connection removing methods are combined; the number of parameters of the recurrent neural network is effectively reduced, the storage amount of the parameters is greatly reduced throughfurther K-means clustering and encoding storage of a sparse matrix, training of the network is combined in the series of processes, and it is guaranteed that a large compression multiple is achieved under the condition that the influence on the network performance is not large.

Description

technical field [0001] The invention relates to the technical field of deep learning and artificial intelligence, in particular to a compression method for deep cyclic neural networks based on SVD and pruning. Background technique [0002] In recent years, due to the emergence of large-scale data and the development of hardware platforms, deep learning algorithms have achieved a series of amazing results in the field of artificial intelligence. Among them, the proposal of deep recurrent neural network effectively solves the problem of identification and detection of data with time series information, and has a wide range of applications in the fields of language translation, video detection, and online handwritten character recognition. However, while the deep recurrent neural network has good performance, its parameter storage capacity hinders the application of the deep recurrent neural network in embedded devices such as mobile terminals. Therefore, it is of great signif...

AI Technical Summary

Problems solved by technology

However, while the deep recurrent neural network has good performance, its parameter storage hinders the application of the deep recurrent neural network in embedded devices such as mobile terminals.

Images