Compression method and device of neural network model

Abstract

The invention provides a compression method and device of a neural network model. The compression method of the neural network model comprises the following steps: acquiring a parameter matrix in an embedded layer of the neural network model; receiving an order raising instruction, and performing order raising on the parameter matrix according to the order raising instruction to obtain a high-order tensor; decomposing the high-order tensor to obtain a decomposition result of the high-order tensor; and updating the parameter matrix according to the decomposition result to obtain a compressed neural network model. According to the method provided by the invention, on the basis of keeping the performance of the neural network model, the parameter quantity in the embedded layer of the neural network model is effectively reduced, and the volume of the neural network model is remarkably reduced.

Description

technical field [0001] The present application relates to the field of computer technology, and in particular to a neural network model compression method and device, computing equipment, and a computer-readable storage medium. Background technique [0002] With the development of computer technology, neural network models are more and more widely used, such as the Transformer translation model, the Transformer translation model includes an encoder and a decoder, and the encoder and decoder include an embedding layer and 6 networks with the same structure. Layer, that is, the encoder includes 6 encoding layers, and the decoder includes 6 decoding layers. Each encoding layer or decoding layer includes a self-attention layer and a feed-forward network layer, and the serial connection between the 6 encoding layers , serial connection between six decoding layers. [0003] The parameter quantity in the transformer translation model is mainly concentrated in the embedding layer. ...

AI Technical Summary

Problems solved by technology

[0003] The parameter quantity in the transformer translation model is mainly concentrated in the embedding layer. In order to improve the performance of the translation model, the vocabulary size in the embedding layer is mostly tens of thousands. As the vocabulary size increases, the parameter quantity also increases linearly. At the same time, the feedforward network layer is composed of two linear layers, using a bottleneck network structure. The feedforward network layer in the transformer translation model has 25 million parameters, so the parameter quantity in the transformer translation model is huge. At the same time There is also a lot of redundancy, resulting in a huge size of the transformer translation model

Images