Neural network model compression method, corpus translation method and devices thereof

Abstract

The invention relates to a neural network model compression method, a corpus translation method and devices thereof. The neural network model compression method comprises the steps of acquiring a training sample set comprising a plurality of training sample pairs, wherein each training sample pair comprises source data and target data corresponding to the source data; training an original teachermodel by taking the source data as input and taking the target data as verification; training an intermediate teacher model based on the training sample set and the original teacher model, and forminga teacher model set by one or more intermediate teacher models; training a plurality of candidate student models based on the training sample set, the original teacher model and the teacher model set, and forming a student model set by the plurality of candidate student models; and according to the training results of the plurality of candidate student models, selecting one candidate student model as a target student model. By introducing a plurality of teacher models, multistage guidance of student model training is realized, so that the quality of the student model is improved.

Description

Technical field [0001] The present disclosure relates to the field of machine translation, and in particular to neural network model compression methods, corpus translation methods and devices for machine translation, as well as electronic devices and computer-readable storage media. Background technique [0002] With the development of artificial intelligence technology, deep learning based on neural networks has achieved good performance on multiple machine learning tasks such as image classification, machine translation, and speech recognition. [0003] At present, in order to obtain the best network model output results, a deep neural network model containing multiple network layers and ultra-large-scale parameters is usually used. Although this complex network structure significantly improves the output results of the model, it also makes it difficult to Deployment on mobile devices with small storage space, on the other hand, also leads to excessive inference delays on low-po...

AI Technical Summary

Problems solved by technology

[0005] However, in the process of knowledge distillation training, a teacher model is used to guide a student model, which is a single-stage knowledge distillation method. Since the prediction accuracy of the teacher model is generally much better than that of the student model, it is difficult to integrate the knowledge of the teacher model through single-stage knowledge distillation. Efficient transfer to the student model, so it is difficult to obtain a neural network student model with better prediction accuracy for mobile deployment

Images