Processing method and device of lip language recognition model, computer equipment and storage medium

Abstract

The invention relates to a processing method and device of a lip language recognition model, computer equipment and a storage medium. The method relates to a computer vision technology of artificial intelligence, the whole distillation process is divided into a student training stage and a master training stage which are alternately trained, and in the master training stage, a temporary training sample is utilized to update a student model updated by the previous alternate training again, the obtained temporary student model feeds back a current learning state to the master model through a verification sample, and the master model is guided to adaptively adjust teaching knowledge according to the current feedback; and, in addition, the master model is supervised by master training samples, and teaching content is adjusted through master identification loss determined by the master training samples. Then, the student model in a student training stage is trained, and repeated iteration is performed for multiple times to obtain a lip language recognition model according to the student model. According to the scheme, the teaching content can be flexibly adjusted while the knowledge accuracy of master model teaching is improved, and the knowledge distillation effect is improved.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a processing method, device, computer equipment and storage medium of a lip recognition model. Background technique [0002] Lip language recognition aims to predict speech content from silent lip videos or face videos. This visual task usually uses knowledge distillation to allow the student model to learn the ability of lip language recognition from the trained teacher model. [0003] Knowledge distillation can transfer knowledge from the teacher model to the student model. However, the current teacher model is usually a pre-trained model, without considering the current ability of the student model to perform lip recognition tasks. Due to the neglect of the needs of the student model, the teacher model often lacks flexibility when adjusting teaching knowledge. It is impossible to dynamically adjust the teaching content according to the development of the student m...

AI Technical Summary

Problems solved by technology

However, the current teacher model is usually a pre-trained model, without considering the current ability of the student model to perform lip recognition tasks. Due to the neglect of the needs of the student model, the teacher model often lacks flexibility when adjusting teaching knowledge. It is impossible to dynamically adjust the teaching content according to the development of the student model, thus affecting the effect of knowledge distillation

Images