No-reference video quality evaluation method based on short-time space-time fusion network and long-time sequence fusion network

Abstract

The invention discloses a non-reference video quality evaluation method based on a short-time space-time fusion network and a long-time sequence fusion network. Quality prediction of video frames is realized through two networks for calculating different time lengths in sequence. And the short-time space-time fusion network is used for extracting and fusing the space-time characteristics of the current frame to obtain a result after the time characteristics in the current frame act on the space characteristics. And the long-time sequence fusion network is used for modeling a result of interaction between frames within a period of time and predicting a quality score of the video under the guidance of priori knowledge. According to the method, video frames serve as input, a network is designed on the frame level to fuse time and space features, the inter-frame relation is considered on the sequence level to further refine the features of the current frame, the quality of each frame in a video is predicted through deep learning, and finally the task of evaluating the overall quality of the video is completed. According to the method, the characteristics of the video frames are refined and enriched, so the model performance is remarkably improved.

Description

technical field [0001] The invention relates to a non-reference video quality evaluation method based on a short-time spatio-temporal fusion network and a long-time sequence fusion network, and belongs to the technical field of digital video processing. Background technique [0002] As a complex source of visual information, video contains a lot of valuable information. The quality of video directly affects people's subjective feelings and information acquisition, and can guide other video tasks such as related equipment development, system monitoring, and quality restoration. Research on Video Quality Assessment (VQA) has also been carried out in recent years. years have received extensive attention. [0003] Video quality evaluation methods can be divided into subjective evaluation methods and objective evaluation methods. Subjective evaluation is the subjective evaluation of video quality by observers. Although the evaluation results are in line with people's subjective...

AI Technical Summary

Problems solved by technology

[0011] Aiming at the poor performance of no-reference video quality evaluation in the existing video quality evaluation, the present invention proposes an objective no-reference quality evaluation method. The present invention divides the video into video frames, and each video frame is obtained through a short-time spatio-temporal fusion network The 64-dimensional feature vector of each video frame and the preliminary predicted quality score are then combined into a feature sequence according to the time sequence of the feature vector, and the preliminary predicted quality score is converted into an impact factor between frames under the guidance of prior knowledge. Then use them as the input of the long-term sequence fusion network to get the overall quality score of the video and complete the quality evaluation process

Images