A method for full frame loss error concealment in free viewpoint video

Abstract

The invention discloses a free viewpoint video whole-frame loss error concealment method, which comprises the following steps of: performing bidirectional motion compensation restoration on each image block in a lost frame by fully utilizing a forward reference color image and a backward reference color image in the lost frame time domain according to a prediction relationship of an encoding prediction structure; performing bidirectional parallax compensation restoration on each image block in the lost frame by fully utilizing a left reference color image and a right reference color image in the lost frame viewpoint domain aiming at multiple viewpoints in the free video system; and obtaining a final restoration frame according to the bidirectional motion compensation restoration result and bidirectional parallax compensation restoration result. According to the restoration mode, the reconstruction quality of the image video after error concealment is greatly improved; and moreover, the time domain of the lost frame is restored in a bidirectional mode, bidirectional restoration between viewpoints is performed, the redundant information of lost frame restoration is increased due to the reference information between the bidirectional viewpoints, and the reconstruction quality of the free viewpoint video is greatly improved.

Description

technical field [0001] The invention relates to a video error concealment method, in particular to a free viewpoint video full frame loss error concealment method. Background technique [0002] Since the beginning of this century, with the continuous development of multimedia technology, it is difficult for traditional two-dimensional video systems to meet the needs of users for three-dimensionality and viewpoint interaction when representing natural scenes. The free viewpoint video system can make people freely choose the viewpoint to appreciate the scene and give people an immersive feeling. Therefore, it has very broad application prospects in the fields of film and television entertainment, education, product display, medical care and security monitoring. It is a new generation of multimedia The development direction of the video system. Typically, free viewpoint video systems such as figure 1 As shown, it mainly includes modules such as video capture (video capture), ...

AI Technical Summary

Problems solved by technology

The existing whole frame loss error concealment technology is concentrated in single-channel video and binocular stereoscopic video. Since multi-view video mostly adopts the prediction structure of hierarchical B frame (HBP), its prediction structure determines that it cannot be applied to single-channel video. Or the error concealment technology in binocular stereo video is directly applied to the free viewpoint video system

However, the existing multi-view video error concealment methods mainly focus on the loss of some macroblocks. The main process of recovery is to recover several damaged macroblocks from the reference frame of the lost frame through multiple possible motion vector estimation values. possible restoration blocks, and then use the boundary matching criterion to find the best matching restoration block of the damaged macroblock from these several possible restoration blocks as the final restoration macroblock; however, there is no available Adjacent positively decoded macroblocks for boundary matching

Images