Method for guiding video coding by utilizing scene semantic segmentation result

Abstract

The invention provides a method for guiding video coding by utilizing a semantic segmentation result. Firstly, on the basis of an image semantic segmentation result, inter-frame optical flow estimation is combined, semantic segmentation of a video flow sequence is achieved in a mode that a plurality of flow propagation gating circulation units are connected in series, and the precision and speed of video semantic segmentation are improved. Furthermore, the video semantic segmentation result is applied to adaptive video coding under the guidance of scene content classification;, the target category and motion characteristics in the video content can be effectively utilized to realize self-adaptive quantization, low-rate compression is carried out on key objects and motion targets, high-ratecompression is carried out on non-key objects, the storage consumption and bandwidth occupation of the video are reduced, and effective reference is provided for video coding compression, especiallyapplication in the monitoring field.

Description

technical field [0001] The present application relates to a method for guiding video encoding by using scene semantic segmentation results, which belongs to the technical field of computer vision and video image processing. Background technique [0002] Currently, video capture devices are widely used, and the amount of data obtained is very large. According to Cisco statistics, taking Internet data as an example, video content accounts for about 90% of the total Internet traffic. In addition, due to the proliferation of mobile devices, in the rapidly developing mobile network, the proportion of video traffic is as high as 66%, and it is growing at a compound annual growth rate of more than 92%. Not to mention the surveillance cameras that can be seen everywhere in the construction of a safe city, and a large number of videos have been shot under 24-hour uninterrupted surveillance. It can be seen that video data occupies a dominant position in big data. On the one hand, th...

AI Technical Summary

Problems solved by technology

Traditional predictive / transform coding faces bottleneck problems: firstly, predictive coding usually adopts the simplest translational motion prediction model, and for sequences with complex motions, the prediction accuracy drops sharply; secondly, the adaptive degree of image content is not high, which limits Improvement of Coding Efficiency

Although content-based video coding distinguishes various regions of the image from the perspectives of background / foreground, interested objects / non-interested objects, etc., guides video coding with regional semantic differences, and considers the correlation between video frames and frames from a higher level , but there are still some problems in content-based video coding and compression at present, which cannot be practical: 1) Traditional image content segmentation, using middle and low-level information such as edges and color blocks in the image, cannot obtain accurate semantics. Although the deep network can learn through training More content, but the target scale and positioning are still not accurate enough, and the calculation complexity is high; 2) The existing video semantic segmentation methods often do not make full use of the inter-frame information, and some methods even directly perform image semantic segmentation on each frame of the video, which will As the result of video segmentation, its accuracy is not high, and real-time performance cannot be guaranteed; 3) The encoding technology is seriously disconnected from the processing and analysis of video content, and there is no good idea on how to integrate video content and encoding schemes. For example, video content contains Different semantic objects have different motion characteristics. These differences have a great impact on the coding accuracy and speed. Existing methods fail to make full use of the deeper content information of these video sequences.

Images