Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Real-time multi-frame bit enhancement method based on content and continuity guidance

A continuity and frame bit technology, applied in the field of neural networks, can solve the problem that the continuity between frames cannot be guaranteed

Active Publication Date: 2020-04-14
TIANJIN UNIV
View PDF2 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In addition, when using independent image-based methods, the continuity between frames cannot be guaranteed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time multi-frame bit enhancement method based on content and continuity guidance
  • Real-time multi-frame bit enhancement method based on content and continuity guidance
  • Real-time multi-frame bit enhancement method based on content and continuity guidance

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0033] The embodiment of the present invention proposes a real-time convolutional neural network based on content and continuity guidance for video bit enhancement, and optimizes the network model through the image content and inter-frame continuity loss function. The method includes the following steps:

[0034] 101: Preprocessing the video sequences in the Sintel database with high-bit lossless image quality, that is, quantizing high-bit images to low-bit images, padding the low-bit images with zeros to obtain zero-filled high-bit images as the training set of the network;

[0035] Wherein, the Sintel database is derived from a short animated film with lossless image quality, which is well known to those skilled in the art.

[0036] Randomly select 1000 image sequences in the Sintel database, each containing five pictures, as the training set, and 50 sets of sequences in the Sintel database other than the training set as the test set.

[0037] 102: The improved SS-VBDE netwo...

Embodiment 2

[0044] The scheme in embodiment 1 is further introduced below, see the following description for details:

[0045] 201: Since the Sintel database is entirely an animated image video generated by computer software, the image sequence has no noise influence, so the image sequence in the Sintel database often has a smoother color gradient structure, and the edges and textures in the image sequence are also clearer.

[0046] This near-ideal structure can help the neural network learn the characteristics of smooth regions and edge structures, and is of great help to the color gradient structure and contour reconstruction in image sequences. Therefore, the deep neural network proposed in this method is trained with Sintel animation images. 50 groups of Sintel database image sequences other than the training set are used as test sets to verify the effect of the present invention.

[0047] In terms of video bit enhancement, it is necessary to fully consider the correlation between con...

Embodiment 3

[0077] Below in conjunction with concrete experimental data, the scheme in embodiment 1 and 2 is carried out effect assessment, see the following description for details:

[0078] 301: Data composition

[0079] The training set consists of 1000 image sequences randomly selected from the Sintel database, each with five images.

[0080] The test set is composed of 50 image sequences randomly selected by Sintel in addition to the training set.

[0081] 302: Evaluation Criteria

[0082] The present invention mainly adopts two kinds of evaluation indicators to evaluate the quality of the reconstructed high-bit image sequence:

[0083] PSNR (Peak Signal to Noise Ratio, Peak Signal to Noise Ratio) is a commonly used and widely used image objective evaluation index. It is based on the error between corresponding pixels and the spatial distance of the entire image. It is an error-sensitive image quality. evaluation index. The larger the PSNR value between two images, the more simil...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a real-time multi-frame bit enhancement method based on content and continuity guidance. The method comprises the steps of quantizing a high-bit image into a low-bit image, carrying out low-bit zero padding of the low-bit image, and obtaining a zero-filled high-bit image as a training set of a network; removing a motion compensation module and a time sequence symmetric sub-network in the SS-VBDE network, and connecting the feature maps, meeting the symmetric positions in space, of a convolution layer and a deconvolution layer to realize spatial symmetric jump connection; performing low-bit zero filling on the low-bit image of the training set to obtain a zero-filled high-bit image as network input; combining the image content loss and the inter-frame continuity lossbetween the multi-frame image sequence generated by the improved network and the real image sequence to serve as a loss function, and training improved network model parameters through an Adam optimizer; and performing low-bit zero filling on the low-bit image of a test set to obtain a zero-filling high-bit image, inputting a high-bit image sequence into the improved SS-VBDE network after the network model parameters are loaded, and outputting a processed high-bit image sequence.

Description

technical field [0001] The invention relates to the field of neural networks, in particular to a real-time multi-frame bit enhancement method based on content and continuity guidance. Background technique [0002] With the rapid improvement of network information transmission capabilities, people's requirements for the quality and perception of multimedia resources are also increasing. Users' expectations for resources such as video and images are not limited to the guarantee of the accuracy of the transmission content, but have begun to pursue the visual effect of the picture. At the same time, bit-depth enhancement (BDE) technology and high dynamic range (high dynamic range, HDR) multimedia resources and display devices have also begun to attract more and more researchers' attention. Normally, the larger the dynamic range of pixel changes in the picture, the more brightness and detail information it can provide, and the more vivid its shading and color performance, which ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T5/00G06T5/50
CPCG06T5/50G06T2207/10016G06T2207/20081G06T2207/20084G06T2207/20221G06T5/90Y02T10/40
Inventor 苏育挺王蒙蒙刘婧
Owner TIANJIN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com