Multi-sensor video fusion and de-noising method and device

Abstract

The invention discloses a multi-sensor video fusion and de-noising method and device. The method comprises the steps of converting the input video frame to a frequency domain, and separately generating a low-frequency sub-band coefficient and a high-frequency sub-band coefficient; separately fusing the low-frequency sub-band coefficient and the high-frequency sub-band coefficient; and performing inverse transformation on the fused low-frequency sub-band coefficient and high-frequency sub-band coefficient to obtain fused video. Therefore, the multi-sensor video fusion and de-noising method and device can realize multi-sensor video trans-scale fusion and de-noising by video frame decomposition based on three-dimensional shearlet transformation, high-frequency coefficient fusion strategy based on 3D PCNN (Pulse Coupled Neural Network) and low-frequency coefficient fusion strategy based on remarkable 3D PCNN.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to a method and device for multi-sensor video fusion and noise reduction. Background technique [0002] In order to obtain the complete information of the scene, the content of the same scene needs to be captured by multiple different sensors at the same time. In order to fully and efficiently utilize these video contents captured from multiple sensors, multiple different video contents need to be combined into one video sequence, which is achieved using image sequence fusion methods. Image sequence fusion can synthesize multiple image sequences from different sensors into a single image sequence. The synthesized image sequence contains all the important information in the original image, eliminating redundancy and improving the usability of information. The external environment where the moving target is located is complex and changeable. During the process of capturing i...

AI Technical Summary

Problems solved by technology

The external environment where the moving target is located is complex and changeable. During the process of capturing images by the sensor, it is often affected by interference factors such as noise. When the image is collected, interference information is also introduced, which reduces the quality of the image captured by the sensor. The visual quality is degraded, so it is necessary to eliminate noise and other interference information at the same time during the fusion process

[0003] The traditional fusion method is a direct weighted fusion of the original image, which easily leads to a decrease in contrast and the introduction of interference

Fusion methods based on multi-scale decomposition, image fusion based on high-order singular value decomposition, and image fusion based on Markov random fields implemented in the transform domain can ensure the quality of single-frame images through frame-by-frame fusion, but they are very difficult to achieve. It is difficult to maintain the temporal consistency and stability of the image sequence, and satisfactory results cannot be obtained due to the insufficient ability to represent motion information

[0004] Most of the existing image fusion methods still only focus on the combination of useful pixels in the original image, and rarely consider the processing of noise and other interference information.

Noise is easily introduced into the fused image together with useful pixels if there is no further processing to remove noise and other interfering information

Inability to ensure temporal stability and consistency of image sequences while improving image quality

Images