Outdoor scene illumination parameter restoration device and working method thereof

Abstract

The invention discloses an outdoor scene illumination parameter restoration device and a working method thereof. The device comprises a computer and a video camera. The basic working method comprises the following steps: the image or video stream which is shot by the video camera is input to the computer and processed by the computer, and the restoration result of the illumination parameters of the scene is output. The process is divided into a preprocessing phase and an on-line phase. The device has the working principle that any outdoor scene image can be presented as a linear combination of a sunlight and skylight base image, so that the problem of the restoration of the incident light brightness values of the sunlight and the skylight in the image can be included as a problem of optimizing the real-time solving. The method does not need three-dimensional geometric information and material information of the scene, and can acquire the incident light brightness values of the sunlight and the skylight in the outdoor scene in real time. The method can be directly used in an augmented reality system, and can also be used for outdoor scene video sequence preprocessing, such as image segmentation, pattern recognition and the like.

Description

technical field [0001] The present invention relates to outdoor scene lighting parameter recovery, in particular to an outdoor scene lighting parameter recovery device and a working method thereof. Background technique [0002] Synthesis of a computer-generated virtual object into a photo of a real scene was first proposed by E. Nakamae in 1986 [NHIN86]. The author establishes equations through the shadow points and non-shadow points of some diffuse materials in the user's simple interaction scene to solve the values ​​of the red, green, and blue channels of the sun light and sky light, and then use the restored two light sources to draw Virtual objects, so that the lighting of virtual objects and real photos looks consistent. This method is still operable for the lighting restoration of one or several frames of images, but it cannot be interactively operated for online video streams, so it cannot be processed at all. Since then, since the appearance of the scene is largel...

AI Technical Summary

Problems solved by technology

However, this method is not suitable for restoring the lighting environment of outdoor scenes

There are two reasons: 1) In outdoor lighting, the intensity of sunlight in sunny weather is very large. For existing cameras, even if the minimum exposure speed is used, the reflected light of the sun on the mirror ball will still be oversaturated and difficult to recover. The correct brightness of the sun light, causing the light restoration to fail

2) In outdoor scenes, there are often clouds drifting by, or there is a breeze blowing, so it is difficult to ensure that the pixels of multiple photos taken with different exposures are completely still or completely aligned, so the obtained high dynamic range images are often inconsistent. precise

However, the existing indoor lighting algorithms generally require the three-dimensional representation of the known scene, so the indoor lighting analysis method is not suitable for the lighting analysis of the outdoor scene.

(2) Lighting complexity

Natural light sources (sun light and sky light) in outdoor scenes have some special properties. For example, the lighting of outdoor scenes is greatly affected by clouds and weather conditions, and cannot be controlled artificially. Due to the complex structure of the scene, it is directly based on the accurate lighting model It is difficult to reversely calculate the lighting parameters, and it is even more difficult to meet the real-time requirements

Images