Integrated sensor based on microphone array and stereoscopic vision

Abstract

The invention discloses an integrated sensor based on a microphone array and stereoscopic vision. The integrated sensor comprises a hardware system and a control system. The hardware system comprises the microphone array and a stereoscopic vision device. The microphone array is responsible for receiving audio information in an environment and generating multichannel analog audios. The stereoscopic vision device is responsible for collecting environment images. The control system comprises an audio working area, a video working area and a video / audio synthesis working area. The audio working area is responsible for denoising audios, positioning audio sources, and tracking the audio sources (through audio processing). The video working area is responsible for positioning and tracking target audio sources, namely calibrating, positioning and precisely tracking the target audio sources through image processing and image identification. The video / audio synthesis working area is responsible for enhancing audio signals (improving a signal-to-noise ratio of the audio signals) and obtaining and outputting an integrated signal of the video / audio signals.

Description

technical field [0001] The invention mainly relates to an integrated sensor based on a microphone array and stereo vision, in particular to a sound source positioning and tracking technology based on a microphone array and stereo vision, belonging to the field of integrated sensors. Background technique [0002] In today's era, to record events, scenes, etc., audio and video are undoubtedly the most reductive methods; with the rapid development of Internet technology, more and more communication between people is carried out through the Internet, such as video conferencing . However, in the face of the complex sound field environment and the unpredictable position of the sound source, the traditional audio collection and video recording methods are facing major challenges. The sound field contains not only the target sound source, but also other interference sound sources; therefore, the collected sound includes not only the sound of the target sound source, but also enviro...

AI Technical Summary

Problems solved by technology

[0005] The audio processing involved in the above-mentioned patents is used to track the sound source by continuously scanning the space (or calculating the correlation) in continuous time, which has high time and space complexity, high requirements on the processor, and reduces the sound quality. The real-time nature of source tracking increases the interference factor (environmental interference sound source) when operating the entire space, which reduces the accuracy of positioning tracking; moreover, there are large errors in direct use of the sound source coordinates obtained through audio processing, which is not competent High-precision tracking work

Images