Autonomous system sensing method and system based on multi-modal fusion

Abstract

The invention discloses an autonomous system sensing method and system based on multi-modal fusion, and the method comprises the steps: obtaining data collected by sensors of a plurality of vehicles, processing the data collected by the sensors through employing a trained multi-modal data fusion frame, fusing the processed results, obtaining a primary fusion result, processing the primary fusion result based on a target recognition service, retaining region-of-interest information, constructing a cooperation-level fusion architecture by using an existing multi-view fusion algorithm, performing secondary fusion on the region-of-interest information uploaded by a plurality of vehicles to obtain a secondary fusion result, and issuing the secondary fusion result to the vehicles so that the vehicles can make control decisions according to the secondary fusion result. According to the method and system, the vehicle cascade cooperation data fusion framework is constructed, the environment view can be improved and expanded through the cascade framework, the view limitation of a single automatic driving vehicle is solved, the uncertainty is reduced, and the performance of a sensing algorithm is improved.

Description

technical field [0001] The invention relates to the technical field of automobile automatic driving, in particular to an autonomous system perception method and system based on multimodal fusion. Background technique [0002] With the rapid development of sensor technology, self-driving cars are becoming more and more popular. In the process of self-driving vehicles perceiving the surrounding environment, it is necessary to solve the limitation of the field of view of a single self-driving vehicle under the requirement of ensuring its real-time performance and perception accuracy. , one of the basic challenges of computer vision for self-driving cars is to achieve high-precision various applications under different road, weather, lighting and working conditions, but the two-dimensional computer vision based on existing cameras is greatly affected by environmental conditions, It is easy to cause unstable working status of related applications of self-driving cars. [0003] T...

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present application provides an autonomous system perception method and system based on multi-modal fusion, aiming to solve the problem of unstable working status of the existing perception system

Images