MaskRCNN-based real-time on-plane object detection method

Abstract

The embodiment of the invention provides a MaskRCNN-based real-time on-plane object detection method, which is characterized by comprising the following steps of: 1, acquiring each frame of image in avideo stream, calculating the pose of the frame based on ORBSLAM2, and storing the pose of the frame and the corresponding image into a global array; 2, adding a deep learning detection thread basedon ORBSLAM2, the deep learning thread extracting data from a global array; extracting two adjacent frames of images in the array; calculating projection images of two adjacent frames through poses respectively; detecting pixel points contained in an object on the second frame of projection image through a MaskRCNN; according to feature matching of the two adjacent frames of projection images, obtaining a translation relation of feature points so as to obtain pixel points of an object on the first frame of projection image, carrying out inverse transformation on the pixel points of the two frames of images according to poses, and carrying out triangularization on matching points after inverse transformation so as to calculate world coordinates of the object; and step 3, calculating the pixel coordinates of the current frame of the object according to the pose of the current frame and the world coordinates of the object, and if the pose of the camera meets a plane, not rendering the detected object, only rendering a non-detected object, and inserting an AR object into the detected object.

Description

technical field [0001] The invention relates to the technical field of image recognition, in particular to a method for detecting objects on a real-time plane based on MaskRCNN. Background technique [0002] With the rapid development of science and technology, all possible fantasies are gradually becoming reality. The cool movie "Iron Man" has brought us a strong visual shock, which uses augmented reality technology to realize Iron Man's battle service operation. Initially, augmented reality was achieved by pasting logos in reality, reading each frame of the video stream in real time and making similarity judgments with the logos, but in many scenarios it is not possible to paste logos, so augmented reality is realized by recognizing natural features The system has become the focus of research. [0003] There are many natural features, and the plane is the most common and easy-to-use structure. When the plane is recognized, the virtual object can be rendered on the plane ...

AI Technical Summary

Problems solved by technology

Deep learning is more robust than traditional algorithms. There are many algorithms for instance segmentation. Among them, MaskRCNN is an algorithm proposed by He Kaiming's team. This algorithm has a better recognition effect, but it takes about 2 seconds to recognize a frame, which makes deep learning high accuracy. Advantages are difficult to apply to real-time systems

Images