Real-time plane object detection method based on maskrcnn

Abstract

The embodiment of the present invention provides a real-time object detection method on a plane based on MaskRCNN, which is characterized in that it includes: step 1 acquiring each frame of image in the video stream, calculating the pose of the frame based on ORBSLAM2, and saving the pose of the frame and the corresponding image into the global array; step 2 increases the deep learning detection thread based on ORBSLAM2, and the deep learning thread extracts data from the global array, extracts two adjacent frames of images in the array, and calculates the adjacent two frames through the pose and pose respectively The projection map, the pixels contained in the object on the second frame of the projection map are detected by MaskRCNN, and the translation relationship of the feature points can be obtained according to the feature matching of two adjacent frames of the projection map, so as to obtain the pixel points of the object on the first frame of the projection map, And the pixel points of the two frames of images are reverse-transformed according to the pose, and the matching points after the inverse transformation are calculated by triangulation to calculate the world coordinates of the object; step 3 is calculated according to the pose of the current frame and the world coordinates of the object If the pixel coordinates of the current frame of the object and the camera pose satisfy the plane, the detected object will not be rendered, only the non-detected object will be rendered, and the AR object will be inserted on the detected object.

Description

technical field [0001] The invention relates to the technical field of image recognition, in particular to a method for detecting objects on a real-time plane based on MaskRCNN. Background technique [0002] With the rapid development of science and technology, all possible fantasies are gradually becoming reality. The cool movie "Iron Man" has brought us a strong visual shock, which uses augmented reality technology to realize Iron Man's battle service operation. Initially, augmented reality was achieved by pasting logos in reality, reading each frame of the video stream in real time and making similarity judgments with the logos, but in many scenarios it is not possible to paste logos, so augmented reality is realized by recognizing natural features The system has become the focus of research. [0003] There are many natural features, and the plane is the most common and easy-to-use structure. When the plane is recognized, the virtual object can be rendered on the plane ...

AI Technical Summary

Problems solved by technology

Deep learning is more robust than traditional algorithms. There are many algorithms for instance segmentation. Among them, MaskRCNN is an algorithm proposed by He Kaiming's team. This algorithm has a better recognition effect, but it takes about 2 seconds to recognize a frame, which makes deep learning high accuracy. Advantages are difficult to apply to real-time systems

Images