Assembly part relative pose estimation monitoring method based on deep learning

Abstract

The invention relates to an assembly part relative pose estimation monitoring method based on deep learning, and the method comprises the following steps: shooting images of a target assembly at different angles through a camera, and building a sample data set through the collected images; performing feature extraction and 3D key point detection on the sample data set through a deep learning network to obtain a 3D key point set of each part in the assembly; carrying out semantic segmentation according to the collected image, and distinguishing different parts in the image; according to the 3D key point set and the point cloud data set of each part, using a least square fitting algorithm to obtain a pose prediction value of each part under a camera coordinate system; selecting a part as a reference system part, establishing a world coordinate system by taking the geometric center of the reference system part as an original point, and calculating a pose true value of the reference system part under a camera coordinate system; and respectively calculating a relative pose relationship between each other part and the reference system part, wherein the relative pose relationship comprises a spatial geometric distance, a relative rotation matrix and a relative angle.

Description

technical field [0001] The invention relates to a method for estimating and monitoring relative poses of assembly parts based on deep learning, which belongs to the technical fields of computer vision and intelligent manufacturing. Background technique [0002] Computer vision is of great significance to the reform and upgrading of intelligent manufacturing, especially the emergence of deep learning networks has promoted the development of modern industry. In traditional manual assembly operations, workers need to compare the assembly process drawings. The assembly information is cumbersome and the degree of visualization is low, resulting in difficulties for workers to understand and low assembly efficiency. At the same time, it is necessary to check the assembly quality of the assembly. The inspection process is complicated and there are many review information records. It is time-consuming and labor-intensive to record mainly in paper documents, and it is easy to cause er...

AI Technical Summary

Problems solved by technology

However, most mechanical parts have no texture, no color, and reflective features, and there are serious occlusions in assembly parts, which bring challenges to the pose estimation of parts.

At present, the pose estimation of mechanical parts is mostly researched on scattered parts, estimating the pose of a single part in the camera coordinate system, lacking overall correlation, and failing to estimate the relative pose between parts of the assembly.

Images