Mechanical arm vision grabbing system and method based on self-supervised-learning neural network

Abstract

The invention provides a mechanical arm vision grabbing system based on the self-supervised-learning neural network. A field depth camera, a living example dividing module, a posture estimation neuralnetwork module, a three-dimensional posture obtaining module and a mechanical arm control module are included. The field depth camera outputs colorful images to the living example dividing module andoutputs field depth images to the three-dimensional posture obtaining module. The living example dividing module inputs at least one of classification, frame body or dividing information into the posture estimation neural network module. The posture estimation neural network module then outputs the plane posture to the three-dimensional posture obtaining module, the plane posture is fused with the field depth image, and a three-dimensional posture is obtained. The mechanical arm control module obtains the three-dimensional posture and achieves mechanical arm grabbing operation according to the three-dimensional posture. By means of the mechanical arm vision grabbing system, the rough outline position of an object needing to be grabbed can be recognized, and further accurate grabbing position and grabbing angle calculation can be achieved through a neural network with a full connection layer.

Description

technical field [0001] The present disclosure relates to the field of robotic arm technology, in particular to a robotic arm visual grasping system and method based on a self-supervised learning neural network. Background technique [0002] The robotic arm is an automatic operating device that can imitate certain movement functions of the human hand and arm to grab, carry objects or operate tools according to a fixed program. The mechanical arm can replace the heavy labor of people to realize the mechanization and automation of production, and can operate in a harmful environment to protect personal safety, so it is widely used in machinery manufacturing, metallurgy, light industry and atomic energy and other fields. [0003] In industry, the grasping operation of manipulators mostly adopts the traditional teaching method. However, for a new operating object or a new operating environment, it is necessary to re-teach the manipulator manually. With the development and appli...

AI Technical Summary

Problems solved by technology

Therefore, this type of method can improve the grasping of objects with missing textures, but it is prone to misestimation for objects whose outlines are partially occluded

In addition, existing methods are only based on machine vision, but do not take into account the actual mass distribution of the grasped object, which is an important factor in actual grasping

Images