Teleoperation method and system for aerial work based on stereo vision and gesture control

Abstract

The invention discloses a high-altitude operation remote operation method and system based on stereo vision and gesture control. The PC terminal receives the real scene information of the high-altitude operation area collected by the binocular stereo camera, and displays it in a wearable stereoscopic display in real time; according to the wearable The three-dimensional real-scene working conditions displayed on the three-dimensional display, perform hand operations and track the movement information of hands and fingers in real time, and send the tracked hand and finger movement information to the PC terminal, and convert them into motion instructions of bionic manipulators after processing; bionic manipulators Receive and process the converted motion commands, and perform corresponding follow-up to complete the remote operation of high-altitude operations. The remote operation method and system combined with stereo vision and gesture control in the present invention can ensure the personal safety of personnel working at heights, improve the operator's three-dimensional perception of the real scene during remote operation, and at the same time enhance the interactive naturalness and immersion of remote operation .

Description

technical field [0001] The invention belongs to the technical field of high-altitude operation teleoperation, and in particular relates to a high-altitude operation teleoperation method and system based on stereo vision and gesture control. Background technique [0002] In recent years, with the rapid development of economy, there are more and more high-rise buildings in our country. At present, high-altitude operations involve fields such as power maintenance, elevated construction, high-rise equipment installation, and cleaning. Operators are required to climb or stand on very high tower cranes for manual operations, which is very dangerous, and accidents may occur if they are not careful. , It will cause irreparable losses to society and enterprises, so it is especially necessary to replace spiders with robots to complete extremely dangerous high-altitude operations. [0003] Most of the current high-altitude operation robots can only complete some simple and repetitive ...

AI Technical Summary

Problems solved by technology

Most of the existing teleoperation modes are for the operator to understand and judge the working conditions of the high-altitude operation area through the two-dimensional monitoring screen at the operation end. It is difficult for the operator to accurately judge the three-dimensional relationship between the various components at high altitude; Two-dimensional working condition information uses a joystick or a joystick to control the manipulator to complete the job. This interactive mode is blunt, the immersion is poor, and there is a certain rate of misoperation, which is difficult for untrained operators.

[0004] Therefore, there is an urgent need for a high-altitude operation teleoperation system and method with a strong sense of immersion and natural interaction to solve the problems of lack of three-dimensional information in traditional teleoperation methods, difficulty in using interactive devices, and weak immersion.

Images