Human-machine cooperative picking robot based on Leap Motion and machine vision and control method thereof

Abstract

The invention discloses a human-machine cooperative picking robot based on Leap Motion and machine vision and a control method thereof. The human-machine cooperative picking robot comprises a walkingmechanism and a picking mechanical arm. A power source, a picking collecting basket and a robot controller are fixedly arranged on the walking mechanism, and the picking mechanical arm is arranged onthe walking mechanism. A picking mechanical hand is arranged on the picking mechanical arm. The human-machine cooperative picking robot also comprises a remote control system, and the remote control system comprises a computer and a Leap Motion somatosensory sensor, the Leap Motion somatosensory sensor communicates with the computer, and the computer communicates with the robot controller. A camera is arranged on the picking mechanical hand and communicates with the robot controller, and the camera and the robot controller are electrically connected with the power source. The human-machine cooperative picking robot can recognize objects to be picked in a complex background environment by using the excellent visual system of a human and is remotely controlled to move to pick fruits by the non-contact somatosensory sensor.

Description

technical field [0001] The invention relates to the technical field of modern agricultural equipment, in particular to a human-machine collaborative picking robot based on Leap Motion and machine vision and a control method thereof. Background technique [0002] In the fruit planting industry, fruit picking and harvesting account for about 50% of the workload of the entire industry. Due to the complexity of the fruit picking process, most fruit industry areas still use manual work when picking fruits; of fruits have been partially mechanized harvesting operations, but the level of automation is still low. Intelligent fruit picking is the combination of intelligence and automation technology to achieve intelligent fruit picking, thereby reducing the amount of manual labor and improving work efficiency. Among them, the identification and positioning of fruits are mainly based on machine vision technology. [0003] In the field of intelligent fruit picking technology, the pate...

AI Technical Summary

Problems solved by technology

[0004] According to the fruit positioning information of the vision system, the above-mentioned patent automatically controls the walking mechanism to adjust the position and the picking of the picking manipulator. Although the intelligent picking of the fruit is realized, the following problems still exist: due to the randomness and complexity of fruit growth on the fruit tree, that is, The fruit may be a single fruit, or it may be clustered. According to the machine vision recognition algorithm of the above patent, it is still difficult to distinguish the fruits in complex backgrounds such as leaves occluding fruits, branches occluding fruits, and fruits occluding or overlapping each other.

Images