Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Natural Language-Based Auxiliary Programming Method for Industrial Robots

An industrial robot and natural language technology, applied in the field of robot programming, can solve the problems of not being able to meet the needs of industrial intelligent manufacturing, not providing source codes for industrial engineers, and unfavorable reuse of similar codes, so as to improve development efficiency and simplify programming complexity , the effect of simplifying the development burden

Active Publication Date: 2021-03-02
HANGZHOU DIANZI UNIV
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, such methods only output the behavior and state of the robot, and do not provide the source code needed by industrial engineers
This programming method cannot be modified offline at the code level when the scheme needs to be adjusted in industrial production
Because no code text is generated, it is not conducive to the reuse of similar code in other projects
[0007] In this case, the existing programming technology cannot meet the needs of industrial intelligent manufacturing due to its inherent defects

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Natural Language-Based Auxiliary Programming Method for Industrial Robots
  • Natural Language-Based Auxiliary Programming Method for Industrial Robots
  • Natural Language-Based Auxiliary Programming Method for Industrial Robots

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] The present invention includes three subtasks, and the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0024] figure 1 It is a structural schematic diagram of the overall model of the present invention.

[0025] The inventive method is divided into three connected subtasks, and the concrete steps are as follows:

[0026] Task 1. Identify the target object

[0027] Step (1), preprocessing the input language instruction and environment image. The preprocessing includes using Bi-RNN with LSTM to extract the language features of language instructions and using F-RCNN to preprocess the environment image, so as to obtain target candidate region features. Specific steps are as follows:

[0028] 1.1 Instruction encoding: the instruction I composed of i words i ={x 1 , x 2 , x 3 ,...,x i} Enter the RNN network. Recursively generate hidden state sequence I by encoding language instructions via Bi-RNN wit...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an auxiliary programming method for industrial robots based on natural language, which generates corresponding robot execution codes according to language instructions and environment images. The present invention is divided into three parts: 1) using bidirectional recurrent neural network (Bi-RNN) with long short-term memory (LSTM) and fast regional convolutional neural network (F-RCNN) to extract language instructions and characteristics of factory environment respectively. 2) Propose a "multi-attention mechanism" model and machine translation alignment algorithm to correctly match objects in the environment with instructions, thereby identifying the specified object and outputting the coordinate point where the object is placed. 3) Use the output results of the above model and the CoBlox modular programming method to generate the robot code for the operation. The "multi-attention mechanism" model adopted by the invention improves the recognition accuracy and solves the problem that the current method cannot accurately recognize objects in the industrial environment. The modular programming technology solution simplifies the programming complexity of engineers and effectively improves development efficiency.

Description

technical field [0001] The application belongs to the technical field of robot programming, and in particular relates to the robot programming technology based on natural language and machine vision. Background technique [0002] With the rapid development of robot technology in recent decades, the concept of intelligent manufacturing has been deeply rooted in the hearts of the people. Robot arm technology has been widely used in industrial production environments. Collaborative robots combine the advantages of humans and mechanical equipment, and work closely with workers on the production line, which can significantly improve production efficiency. [0003] At present, all mechanical tasks must be carefully designed and coded by engineers in order to assist and replace workers to perform a single mechanical task. Engineers usually use online or offline programming methods to write robot codes. However, these programming methods are too time-consuming, and the timeliness i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): B25J9/16
CPCB25J9/1658
Inventor 胡海洋刘翰文陈洁李忠金黄彬彬
Owner HANGZHOU DIANZI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products