Weakly supervised rpa element recognition method and system based on deep learning

Abstract

The invention belongs to the technical field of RPA element recognition, and in particular relates to a method and system for weakly supervised RPA element recognition based on deep learning. Including steps: S1, for supervised data, sample several element image samples from samples of each category without replacement; for unsupervised data, randomly sample several element images from samples of each category without replacement sample; S2, perform multiple data enhancement processing on each element image sample to obtain multiple processed element image samples; S3, extract the features of the obtained element image samples, and perform registration learning, metric learning, representation learning, The features of element image samples are identified by means of self-supervised learning and clustering learning. The present invention has the characteristics of being able to learn difficult-to-segment samples based on a small amount of manually labeled data, and combining the distribution rules of large element data to improve the generalization ability and realize efficient and robust element recognition functions.

Description

technical field [0001] The invention belongs to the technical field of RPA element identification, in particular to a weakly supervised RPA element identification method and system based on deep learning. Background technique [0002] RPA (Robotic Process Automation) is a rapidly developing computer software automation technology. Element picking is an important component of RPA, including element detection and identification. Common operations for identifying elements are categorization, matching, and identifying content within elements (such as text elements). Element classification is the process of classifying elements into different categories according to specific classification rules; element matching is the process of finding the same or similar elements for a given template element, or finding the same or similar template for a given element (at this time, it is equivalent to classifying the elements); the content in the identification element is mainly for text e...

AI Technical Summary

Problems solved by technology

[0010] The purpose of the present invention is to overcome the problem that in the prior art, the existing RPA element identification method needs to pay more labor costs, and the identification is not accurate, unstable, and poor in robustness, so that the RPA operation results have greater uncertainty. , provides a weakly supervised RPA element recognition method based on deep learning that can learn difficult samples based on a small amount of manually labeled data, and combine the distribution rules of element big data to improve generalization ability and realize efficient and robust element recognition function and system

Images