Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Retraining robotic vision models for robotic process automation

A retraining and model technology, applied in computing models, machine learning, biological neural network models, etc.

Pending Publication Date: 2021-01-19
UIPATH INC
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Certain embodiments of the present invention may provide solutions to problems and needs in the art that have not been fully identified, understood, or addressed by existing CV modeling techniques

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Retraining robotic vision models for robotic process automation
  • Retraining robotic vision models for robotic process automation
  • Retraining robotic vision models for robotic process automation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] Some embodiments relate to identifying misidentified or unidentified graphical components, and retraining the CV model for RPA generated by the ML system for more accurate computer image analysis. A screenshot image of the visual display may be captured, including the graphical user interface (GUI) of the application to be automated. In a virtual machine (VM) embodiment (such as remote desktop, etc.), or in some Flash, Silverlight or PDF documents, images can only be displayed for a given application. Images may include windows, documents, financial receipts, invoices, and / or any other graphical elements without departing from the scope of the present invention. While in some embodiments images may include unstructured data, in some embodiments the data is structured.

[0027] The CV model can then be executed on the screenshot image (potentially combined with a text recognition model from OCR), and the specific graphical parts identified in the image can be provided...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Embodiments of the present disclosure relate to retraining robotic vision models for robotic process automation. A Computer Vision (CV) model generated by a Machine Learning (ML) system may be retrained for more accurate computer image analysis in Robotic Process Automation (RPA). A designer application may receive a selection of a misidentified or non-identified graphical component in an image form a user, determine representative data of an area of the image that includes the selection, and transmit the representative data and the image to an image database. A reviewer may execute the CV model, or cause the CV model to be executed, to confirm that the error exists, and if so, send the image and a correct label to an ML system for retraining. While the CV model is being retrained, an alternative image recognition model may be used to identify the misidentified or non-identified graphical component.

Description

technical field [0001] The present invention relates generally to robotic process automation (RPA), and more particularly to identifying misidentified or unidentified graphical parts, and retraining computer vision (CV) for RPA generated by a machine learning (ML) system. ) model for more accurate computer image analysis. Background technique [0002] Currently, training data is obtained by generating synthetic data and collecting screenshots (i.e., digital images) of actual user interfaces of various software applications (whether from real-time applications or the Internet) , Algorithmic automation of CV models generated by ML for RPA. Synthetic data is data produced for the specific purpose of training ML models. This is different from "real" or "organic" data, which is data that already exists and only needs to be collected and labeled. In this case, organic data includes screenshots collected through various mechanisms and tagged. [0003] Another source of training...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F30/20G06K9/62G06F30/27G06N3/08G06F9/451G06V30/40
CPCG06F30/20G06F30/27G06N3/08G06F9/452G06F18/214G06V30/40G06T7/11G06T1/60G06F3/14G06N20/00G06T2207/20081G06V10/22B25J9/163B25J9/1697G06F18/2178G06F18/40G06V30/10
Inventor C·沃伊库
Owner UIPATH INC
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com