Data annotation management method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a data annotation management method and device, electronic equipment and a readable storage medium. The method comprises the steps: obtaining a reference annotation data set according to to-be-annotated data corresponding to a to-be-annotated task and historical annotation behavior data corresponding to a target annotator; obtaining a first annotation result of the target annotator for the evaluation annotation data and first reference annotation data distributed in the evaluation annotation data, the evaluation annotation data being a part of the to-be-annotated data and having a correct annotation answer, and the first reference annotation data belonging to a reference annotation data set; If the accuracy corresponding to the first annotation result is greater than or equal to a preset accuracy threshold, determining whether to allow the target annotator to continue to execute the to-be-annotated task or not according to a second annotation result of the target annotator for second reference annotation data already distributed in the to-be-annotated data, the second reference annotation data belonging to a reference annotation data set. According to the embodiment of the invention, the quality and efficiency of data annotation can be improved.

Description

technical field [0001] The invention relates to the field of automated testing, in particular to a data label management method and device, electronic equipment and a readable storage medium. Background technique [0002] Artificial intelligence (AI, Artificial Intelligence), also known as omniscience and machine intelligence, refers to the intelligence exhibited by machines manufactured by humans. Today, with the rapid development of artificial intelligence, machines already have the capabilities of human vision, hearing, and semantic recognition. [0003] If you want to make machines more and more "smart", you need a large amount of high-quality human-labeled data to improve the accuracy of machine learning. Moreover, with the widespread application and popularization of artificial intelligence and the deepening of machine learning, the demand for high-quality labeled data is also increasing. Therefore, a large number of annotators are required to use the management plat...

AI Technical Summary

Problems solved by technology

[0006] Embodiments of the present invention provide a data labeling management method and device, electronic equipment, and a readable storage medium, aiming to solve the problem of low data labeling quality and speed caused by the uneven professional level of labelers

Images