Crowdsourcing system-oriented simulation test case generation method and device

Abstract

The invention discloses a crowdsourcing system-oriented simulation test case generation method and device, and the method comprises the steps: firstly inputting a sample obtained by a crowdsourcing platform into a VAE encoder, and learning a mean value and a variance of worker capability and task difficulty labeled for a real sample, namely obtaining the normal distribution of the corresponding worker capability and task difficulty; meanwhile, generating Gaussian mixture distribution of the capabilities and task difficulties of all workers participating in labeling; secondly, obtaining hidden variables of the VAE by using a reconstruction technique, inputting the hidden variables into a decoder of the VAE, and training the decoder by using gradient descent; and finally, inputting the worker ability and task difficulty randomly generated by Gaussian mixture distribution into a trained decoder, and generating a large number of crowdsourcing test cases which are independent and identically distributed with a real sample. According to the method, the neural network is combined to solve the problem that the crowdsourcing test cases are few and the coverage is narrow due to the cost, so that the result convergence method is difficult to comprehensively evaluate, and the crowdsourcing result convergence method can be comprehensively and effectively evaluated.

Description

technical field [0001] The invention uses a deep learning method to solve problems in the field of crowdsourcing, and in particular relates to a simulation test case generation method and device for a crowdsourcing system. Background technique [0002] Crowdsourcing is an emerging business model that utilizes distributed computing under the current Internet booming environment. It generally consists of three parts, the crowdsourcing platform, the publisher of the crowdsourcing task, and the workers participating in the crowdsourcing task. And after a certain cost, the crowdsourcing task is distributed redundantly in the crowdsourcing platform, and Internet users are hired to perform redundant processing on the task, and the redundant data is aggregated by the result aggregation method to generate high-quality data annotation. In this type of system development and testing phase, the test evaluation is mainly carried out by means of trial operation. However, due to the high...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the few data sets in the current crowdsourcing field, the high cost of collecting data, and narrow coverage, the present invention provides a simulation test case generation method and device for crowdsourcing systems, which are used to generate similar and identical real crowdsourcing annotation tests. A large number of simulated crowdsourced test cases for set functions

Images