A method and system for batch corpus generation

Abstract

A method and system for batch generation of corpus, the method includes the following steps: S1, setting a scene of corpus application; S2, setting an intention database for the scene, wherein the intention database contains at least one intention corresponding to the scene; S3, setting a situation database and a sentence pattern database, wherein the situation database comprises at least one situation, and the sentence pattern database comprises at least one sentence pattern, and at least one sentence pattern corresponding to each situation is respectively set; S4: selecting at least one situation corresponding to each intention; S5: generating a plurality of phrases required for a complete sentence pattern for each sentence pattern setting in each situation according to the scene, intention and situation to which the sentence pattern belongs; S6, applying a plurality of phrases to corresponding sentence patterns to obtain a primary selected corpus; S7: Screening the primary corpus and selecting the high-quality corpus. The corpus generated by the invention is rich in data, complete and has good scene reusability, can save a lot of manpower and time, and has strong practicability.

Description

technical field [0001] The present invention relates to the field of natural language generation, in particular to a method and system for generating corpus in batches. Background technique [0002] In recent years, with the development of the information age and the Internet age, the number of users and the penetration rate of e-commerce platforms have grown rapidly. In e-commerce platforms, artificial customer service is generally set up on the enterprise side to deal with user inquiries and after-sales services. Tracking and other requirements. Due to the rapid increase in the number of e-commerce platform users, the user needs to be dealt with have also increased rapidly. At the same time, the cost of traditional manual customer service services has continued to rise, making it difficult for manual customer service to cope with the huge service demand. Therefore, in order to meet the increasing service needs of users and improve user experience, it has become the active...

AI Technical Summary

Problems solved by technology

[0005] (1) The stages of data collection, statistics and labeling all take considerable manpower and time, and the efficiency is low;

[0006] (2) The corpus data obtained from the above data source channels are not rich and complete, and cannot cover all possible corpus;

[0007] (3) For different application scenarios, data collection, statistics and labeling stages need to be carried out separately, and the reusability of scenarios is poor

Images