Traditional Chinese medicine literature content analysis method and device

Abstract

The invention discloses a traditional Chinese medicine literature content analysis method and device. The method comprises the following steps: preprocessing an obtained classical Chinese text to obtain unsupervised pre-training data to pre-train a selected large-scale language model Bert; combining the pre-trained model Bert with a conditional random field model to obtain a sequence labeling model; training the obtained sequence labeling model by using the labeled traditional Chinese medicine literature content analysis data; segmenting each paragraph of the to-be-analyzed traditional Chinese medicine literature into clauses, inputting the clauses into the sequence labeling model to obtain a coding sequence of each clause, and generating a probability distribution sequence of a tag to which the corresponding clause belongs according to the coding sequence of the clause; inputting the probability distribution sequence of the clauses into a conditional random field model to obtain the probability that the sequence of the clauses is labeled as different tag sequences; and selecting the tag sequence with the maximum probability as a prediction result, combining adjacent clauses predicted as the same tag, and connecting paragraphs of the literature to obtain a content analysis result of the traditional Chinese medicine literature.

Description

technical field [0001] The invention belongs to the field of natural language sequence labeling and retrieval, and relates to a content analysis method of traditional Chinese medicine literature based on a large-scale classical Chinese pre-trained language model. This method can use a large-scale pre-trained language model and conditional random field to analyze the content of TCM-related texts, segment the text segments and label the attribute labels of each text segment, such as the five luck and six qi of the year, related functions, disease manifestations, Symptoms and diagnosis and treatment methods, etc. Background technique [0002] With the development of machine learning and artificial intelligence technology, machines have achieved excellent performance in many natural language processing tasks. Especially in some repetitive work that requires a lot of manpower, the machine has achieved excellent results and helped people save a lot of time. The machine sequence ...

AI Technical Summary

Problems solved by technology

Sequence labeling tasks are usually cumbersome and require certain professional knowledge, and it is difficult to obtain a large amount of training data for neural network training

Images