A method and system for region extraction of news text based on xlnet

Abstract

The invention discloses a method and system for extracting news text regions based on XLNet. The method includes the following steps: S1. Using the Internet to obtain a large amount of unmarked raw corpus, inputting it into the XLNet pre-training model for pre-training; S2. Pre-training the pre-trained The processed data is input into the pre-trained XLNet pre-training model in step S1 for encoding, the encoded hidden state is input into the BiLSTM+CRF model for identification, and the identified regional entity is output; S3, regional entity disambiguation; S4. Regional entity summary; S5. Regional entity completion operation; the system includes a regional entity recognition module, an entity splicing module, a regional disambiguation module and a regional summary module. The regional entity recognition module is composed of the XLNet pre-training model and the BiLSTM+CRF model . The two-stage training process of the present invention overcomes the problem of inconsistency between the pre-training stage and the training stage in the prior art, solves the pain point that the traditional autoregressive model cannot learn context information at the same time, and realizes complete modeling.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method and system for extracting news text regions based on XLNet. Background technique [0002] The regional attribute of news text contains the location of news events, which is an important reference dimension for statistics and analysis of news events. Therefore, the automatic extraction of news text regions is realized by computer, and the downstream tasks such as recommendation system, public opinion analysis, text summarization are realized. etc. have a very important driving effect. The current mainstream geographic extraction methods include machine learning methods and deep learning methods, both of which require manually annotated geographic entity datasets for training. [0003] The BERT+BiLSTM+CRF model is one of the deep learning methods (two-way conversion coding pre-training model + two-way long short-term memory network + conditional random field model), but...

AI Technical Summary

Problems solved by technology

[0003] The BERT+BiLSTM+CRF model is one of the deep learning methods (two-way conversion coding pre-training model + two-way long-term short-term memory network + conditional random field model), but the BERT model has the following shortcomings: 1. The introduction of masking is used in the pre-training stage 15% of the words are masked, but the words that are added to the masking marks are not included in the training phase, resulting in inconsistent usage patterns between the pre-training phase and the training phase; 2. In the pre-training phase, among the 15% words that are randomly The terms are conditionally independent and there is no correlation, but some natural language words are correlated, which leads to the performance loss of the model and cannot learn context information at the same time; 3. Only fixed-length text sequences can be modeled, and News texts are often long text sequences, making it impossible to fully model them

Images