Named entity identification method oriented to field of network security

Abstract

The invention discloses a named entity identification method oriented to the field of network security. The method comprises two stages of model training and sample selection. The first stage comprises the steps: acquiring an initial character vector which contains semantic information and changes dynamically through ALBERT training of a pre-training language model, sending the initial character vector into a Bi-LSTM + CRF network to be trained, and outputting a label sequence with the maximum input text sequence probability is output. The second stage comprises the steps: based on the model obtained through training in the first stage, selecting network security text data with marking value and training value for manual and machine marking in the mode of combining active learning and self-learning, and iteratively training the model after the network security text data with marking value and training value are added to existing marked text data. According to the method, the accuracy of network security entity identification is remarkably improved, and the problems of lack of marking corpora, high marking cost and the like in the network security field are effectively relieved.

Description

technical field [0001] The invention relates to the field of natural language processing of network security text data, in particular to a named entity recognition method oriented to the field of network security. Background technique [0002] With the rapid development and wide application of Internet technology and artificial intelligence technology, the amount of information on the Internet has shown explosive exponential growth, and today's society has entered the era of informatization and big data. With the rapid development of network information technology, the network environment has become increasingly complex. A large number of illegal organizations and individuals use viruses or loopholes to launch extensive and continuous network attacks on targets in multiple fields through the Internet to steal relevant confidential information or cause relevant damage. At present, people's production and life are increasingly dependent on network information, and the number o...

AI Technical Summary

Problems solved by technology

Methods based on rules and dictionaries have higher recognition accuracy, but building domain rules and dictionaries will consume a lot of manpower, and have poor applicability and transferability; methods based on machine learning, due to the existence of a large number of professional vocabulary and Chinese-English mixed vocabulary in network security text data , one-word polysemous vocabulary and unregistered vocabulary, artificially constructed features are not universal and performance depends on the size of training samples; methods based on deep learning can realize automatic feature extraction, but traditional deep neural networks cannot fully extract effective features, and at the same time A large amount of labeled corpus is required as training data, and it takes a lot of money and manpower to label network security text data, and the training cost is extremely high

Images