Water conservancy field text retrieval method and system based on topic mining

Abstract

The invention discloses a topic mining-based water conservancy field text retrieval method and system. The method comprises the following steps of collecting a data set; preprocessing the data set; the method comprises the following steps: constructing a topic attention model GAN-BiGRU Topic Attention Model in which a bidirectional adversarial neural network based on topic mining and a bidirectional GRU are combined; verifying and testing the test set; performing theme-related sorting; performing theme retrieval; the system comprises a data crawling module, an index construction module and a data retrieval module. According to the method, the accuracy and recall rate of theme diversity detection, theme coherence detection and downstream classification tasks are obviously higher than those of an existing method, and a new solution is provided for research in related fields; based on sufficient field data reserve support, the network model formed by combining the bidirectional adversarial neural network and the bidirectional GRU is used in the field of water conservancy retrieval for the first time, and contribution is made to application of topic model retrieval in the field of water conservancy.

Description

technical field [0001] The invention relates to natural language processing and information retrieval, in particular to a text retrieval method and system in the field of water conservancy based on topic mining. Background technique [0002] Now we are in an era of informationization. With the popularization and use of computers, big data technology, cloud computing technology, artificial intelligence technology, etc. in the information field, the electronic digitization of various materials and documents has given traditional information Retrieval has had a huge impact, so information retrieval technology and related technologies are still widely used today, and continue to develop and innovate. Since modern times, my country's water conservancy industry has developed vigorously, and a large amount of text data has been accumulated in related water conservancy fields. The problem of water conservancy information overload has also become more and more severe. It has become m...

AI Technical Summary

Problems solved by technology

However, this method still has the following shortcomings: first, the inference process needs to be customized, the inference complexity increases significantly with the increase of model complexity, and the design of the inference process is difficult to automate

Second, it is difficult for this method to effectively scale up large text collections and use GPU for parallel computing

Third, the method is not scalable and cannot be jointly trained with other deep neural networks

Images