Inquiring term rewriting method merging term vector model and naive Bayes

Abstract

The invention provides an inquiring term rewriting method merging a term vector model and naive Bayes and relates to information processing technologies. The method includes the steps that word2vec is used for training the vector model, first N terms most similar to an inquiring term are calculated, and an initial relevant lexicon is formed; then, relevancy calculation and analysis are conducted, candidate terms for inquiring and rewriting the lexicon are filtered, and terms with high relevancy are reserved. The method can effectively improve the accuracy and recall degree of inquiring results and effectively solve the problem that search inquiring is in vain or few valid results are sent back.

Description

technical field [0001] The invention relates to the technical field of computer information processing, in particular to a query word rewriting method in data mining technology. Background technique [0002] The word vector model is a technology that uses a neural network to map each word in a high-dimensional discrete space (the dimension is the number of words in the dictionary) into a real-number vector in a low-dimensional continuous space (ie, word embedding). In natural language processing tasks, word embeddings provide better semantic-level word distributed feature representation, which brings many conveniences to text processing tasks. The goal of word embedding representation is to learn a vector representation of each word and use this vector representation for different text processing tasks. The learned word vectors can either be input into task-specific supervised learning algorithms as complete word features, or can be useful extensions that rely on specific e...

AI Technical Summary

Problems solved by technology

[0004] The Skip-Gram model of the Hierarchical Softmax algorithm is used for query rewriting. The rewritten words are only calculated from the semantic relevance, without considering the connection between the query word and the context of the search recall results, and there are not many semantically related words , requires a lot of anticipation to be effectively mined out; while the query rewriting method based on Naive Bayes mines rewritten words from the co-occurrence probability between the query word and the context of the search recall result, although the context between , but the semantic correlation between the query term and the rephrased term is weak

Images