Chinese ambiguity word processing method based on dependency parsing

Abstract

Part-of-speech tagging of a Chinese ambiguity word is one of the major problems influencing performance of a Chinese information processing system currently. The ambiguity word is a grammatical phenomenon of one word with multiple parts of speech existing in Chinese, such as a gerund isomorph. The invention discloses a Chinese ambiguity word processing method based on dependency parsing. The method comprises the following three steps: a sentence containing a large number of ambiguity words is firstly selected to serve as a training language material, and an ambiguity word semantic role statistical rule with high accuracy and coverage is obtained through dependency parsing; then, sentence dividing, word dividing, part-of-speech tagging and dependency parsing pre-treating are carried out on a to-be-processed text, and on the base of a morphology, syntax and context rule bank, parts of speech of ambiguity words in a word dividing result are tagged; and finally, the ambiguity word semantic role statistical rule obtained in advance is adopted, the part of speech of an ambiguity word in a different context environment can be accurately recognized through dependency parsing. The method can effectively improve part-of-speech tagging accuracy in the case of Chinese text processing, and can be widely applied to various Chinese information processing systems.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to part-of-speech consistency checking of Chinese information processing and Chinese concurrent word processing. Background technique [0002] Conjunctive words mean that a word has two or more types of grammatical functions in different contexts, that is, the vocabulary has different parts of speech in different contexts. Specifically, the concurrent words have the same pronunciation and the same symbolic form, and there is a certain connection in the meaning of the words, that is, the same sound, the same shape, and the meanings are not exactly the same but closely related. Concurrent words are a unique grammatical phenomenon in Chinese. In English, words with the same root but different parts of speech usually have different morphological forms. However, in the Chinese context, the phenomenon of concurrent part of speech in a word with multiple parts of speech is more c...

AI Technical Summary

Problems solved by technology

Common part-of-speech tagging tools such as Stanford NLP, LTP, and Zpar do not specifically consider the phenomenon of Chinese part-of-speech words, so the accuracy of part-of-speech recognition for part-of-speech words is low

Images