Chinese and English-named entity identification method and system based on conditional random field (CRF)

Abstract

The invention provides a Chinese and English-named entity identification method and a system based on a conditional random field (CRF). The method comprises the following steps: (101) converting inquiry voice of a user into a text; (102) separating text information into Chinese characters and English letters on the basis of a finite state machine; (103) extracting the characteristics of a text of separated vocabularies; (104) performing entity identification on the text by adopting a training CRF model according to a characteristic extraction result, and marking an entity type, wherein the CRF model is a conditional random field model of a linear chain structure. The step (102) further comprises the following steps: (102-1) performing character separation on Chinese and English; (102-2) identifying English word strings by using the finite state machine, namely, combining adjacent English letters, blank spaces and symbols in English; (102-3) performing word segmentation on the English word strings.

Description

Technical field [0001] The invention relates to a sequence labeling model of a finite state machine and a conditional random field. It mainly aims at the phenomenon that the user query sentence has Chinese and English mixed in the process of human-computer interaction, and proposes a method and system for recognizing a sentence with a mixed Chinese and English named entity. Background technique [0002] The human-computer interaction system is where users put forward query requests through spoken language, and the system provides information services. A typical human-computer interaction system includes four components: automatic speech recognition, oral comprehension, dialogue management and speech synthesis. The part of oral comprehension is to transform the query sentence after speech recognition into corresponding semantic representation. However, with the great integration of internationalized information, multilingualism can be seen everywhere, which brings difficulties to...

AI Technical Summary

Problems solved by technology

Entity recognition in pure English does not require word segmentation because there are intervals between English words, and the recognition is easier; entity recognition in pure Chinese is more difficult than entity recognition in pure English, but for spoken Chinese and English mixed Entity recognition is even more difficult. At the same time, spoken language has the characteristics of irregular grammar and randomness compared with written language. Therefore, it is impossible to solve it only by using the named entity recognition method of the existing technology.

Images