A method and device for information word segmentation

Abstract

The invention discloses an information word segmentation method and apparatus, and relates to the field of information processing. The method comprises the steps of obtaining a target text required to be subjected to word segmentation; determining a preliminary word segmentation text corresponding to the target text according to characteristic information in the target text; and performing word segmentation on the preliminary word segmentation text according to specific characters to obtain a word segmentation result of the target text, wherein the target text is composed of the characteristic information. Through embodiments of the method and the apparatus, the target text can be correctly subjected to the word segmentation without a dictionary base and a corpus base, so that the word segmentation precision is improved and the word segmentation speed of the target text is increased.

Description

technical field [0001] The present invention relates to the field of information processing, in particular to an information word segmentation method and device. Background technique [0002] Word segmentation refers to dividing a word sequence into meaningful word sequences, also known as word segmentation. The process of automatically converting the word strings that make up the text into word strings by the word segmentation system is called automatic segmentation. Relatively speaking, there are spaces and other symbols between words in Indo-European languages ​​such as English, so segmentation is easier. The Chinese text is a continuous string of Chinese characters, and there is no clear separation mark between words. The word segmentation system needs to automatically identify word boundaries and divide the string of Chinese characters into correct word strings. [0003] The Chinese word breaker belongs to the technical category of natural language processing, and the...

AI Technical Summary

Problems solved by technology

[0006] The present invention provides an information word segmentation method and device, which are used to solve the problems of low word segmentation accuracy and poor word segmentation performance of the word segmenter in the prior art

Images