Frequency dictionary building method, word segmentation method, server and client device

Abstract

The invention provides a frequency dictionary building method, a word segmentation method, a server and a client device, wherein, the frequency dictionary building method comprises the following steps: obtaining search behavior data, wherein, the search behavior data comprises a plurality of search terms and an object name clicked by a user in a data object returned based on each search term. Thecommon character strings of each search term and the corresponding clicked object name and the frequency of each common character strings are counted. Frequency dictionaries are generated from the counted frequencies of a plurality of common strings, wherein the frequency dictionaries are used for word segmentation of the text to be segmented. The technical proposal provided by the embodiment of the application solves the technical problems that the existing word segmentation mode can not effectively divide new words and the manual labeling cost is too high, and achieves the technical effect of simply and efficiently realizing word segmentation.

Description

Technical field [0001] This application belongs to the field of Internet technology, and in particular relates to a frequency dictionary building method, word segmentation method, server and client device. Background technique [0002] With the rapid development of e-commerce, people are increasingly shopping through shopping websites. Word segmentation is often required when the shopping websites are classified into categories or target objects are matched. For example: the segment "domestic counter high-end chiffon skirt" to be segmented is divided into: domestic counter / high-end / chiffon skirt by word segmentation method. [0003] At present, the commonly used word segmentation methods mainly include: word segmentation methods based on dictionary matching and word segmentation methods based on statistical machine learning. [0004] Among them, the word segmentation method based on dictionary matching needs to rely heavily on the word segmentation dictionary. Therefore, if a new w...

AI Technical Summary

Problems solved by technology

The word segmentation method based on statistical machine learning requires a large amount of manually labeled corpus, which is cumbersome to implement, and the staff needs to spend a lot of time and energy, and the word segmentation method based on statistical machine learning needs to be segmented. The text and the field of training corpus are related , otherwise word segmentation accuracy is very low

[0005] For the above problems, no effective solutions have been proposed so far

Images