Hot event aggregating method and apparatus

Abstract

The embodiments of the invention provide a hot event aggregating method and apparatus. The method comprises the following steps: based on a headline of a hot event, acquiring an original report; basedon the headline of the hot event and the original report, determining a seed report and multiple nonseed reports; generating a hot event cluster by using the seed report; calculating a similarity between each of the nonseed reports and the headline of the hot event, as well as each of reports in the hot event cluster; acquiring a nonseed report having a highest similarity; judging whether a similarity of the nonseed report having the highest similarity is greater than a similarity threshold value or not; and if yes, storing the nonseed report having the highest similarity to the hot event cluster. According to the hot event aggregating method and apparatus provided by the embodiments of the invention, by instructing the similarities between the seed report and the hot event as well as thereports to an aggregation process, an aggregation method surrounds the event in itself more, a similarity of a text is balanced more accurately, and a better aggregation effect is acquired.

Description

technical field [0001] The present invention relates to the technical field of information processing, in particular to a hot event aggregation method and a hot event aggregation device. Background technique [0002] Hot event aggregation is an important basic technology of NLP (natural language processing, natural language processing), which plays an important role in recommendation, search, bubble and other businesses. [0003] According to the aggregation of reports related to hot events, most of them currently use the TF-IDF word weight clustering method to achieve a certain effect on the similarity between related reports. After the text is segmented, TF-IDF is calculated as the weight of the corresponding word. After the word vector is generated, the similarity is calculated according to the cosine distance, and then the corresponding reports are aggregated according to the similarity between the texts through the related clustering algorithm. [0004] However, since ...

AI Technical Summary

Problems solved by technology

[0004] However, since TF-IDF does not take into account the influence of the context of the text, it will have some defects in the expression of similarity, and will bring the following disadvantages to the application of aggregation hotspot events:

[0005] 1. Since the calculation of TF-IDF is more dependent on the size and quality of the corpus, the larger the corpus, the better the quality, and the more accurate the calculated TF-IDF is, but it will cost a lot in the process of preparing the corpus

[0006] 2. The calculated TF-IDF is based on the assumption of independence between words, so the corresponding word weights obtained are also independent of each other, but in the actual text, the relationship between words in the text is also close, which directly affects Calculation of subsequent text similarity

[0008] In addition, due to the independence of TF-IDF's own words, when evaluating the similarity between reports and events, there will be a problem of ignoring the emphasis of the text itself, resulting in inaccurate discrimination of similarity

Images