A hot event aggregation method and device

Abstract

An embodiment of the present invention provides a hot event aggregation method and device, the method comprising: obtaining an original report based on the title of the hot event; determining a seed report based on the title of the hot event and the original report and multiple non-seed reports; use the seed report to generate hot event clusters; calculate the similarity between each non-seed report and the title of the hot event and each report in the hot event cluster; obtain the non-seed report with the highest similarity Seed report; determine whether the similarity of the non-seed report with the highest similarity is greater than a similarity threshold; if so, store the non-seed report with the highest similarity in the hotspot event cluster. The embodiment of the present invention introduces seed reports, hot events, and the similarity between reports in the aggregation process, so that the clustering algorithm can focus more on the event itself, measure the similarity of the text more accurately, and obtain better aggregation effects.

Description

technical field [0001] The present invention relates to the technical field of information processing, in particular to a hot event aggregation method and a hot event aggregation device. Background technique [0002] Hot event aggregation is an important basic technology of NLP (natural language processing, natural language processing), which plays an important role in recommendation, search, bubble and other businesses. [0003] According to the aggregation of reports related to hot events, most of them currently use the TF-IDF word weight clustering method to achieve a certain effect on the similarity between related reports. After the text is segmented, TF-IDF is calculated as the weight of the corresponding word. After the word vector is generated, the similarity is calculated according to the cosine distance, and then the corresponding reports are aggregated according to the similarity between the texts through the related clustering algorithm. [0004] However, since ...

AI Technical Summary

Problems solved by technology

[0004] However, since TF-IDF does not take into account the influence of the context of the text, it will have some defects in the expression of similarity, and will bring the following disadvantages to the application of aggregation hotspot events:

[0005] 1. Since the calculation of TF-IDF is more dependent on the size and quality of the corpus, the larger the corpus, the better the quality, and the more accurate the calculated TF-IDF is, but it will cost a lot in the process of preparing the corpus

[0006] 2. The calculated TF-IDF is based on the assumption of independence between words, so the corresponding word weights obtained are also independent of each other, but in the actual text, the relationship between words in the text is also close, which directly affects Calculation of subsequent text similarity

[0008] In addition, due to the independence of TF-IDF's own words, when evaluating the similarity between reports and events, there will be a problem of ignoring the emphasis of the text itself, resulting in inaccurate discrimination of similarity

Images