Network public opinion analysis system based on crawler and text clustering analysis

Abstract

The invention discloses a network public opinion analysis system based on a crawler and text clustering analysis. Relevant technologies, including web crawler and text processing, point of interest data mining and the like, are adopted, a vertical search thought is applied, collected webpage information is subjected to data preprocessing, and webpage information is subjected to personalized search on the basis of a user interest model; then, retrieved webpage text data is classified and clustered and is subjected to content extraction and analysis, finally, a point of interest data statistical report is formed, and information is displayed and pushed to users through the software interface of a personalized search system. A total thought is that the system is user-centered, and a modular design thought is adopted to construct the user interest model; and on the basis, relevant technologies, including webpage text data mining, database retrieval and the like, are deeply fused to establish the personalized search system based on topical crawlers and text processing.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a network public opinion analysis system based on crawler and text clustering analysis. Background technique [0002] Modern science and technology are changing the world every day and affecting the way people live and work. Especially in recent years, with the rapid development of new technologies such as mobile Internet, big data, and cloud computing, the scope of people's activities in cyberspace has continued to expand, and new Internet application models such as Weibo, WeChat, and Weishi have emerged, making people have entered With the era of big data and information consumption, with the continuous development of Web2.0 technology, various network platforms based on social relations and group wisdom have emerged, such as search engines, blogs, WeChat, Weibo, forums, etc. prominent social features. People can obtain video, image, text, sound and other multimedia resourc...

AI Technical Summary

Problems solved by technology

[0005] 2) The depth of information acquisition is not enough

Although the resources on the Internet are very rich and the ways to obtain information are also diverse, it is very difficult for people without knowledge background in related fields to obtain information in a specific field, and it is even more difficult to obtain the accurate information they need

In addition, even if information resources in related fields are obtained, it is difficult for the general public to identify the validity and accuracy of the information, which affects the depth of people's access to information to varying degrees.

[0006] 3) The degree of personalization of information acquisition is not enough

[0007] 4) The user experience of information acquisition is not good enough

With the rapid development of the Internet, especially the mobile Internet, various application distribution platforms, social networking sites, WeChat and Weibo, and other information sources are diversified. Traditional search engines alone cannot effectively integrate and classify these diverse and massive information. , and precise positioning, which restrict the improvement of user experience and also affect user stickiness

Images