Internet data analysis system

Abstract

The invention discloses an internet data analysis system which comprises a data preprocessing module and a data analysis module, wherein the data preprocessing module extracts main content from webpage information of the internet, a text corresponding to each webpage is obtained through filtration, the obtained texts are firstly segmented by a segmentation device to obtain a plurality of segmentation words, and segmentation words highlighting characteristics of the texts are reserved through dimensionality reduction of characteristic values; and the data analysis module selects one or more categories of algorithms from a classification algorithm, a clustering algorithm, an association rule algorithm and special rule matching algorithm according to analysis requirements, each category of algorithm adopts one or more algorithms for processing the segmentation words which are subjected to dimensionality reduction and correspond to the webpages output by the data preprocessing module, and the analysis result is stored. With the adoption of the internet data analysis system, the defect of inaccurate data analysis result caused by a single data mining algorithm is overcome, or the time cost due to the need of secondary system development when other algorithms are used on the basis of one algorithm is saved, and efficiency and accuracy of data analysis are improved.

Description

technical field [0001] This application relates to a data mining (data mining) technology, in particular to a method for analyzing Internet data. Background technique [0002] Data mining refers to the process of revealing hidden, previously unknown and potentially valuable information from a large amount of data. It is mainly based on artificial intelligence, machine learning, pattern recognition, statistics, database, data retrieval and other technologies to achieve the above goals. [0003] The existing data mining process uses one or several fixed data analysis algorithms to build a data analysis system. Since each algorithm has its own advantages and disadvantages, it often causes deviations between data analysis results, resulting in data analysis based on data analysis. It is difficult for upper-level business applications to make decisions. Contents of the invention [0004] The technical problem to be solved in this application is to provide an Internet data ana...

AI Technical Summary

Problems solved by technology

[0003] The existing data mining process uses one or several fixed data analysis algorithms to build a data analysis system. Since each algorithm has its own advantages and disadvantages, it often causes deviations between data analysis results, resulting in data analysis based on data analysis. It is difficult for upper-level business applications to make decisions

Images