Massive data processing method

Abstract

The invention relates to the technical field of internet information processing, in particular to a massive data processing method. By setting filtering conditions of fields and the like, key information units in each document of original data are extracted and corresponding data records are formed; the data records are stored in a database; and based on this, an associative relationship among the information units is obtained by extension according to the same information units contained in different data records. According to the method, associated information paths hidden in massive target information can be analyzed as needed in massive internet information; a very reliable and convenient way is provided for deep mining and application of data information; and effective technical support is provided for target background analysis, market promotion, market segmentation, risk prediction and prevention control, and the like.

Description

technical field [0001] The present invention relates to the technical field of the Internet, in particular to a massive data processing method. Background technique [0002] With the development of society and the advancement of science and technology, the connection between individuals or groups has become closer, and the close connection has promoted the rapid dissemination and growth of information. Today's world has already entered the information age. Growth and accumulation, the era of big data is approaching, the basic characteristics of big data: large amount of data, various types, low value density, fast speed and high timeliness; for big data, correlation is more important than causality, which is The conclusion of the data age, in the face of massive information, how to analyze the correlation between certain types of information, and analyze the hidden value behind the information, can reflect the value of data information at a higher and deeper level, but face ...

AI Technical Summary

Problems solved by technology

[0002] With the development of society and the advancement of science and technology, the connection between individuals or groups has become closer, and the close connection has promoted the rapid dissemination and growth of information. Today's world has already entered the information age. Growth and accumulation, the era of big data is approaching, the basic characteristics of big data: large amount of data, various types, low value density, fast speed and high timeliness; for big data, correlation is more important than causality, which is The conclusion of the data age, in the face of massive information, how to analyze the correlation between certain types of information, and analyze the hidden value behind the information, can reflect the value of data information at a higher and deeper level, but face For such a massive amount of data, it is very difficult to quickly and accurately analyze the relationship between the data

[0003] For a specific information target, how to analyze the relationship between the target and other targets has more practical significance in many fields, because the target with the relationship often has a greater impact than a single simple individual when performing various activities. In addition, when establishing various activities externally, targets with related relationships will be restrained or supported by internal related relationships, which will be more complicated than the activity track of simple targets.

However, in real life, the amount of original information is huge, the distribution of key information is scattered, the value density of information is low, and the relationship between information objects is very complicated and generally hidden. Perceived, it is more difficult to find out whether the goal is related to other goals

Images