Multi-dense-block detection and extraction method for big data

Abstract

The invention discloses a multi-dense block detection and extraction method for big data, and aims to solve the technical problems of low detection accuracy and low recall rate of a dense block detection method in the prior art. The method comprises the following steps: acquiring K-dimensional tensor data D, the number m of dense blocks to be extracted and the size range of the dense blocks; performing suspicious degree measurement on the K-dimensional tensor data D by using a density tracking coefficient based on a piecewise function, and generating a snapshots list according to the suspicious degree and a dense block size range; and extracting m dense blocks from the K-dimensional tensor data D according to the snapshots list. According to the method, the accuracy and recall rate of the dense blocks can be effectively improved while the detection efficiency is ensured.

Description

technical field [0001] The invention relates to a multi-dense block detection and extraction method of big data, belonging to the technical field of abnormal data detection. Background technique [0002] With the advent of the era of big data, the detection of abnormal data is becoming more and more important. Abnormal data containing network attacks usually have "consistency", for example: a group of IP addresses sends a group of target IP addresses to several Fixed ports to send requests; data about fraudulent behaviors such as buying zombie fans to increase influence will show a high degree of consistency in the users that a group of specific users follow. By establishing a tensor model, the "consistency" of the above-mentioned abnormal data will cause dense blocks in the tensor data, so abnormal data such as network attack detection and social network zombie follower detection can be realized by detecting and extracting dense blocks in the tensor data detection function...

AI Technical Summary

Problems solved by technology

[0008] Aiming at the problems of low detection accuracy and low recall rate existing in the existing dense block detection method, the present invention proposes a multi-dense block detection and extraction method for large data, adding a density block based on a piecewise function when judging suspiciousness Tracking coefficient, making dense blocks with high density more suspicious, improving the detection accuracy and recall rate of dense blocks

Images