Multi-dense block detection and extraction method based on Possion distribution

Abstract

The invention discloses a Possion distribution-based multi-dense block detection and extraction method, which comprises the following steps of: carrying out suspicious degree measurement on multi-dimensional tensor data by utilizing a dense block suspicious degree measurement method to obtain a snapshots list containing a plurality of suspicious data snapshots; according to the snapshots list, extracting a single dense block from the multi-dimensional tensor data; removing the extracted single dense block from the multi-dimensional tensor data to obtain updated multi-dimensional tensor data; generating a new snapshots list according to the updated multi-dimensional tensor data, and extracting new dense blocks until m dense blocks are extracted; wherein the dense block suspicious degree measurement method is obtained through Possion distribution derivation containing double factors of counting and density. According to the method, the accuracy and recall rate of the dense blocks can be effectively improved while the detection efficiency is ensured.

Description

technical field [0001] The invention relates to a multi-dense block detection and extraction method based on Possion distribution, which belongs to the technical field of abnormal data detection. Background technique [0002] With the advent of the big data era, data anomaly detection becomes more and more important. Abnormal data will not only bring data fraud, affect the normal data analysis work, but may also lead to the loss and leakage of normal data. Therefore, it is the research focus in the field of data detection to quickly and accurately detect and extract abnormal data from massive data. [0003] There is a type of dense abnormal data in abnormal data. Dense abnormal data often has "consistency" and presents a dense block structure in tensor data. For this type of abnormal data, many dense block detection and extraction have appeared on the market. method, but the existing detection methods also have some shortcomings. For example, the CrossSpot detection method ...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the problems of low detection accuracy and recall rate of existing dense block detection methods, the present invention proposes a multi-dense block detection and extraction method based on Possion distribution, which takes into account the counting of dense blocks on the basis of Possion distribution, Considering the density of dense blocks, a new method for measuring the suspicious degree of dense blocks is proposed, which can effectively improve the efficiency, accuracy and recall of dense block detection

Images