Streaming intensive data desensitization method and data desensitization equipment thereof

Abstract

The invention discloses a streaming intensive data desensitization method and data desensitization equipment thereof. The equipment comprises a FPGA (Field Programmable Gate Array), a TCAM (Ternary Content Addressable Memory), a DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory), a multi-core network processor and an expandable connection back panel. The method comprises the following steps that: taking metadata streams in on-line collection, real-time filtering and offline association network communication applications as targets, carrying out targeted classification, aggregation and association, and creating a bounded data generalization application model and technology and an innovative streaming intensive data desensitization pattern and equipment on the basis of a normalized data sharing level rule and strategy configuration and a customized whitelist. The equipment and the method have the advantages that the sharing degree and the range of network circulation big data can be greatly improved, parallel and asynchronous data generalization calculation, synchronous strategy matching and linear decision boundary expansion are carried out, a customized programmable ability is provided, and the desensitization of non-secret-involved data is instantly carried out at a high speed according to a rule and a strategy.

Description

technical field [0001] The invention relates to a flow-intensive data desensitization method and a data desensitization device thereof, belonging to the technical field of information communication. The invention is applied to the field of on-demand hierarchical sharing of non-confidential big data on the Internet, and is used to solve the technical problems of analysis and data mining of network circulation big data and network security threat situation awareness. Background technique [0002] At present, Internet communication and its applications have spawned and carried the era of big data. Big data is not only a large flow of application data transmitted by the network, high speed, and many types, but more importantly, due to the emergence of personalized behaviors, the Internet has become an open and complex giant system, and it dynamically carries complex and unknown information. issues, including cybersecurity threats and risks. [0003] The emergence of the Intern...

AI Technical Summary

Problems solved by technology

[0016] Since the desensitization of stream-intensive data combines the characteristics of the repetition of communication-intensive tasks and the learning of computing-intensive tasks, it is difficult for systems relying on pure software or software architecture to desensitize stream-intensive data

For general-purpose CPUs, the more flexible and changeable tasks are, the more time is spent on task switching, and the efficiency of CPU execution tasks is lower.

In addition, the CPU needs to receive and send data through the network card, and the overhead of scheduling, waiting, processing, and input and output of streaming data, referred to as IO, inevitably increases the waste of resources and the uncertainty of operation delay.

Images