Network log storage method based on multi-attribute hash deduplication in intrusion detection system

Abstract

The invention relates to network security and data storage, and aims to provide a network log storage method based on multi-attribute hash deduplication in an intrusion detection system. The network log storage method based on multi-attribute hash deduplication in the intrusion detection system comprises log deduplication and log storage, and ensures that the intrusion detection system can store network logs into a local server after deduplication. The intrusion detection system comprises a data acquisition server and a plurality of data storage and analysis servers, and both the data acquisition server and the data storage and analysis servers are connected with a switch. According to the network log storage method provided by the invention, acquisition and deduplication operations of the network logs can be completed with only one server by using a multi-attribute segment hash method, and smaller computing complexity and space requirements can be achieved; more accurate log deduplication can be realized, data cannot be mistakenly lost, and the lower rate of missing report of repeated logs can be achieved; and the storage mode of the network logs adopts a storage method that is proportional to the server performance, and thus the data storage efficiency can be ensured, and the performance of subsequent data analysis tasks can also be increased.

Description

technical field [0001] The invention relates to the fields of network security and data storage, in particular to a network log storage method based on multi-attribute hash deduplication in an intrusion detection system. Background technique [0002] With the popularization of the Internet, various network attacks emerge in an endless stream, and the security of network users is seriously threatened. The purpose of an intrusion detection system is to discover suspicious attack behaviors through the analysis of network data, usually using detection methods based on Bayesian networks, detection methods based on pattern prediction, detection methods based on machine learning, and detection methods based on data mining. Wait. Although the detection methods are different, the detection process generally includes four steps: data collection, data storage, data analysis, and system response. The data collection process collects or samples network data, especially log data, the da...

AI Technical Summary

Problems solved by technology

[0005] To sum up, the current network log deduplication method in the intrusion detection system either lacks real-time guarantee, or has a high false negative rate of duplicate logs. heavy method

However, the log storage method in the existing intrusion detection system does not provide a performance optimization strategy for subsequent data analysis operations, resulting in waste of server storage and computing resources, and limits the room for further optimization of data analysis performance.

Images