Hadoop-based network security event analyzing method

Abstract

The invention discloses a Hadoop-based network security event analyzing method. The method includes that by utilizing the characteristic that hadoop is high in efficiency, fault tolerance, expandability and reliability and open in source during mass data processing and adopting the advantage that an HDFS (hadoop distributed file system) is high in fault tolerance and extensibility, a user is allowed to arrange Hadoop on common and low-price hardware to form a distributed-type system; MapReduce provides a development parallel application program and realizes distributed-type calculating and parallel task processing on a cluster; the HDFS provides support like file operation and storage in the process when the MapReduce processes tasks, a data collecting system collects network security event information from each network security device, generates data files and stores the data files in the HDFS by using an API or an instruction, data are stored on nodes of multiple common hardware resources in a distributed manner, the MapReduce is used to analyze the event information and output analyzing results to demonstrate that the MapReduce realizes distribution, tracking, execution and the like on the basis of the HDFS, the results are collected, and major tasks of the Hadoop distributed-type cluster are completed through interaction.

Description

technical field [0001] The invention relates to the field of network security, in particular to a method for analyzing network security events based on Hadoop. Background technique [0002] The application of the network security management platform can realize the visual monitoring and configuration of the overall network security status, simplify the complexity of manual analysis and management, save human resources for network security, and provide technical basis for rapid emergency response. However, with the increasing scale of informatization, the scope of network security protection is becoming wider and wider, and the complexity of security management is also increasing. The more application systems are used, the greater the amount of secure data, and the requirements for data processing on the network security management platform are getting higher and higher, and the massive data processed may reach TB or even PB level. If the data processing efficiency cannot co...

AI Technical Summary

Problems solved by technology

If the data processing efficiency cannot cope with the increasing data size requirements, it will have an impact on the applicability, availability, reliability, etc. of the network security management platform, and the cost of manual maintenance may also greatly increase

Images