Spark-based log auditing and reversed checking system for big data platforms

Abstract

The invention discloses a Spark-based log auditing and reversed checking system for big data platforms, and belongs to the field of big data and information security. The Spark-based log auditing and reversed checking system comprises a log acquisition module, a log auditing and analyzing module, a log auditing and reversed checking module, a log offline mining module and a view module. Log information in an organization system is acquired by the log acquisition module and is pushed to the log auditing and analyzing module in real time, and the log auditing and analyzing module analyzes the log information in real time, generates warning information and transmits the warning information to front-end pages of the view module. Compared with the prior art, the Spark-based log auditing and reversed checking system has the advantages that the warning information can be analyzed for service staffs in detail, and the Spark-based log auditing and reversed checking system is excellent in practicality and has excellent popularization and application value.

Description

technical field [0001] The present invention relates to the fields of big data and information security, in particular to a log audit and backcheck system based on the Spark big data platform. Background technique [0002] With the rapid development of big data technology in recent years, with the increasingly complex network environment of organizations, the requirements for information security are increasingly intensified. The main manifestations are as follows: 1. The logs in the network environment are complex, and the processing capacity required by the log audit system per second is increasing; 2. The attack methods faced by the organization are also becoming more and more diverse. Whether the update of the rule base is timely becomes a measure of log audit 3. Organizations are paying more and more attention to information security, and the professional skills of users who use the audit system are becoming more and more profound. Therefore, how to realize fast, intui...

AI Technical Summary

Problems solved by technology

The main manifestations are as follows: 1. The logs in the network environment are complex, and the processing capacity required by the log audit system per second is increasing; 2. The attack methods faced by the organization are also becoming more and more diverse. Whether the update of the rule base is timely becomes a measure of log audit The important standard of the system; 3. Organizations pay more and more attention to information security, and the professional skills of users who use the audit system are increasingly deep

Images