Web anomaly detection method and system based on SMOTETomek and LightGBM

Abstract

The invention relates to a Web anomaly detection method and system based on SMOTETomek and LightGBM, and the method comprises the steps: firstly, collecting a mirror image flow PCAP packet from a switch, extracting network flow characteristics, then cleaning redundant data of a training data set, carrying out missing value processing, processing character type flow characteristics, and employing min-max method to normalize the data; then, calculating the flow feature importance based on the GBDT algorithm of the Gini coefficient; carrying out feature selection, carrying out oversampling on afew classes by combining an SMOTE algorithm and a Tomek Links algorithm, training a classifier through a LightGBM algorithm to detect abnormal flow , and finally, carrying out response and feedback processing on a detection result. According to the invention, unknown Web attacks can be detected, and the detection rate and the detection precision on few types of Web attacks are high.

Description

technical field [0001] The invention relates to the technical field of network security, in particular to a method and system for detecting Web anomalies based on SMOTETomek and LightGBM. Background technique [0002] With the rapid development of network technology, Web services have been widely used in network services, so they have also become the main attack target of criminals. Due to the continuous update of attack tools and the continuous improvement of attack technology, it is difficult for the intrusion detection system based on rule matching to detect deformation or unknown Web attacks. There is a certain lag in the method of mining log information to detect attack behavior. Although the model can detect unknown attacks, it is limited by the quality of feature extraction, and the detection rate and detection accuracy need to be improved. From a microscopic point of view, the intrusion behavior of the attacker needs to maintain communication with the target. Theref...

AI Technical Summary

Problems solved by technology

Due to the continuous update of attack tools and the continuous improvement of attack technology, it is difficult for the intrusion detection system based on rule matching to detect deformation or unknown Web attacks. There is a certain lag in the method of mining log information to detect attack behavior. Although the model can detect unknown attacks, it is limited by the quality of feature extraction, and the detection rate and detection accuracy need to be improved

From a microscopic point of view, the intrusion behavior of the attacker needs to maintain communication with the target. Therefore, the analysis of subtle changes in traffic characteristics based on machine learning has become the focus of research. Currently, detection based on abnormal traffic still has the distribution of attack traffic and normal traffic. Problems such as unevenness, large amount of data, difficulty in processing and low efficiency of machine learning algorithms

Images