Malicious encrypted channel detection method based on process behavior analysis

Abstract

The invention relates to a malicious encrypted channel detection method based on process behavior analysis. The method comprises the following steps: step 1, acquiring encrypted session flow data and classifying processes; comprising the steps of encrypted traffic data acquisition, preprocessing and process classification; 2, constructing a process file encryption communication behavior feature tree; constructing a three-level feature tree of a meta feature of the encrypted session, namely an IP layer, a TCP segment load length sequence feature, namely a TCP layer, and an SSL message state transition feature, namely an SSL recording layer; step 3, anomaly detection based on the feature tree; comprising the steps of collecting normal encrypted communication behavior data; constructing a normal encryption communication behavior benchmark; constructing a target PEF encrypted communication behavior model; calculating the dissimilarity degree between the feature trees; and performing anomaly detection based on a threshold value. By means of an anomaly detection method, the limitation that malicious encrypted channels are detected purely from the session level is broken through, and effective detection of malicious process files is achieved.

Description

technical field [0001] The invention relates to a malicious encrypted channel detection method based on process behavior analysis, and belongs to the technical field of information security construction / network security. Background technique [0002] With the rapid development of network technology, the Internet has been widely used in various fields such as military affairs, economy, education, and life. However, while the Internet brings various conveniences to our lives, it also brings various security problems. The number and types of various computer viruses, worms and other malicious software are also increasing rapidly, which brings great harm to the safety of Internet users. great challenge. In order to protect the transmitted data, encrypted transmission has become a widely used method. Cisco's survey shows that from 2016 to 2017 alone, encrypted traffic has increased by more than 90%, and more than 50% of traffic is encrypted traffic. The use of encrypted transm...

AI Technical Summary

Problems solved by technology

However, the current mainstream feature selection contains a large number of noise features that are not strongly related to the encryption behavior itself. For example, the side channel feature extracts the time difference (interval) of the sequential arrival of data packets. However, the time attribute is easily affected by the delay of the network communication environment. Stable; ②The characteristics of TLS handshake protocol parameters can be actively selected and modified, etc., which are not the essential characteristics of malicious encrypted communication; ③A large number of TLS sessions are transmitted by session multiplexing without certificate transmission, so the server-side digital certificate is not an essential feature of encrypted communication; ④The header data of the TCP / IP protocol will introduce too many protocol transmission details, which more serve the needs of stable connections, and are also not the essential characteristics of encrypted communication

The existence of the above noise features makes the classifier model trained by it unable to accurately capture the classification boundaries of encrypted communication behavior patterns, resulting in inevitable high misjudgment results

[0014] (2) Lack of multi-session association pattern analysis

Therefore, the present invention considers that it is necessary to use multi-session correlation analysis of the same process file to identify the malicious correlation among them. However, the current malicious encrypted channel detection method mainly focuses on extracting encrypted flow features for a single session to perform detection tasks, and lacks the mining of inter-session correlation patterns. identification, making it impossible to effectively identify hidden malicious correlations between sessions

The above problems lead to the fact that although the current detection method has a good performance on controllable experimental data in the laboratory environment, when it is applied in the actual production work scene, it is often limited by the limitation of malicious expression in a single session. The accuracy rate seriously restricts the applicability of actual malicious encrypted channel detection

[0015] (3) Excessive reliance on supervised learning classifiers

With the popularity of encrypted communication represented by the TLS protocol (Transport Layer Security, secure transport layer protocol), attackers often use the TLS protocol to construct encrypted channels, making it impossible for DPI technology to analyze the content of the data packet, thereby bypassing the existing security defense mechanism. Carry out confidential data theft, key system attacks, etc., causing huge losses to organizations, enterprises or users

Images