Method and system for effectively identifying machine access behavior

Abstract

The invention discloses a method and a system for effectively identifying a machine access behavior. The method comprises the following steps: inputting a Waf access log; detecting an engine consumption Waf access log in real time, and outputting a Bot user; the management and control center managing and controling the Bot user and feeding back a management and control result to the adaptive learning module; and the adaptive learning module autonomously updating and loading an update result to the real-time detection engine to form a closed loop. The device comprises a Waf access log acquisition module, a real-time detection engine module, a management and control center module and an adaptive learning module. A computer device and a storage medium can realize the process of the method byexecuting a computer program. According to the method, the Gini coefficient, the stability coefficient, the white list, the model adaptive learning and the like are combined, and Bot user entity access behavior detection is realized. The method has the characteristics of high accuracy, interpretability, high generalization ability and the like, and a WAF management background and a protection system can be seamlessly butted, so that user personalized configuration and automatic arrangement response are realized.

Description

technical field [0001] The invention relates to the field of network security, in particular to a method and system for effectively identifying machine access behaviors. Background technique [0002] With the development of the Internet and the involvement of national forces, the war on network security continues to heat up. And Bot traffic is flooding every corner of this battlefield as bullets. The 2019 Bad Bot Report pointed out that in 2018, 37.9% of Internet traffic came from "bots", and malicious Bot traffic accounted for 20.4% of all traffic. In fact, since 2015, the proportion of malicious Bot visits has generally increased year by year. The 2019 report pointed out that among these malicious Bots, scalpers are very eye-catching, and 24.1% of them are advanced Bot programs, which are not easy to prevent and control. These malicious robots are called Advanced Persistent Bots (APBs, advanced persistent robots). They often take the form of botnets that simulate real-l...

AI Technical Summary

Problems solved by technology

[0003] At present, there are several solutions for identifying malicious bot traffic: 1) Manual analysis: Operators can identify malicious traffic by manually viewing and analyzing request logs. Undoubtedly, this method is extremely inefficient; 2) Limit the request speed of the source IP: the The solution is simple to implement and can identify simple bot traffic with high frequency visits, but requires operators to have a deep understanding of their own site business, and the threshold setting requirements are relatively high; in addition, for scenarios such as flash sales and snap-ups of popular products, this method is easy to cause false interception; in addition Some IPs may belong to gateway IPs, and the cumulative request frequency is very high; therefore, in the absence of a reasonable threshold, limiting the source IP request speed will lead to a higher false positive rate, and the consequences are more serious; 3) Using cookies, JS, device fingerprint technology support: Bot supports cookie and JS at a low cost, so it is easy to be bypassed by the Bot team; device fingerprint technology will calculate the hash value of multiple attribute parameters of the browser, but as a network program, Bot, The tampering of attribute information can be realized by itself, and the server cannot verify the authenticity of device fingerprint information. This mechanism is also easily bypassed by the Bot team; in addition, the probability of device fingerprint conflict is high, and many proxy machines are cloud hosts or other For machines installed in a unified way, the information collected by the device fingerprint technology is the same, so in theory, the false positive rate will increase; of course, it is helpful to make the device fingerprint more refined, but the cost is high; 4) Threat Intelligence technology: The core of threat intelligence lies in sharing. However, in the domestic environment, major security vendors are more inclined to cash in their labor results, and users are more willing to use them rather than share them; therefore, the threat intelligence field is The country has been developing slowly. In addition, with the development of the cloud industry in recent years, the cost of changing IPs for Bot groups has become lower and lower, which has also led to a significant reduction in the value of threat intelligence with IP as the core; 5) Conversion based on access links Probabilistic business behavior analysis: Most normal users jump from page A to page B to C, and then to D, while abnormal users may skip the previous steps and go directly to C or D; based on this premise, using a large number of normal Access link data to train the Markov model to obtain a probability transformation matrix to detect outliers. This method is more effective for the scalper scenario, but it lacks generality and cannot cover most Bot traffic; on the other hand, not all All business requests contain referer information, which makes data collection more difficult. In this case, the only way to collect user access link information is through JS core link embedding. However, this method requires the collected sample data Sufficient enough can effectively cover various business scenarios. In theory, the more complicated the business scenario is, the harder it is to fit

In summary, the current existing methods have defects such as high cost, accuracy, and low universality.

Images