Log anomaly detection method

Abstract

The invention provides a log anomaly detection method, and relates to the field of data processing. The method comprises the following steps: acquiring historical log data and real-time log data; preprocessing the historical log data; obtaining an initial data set and a stream data set based on the historical log data set; obtaining an initial log mode library based on the initial data set; updating the initial log mode library to obtain a log mode library; acquiring a mode library mode vector based on the log mode library; obtaining a streaming data mode vector based on the streaming data set; processing the historical log data set based on the mode library mode vector to obtain a log data mode; obtaining a dependency relationship library based on the two mode vectors; obtaining a threshold of anomaly detection based on the two mode vectors; preprocessing the real-time log data; and respectively detecting a real-time log data set based on the log data mode, the threshold value, the log mode library and the dependency relationship library. According to the method, the interpretability is relatively high when the log abnormal condition is detected.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a log anomaly detection method. Background technique [0002] With the continuous development of science and technology, computer applications are playing an increasingly important role in people's production and life. The log is an important carrier for recording the operating status and events of the computer system. It can record the system status and important events at various key points to help debug system failures and perform root cause analysis. Therefore, the log is an excellent source of information for online monitoring and anomaly detection. By detecting abnormalities in system logs, actionable information can be provided at critical moments to ensure efficient and stable operation of the system. [0003] The method for detecting log anomalies in the prior art is generally as follows: firstly extract the log key of the log data based on the word frequency or...

AI Technical Summary

Problems solved by technology

[0006] Aiming at the deficiencies of the prior art, the present invention provides a log anomaly detection method, which solves the technical problem of low interpretability when the prior art cannot detect log anomalies

Images