Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Template-oriented Word2vec-based log exception detection method and device

An anomaly detection and logging technology, applied in computing models, structured data retrieval, special data processing applications, etc., can solve problems such as deviations, affect the results of anomaly detection, and cannot be fully reflected, so as to improve efficiency and reduce training data. effect of scale

Pending Publication Date: 2020-07-28
CHANGSHA UNIVERSITY OF SCIENCE AND TECHNOLOGY
View PDF2 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, there will be deviations in the representation of the log sequence feature vector of this method, which will affect the final result of anomaly detection; secondly, due to the large scale of the system log sequence, it needs to be performed for each word when training, so the computational complexity is relatively high. high
[0005] (2) The log sequence anomaly detection of words as the processing object of Word2vec, which directly takes the original log as input without preprocessing the original log
The disadvantages of directly using the original log as input are: firstly, when some data in the original log is lost, some log messages are incomplete and cannot fully reflect the content expressed by the event; secondly, there is some redundant information in the original log. Take the data set as an example. Each log message includes timestamp, date, node, time, repeated node, message type, component (where the message is generated), message level, statement content, etc. These incomplete log messages and redundant Information will affect the results of log anomaly detection

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Template-oriented Word2vec-based log exception detection method and device
  • Template-oriented Word2vec-based log exception detection method and device
  • Template-oriented Word2vec-based log exception detection method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] The technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Apparently, the described embodiments are only some of the embodiments of the present disclosure, rather than all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present disclosure. It should be noted that, in the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other. In addition, the role of the drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and visually understand each technical feature and overall technical solution of the present disclosure, but they should not be construed as limiting the protection sc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a template-oriented Word2vec-based log anomaly detection method and device, and the method comprises the following steps: carrying out the preprocessing of an original log, obtaining a log template, and carrying out the segmentation of the log template, so as to obtain a log sequence; solving a feature vector of the log template based on Word2vec, wherein the ID serial number of the log template is used as the input of the Word2vec; solving a feature vector of the log sequence according to the feature vector of the log template; and performing machine learning on the feature vector of the log sequence to obtain an anomaly detection model, and performing detection according to the anomaly detection model. Starting from a Word2vec processing object as a template, thescale of training data can be reduced. Moreover, the original log is preprocessed, and the time consumed by log anomaly detection is reduced through preprocessing so as to avoid affecting the final anomaly detection result.

Description

technical field [0001] The invention relates to the technical field of log anomaly detection, in particular to a template-oriented and Word2vec-based log anomaly detection method and device. Background technique [0002] At present, the word is used as the log sequence of the processing object of Word2vec (a language representation model that generates word vectors in natural language processing) (expressed as log events generated by the system in chronological order within a period of time, and the original log is divided by the window And get) the steps of anomaly detection are as follows: First, the original log is used as input, and Word2vec is used to map each word in the original log to the vector space, so that each word has its corresponding coordinates, and then the log event (reflecting the system The coordinates of the operation message) are represented by the centroid of all word coordinates in the event, and the log sequence is represented by the centroid of all...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/24G06N20/00
CPCG06F16/243G06N20/00Y02D10/00
Inventor 王进唐杨宁何施茗赵长庆曹敦
Owner CHANGSHA UNIVERSITY OF SCIENCE AND TECHNOLOGY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products