Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Log processing method, device and equipment and storage medium

A processing method and log technology, which is applied in the field of data processing, can solve problems such as long delays and failure to meet the needs of log early warning processing, and achieve the effects of reducing time delay, improving log processing efficiency, and ensuring accuracy

Pending Publication Date: 2021-04-06
BEIJING TOPSEC NETWORK SECURITY TECH +2
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] For the real-time update scenario of massive logs, the existing solution has a long time delay in the process of log warning processing, which cannot meet the processing requirements of log warning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log processing method, device and equipment and storage medium
  • Log processing method, device and equipment and storage medium
  • Log processing method, device and equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, in the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0030] In the following description, many specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways than described here; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, and Not all examples.

[0031] figure 1 The flow chart of a log processing method provided by an embodiment of the present disclosure may be applicable to how to process logs to determine whether to perform log warning processing. The log processing method can be executed by a log processing device, which can be imple...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention relates to a log processing method and device, equipment and a storage medium, and the method can comprise the steps: obtaining a to-be-processed log, carrying out the clustering of the to-be-processed log, and obtaining a first preset number of candidate log classification sets; calculating the entropy of each log in each candidate log classification set, wherein the entropy value is used for representing the information amount of the log; screening a second preset number of logs from each candidate log classification set according to the entropy to obtain a first preset number of target log classification sets; calculating log distribution stability corresponding to the first preset number of target log classification sets; and if the log distribution stability is greater than a stability threshold and the number of sets obtained by re-clustering is greater than a first preset number, performing log early warning processing. According to the embodiment of the invention, the time delay of log early warning processing is reduced, dual verification is carried out based on the log distribution stability and the classification number obtained by re-clustering, and the accuracy of log early warning is ensured.

Description

technical field [0001] The present disclosure relates to the technical field of data processing, and in particular to a log processing method, device, device and storage medium. Background technique [0002] System logs are usually unstructured text data. Before using data mining algorithms to build model training for system logs, it is necessary to use log parsing algorithms to structure the original log data. Log analysis technology is a key part of log mining. Traditional log parsing algorithms include regular expression matching methods, source code-based methods, or rule-based methods. These methods are highly dependent on expert experience and knowledge. When new log formats appear, parsing rules tend to become inapplicable . Therefore, analyzing changes in server logs and giving timely warnings has become one of the important means for system administrators to evaluate system health. [0003] For the real-time update scenario of massive logs, the existing solution ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/35G06F16/335
CPCG06F16/35G06F16/335
Inventor 赵旭王卓周晓阳
Owner BEIJING TOPSEC NETWORK SECURITY TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com