Log processing method, device, equipment and storage medium

Abstract

The present application provides a log processing method, apparatus, apparatus and storage medium, wherein the method comprises: storing a plurality of logs into N log groups by obtaining a pluralityof logs in a source end in a cyclic overlay manner; and storing the plurality of logs in a cyclic overlay manner. Obtaining the change data of each log group in the P log groups when determining the data change of the logs of the P log groups in the N log groups, wherein P is less than or equal to N; parsing The change data of each log group in the P log groups in parallel to obtain the parsed data corresponding to each log group in the P log groups. The number of logs can be reduced by storing the logs in the log group in the way of cyclic overlay, and no extra space is needed to store the logs. Parallel analysis of the log in the log group to get the parsing data of the change data can improve the efficiency of log parsing, which is conducive to rapid processing and storage of the parsedlog.

Description

technical field [0001] The present application relates to computer technology, and in particular to a log processing method, device, equipment and storage medium. Background technique [0002] With the development of computer technology, a large number of logs will be generated during data processing. The log needs to be parsed, and then the parsed data is sent to the receiving end, for example, stored in various databases. [0003] In the prior art, the log generation method is an incremental process. If the log is modified, a modified log will be generated, thereby generating a large amount of data, which needs to be stored. Then, the logs are parsed one by one to obtain the parsed data, and then the parsed data is sent to the receiving end. [0004] However, in the prior art, when parsing logs, the stored logs are parsed sequentially. This way of parsing logs one by one has low parsing efficiency, which is not conducive to fast processing and storage of parsed logs. C...

AI Technical Summary

Problems solved by technology

[0005] The present application provides a log processing method, device, device, and storage medium to solve the problem of low parsing efficiency in the prior art, which is not conducive to fast processing and storage of parsed logs

Images