A method for storing log data and a log data storage system

Abstract

The invention discloses a method for storing log data and a log data storage system. The method comprises the steps of acquiring a log ID (Identity) and a log data structure which are distributed to a service log to be stored; enabling a log report agent, and interlinking a previously-obtained API (Application Program Interface) in a service program; calling the API in the service program and passing in parameters of the log ID and each field of the log data structure corresponding to the log ID; encapsulating the log data and outputting the encapsulated log data to a shared memory for cache by the API; firstly reading the log data in the shared memory, analyzing information such as the log ID, contained in the log data, and writing the read log data into a cache file according to the information by the log report agent; and reporting the data in the cache file to a log server to store by the log report agent through the network connection with the log server. According to the application of the method for storing the log data and the log data storage system, disclosed by the invention, a cross-business integrated log service system is created, thus the resource use efficiency and the reliability of log data storage are improved.

Description

Technical field [0001] The present invention relates to computer information processing technology, in particular to a method for storing log data and a log data storage system. Background technique [0002] With the rapid development of computer Internet technology, the number of services and applications on the Internet has also ushered in explosive growth. When users use these services and application products provided by operators every day, a large number of business usage records will be generated How to reliably store and collect the business usage record data generated by users using these applications, so as to facilitate the subsequent extraction of the stored business usage record data, and then use data analysis, data mining and other technologies to count and analyze the business usage records The user habits and user characteristic behaviors contained in the data are extremely important to find out valuable information for operators, so as to guide application produ...

AI Technical Summary

Problems solved by technology

However, because each client encapsulates and saves log data according to a custom protocol negotiated with the log server, there is no standardized and unified log data structure, resulting in inconsistent log data structures for logs reported to the server by different services, which requires the log server to To unify the log data structure and store the log data generated by different businesses for subsequent analysis, it needs to consume a certain amount of computing resources, making the resource utilization efficiency of the log data storage system low

Furthermore, if the log server makes an error, or temporarily fails to provide services due to log server upgrade and maintenance, etc., the transmitted log data will be lost, reducing the reliability of log data storage.

Images