Time series data storage method and device

Abstract

The invention discloses a time series data storage method, which comprises the steps of processing to-be-stored time series data into a defined data structure, and assembling the defined data structure into storage data; wherein the data structure at least comprises a measurement name, a label key, measurement and a time precision field of the time series data; wherein the warehouse-in data takesa mapping code at least comprising a measurement name mapping code, a timestamp and a label value included in a label key as a row key in the data point and takes a time difference relative to the timestamp as a limiter in the data point, and the time difference is related to time precision; and writing the storage data into a time sequence database. According to the method and the device, the label data which can be contained in the row key in the data point is improved, the multi-value storage of the time series data is realized, and the storage pressure caused by splitting the time series data is avoided.

Description

technical field [0001] The present invention relates to the field of time-series databases, in particular to a method for storing time-series data. Background technique [0002] Time series data is a series of data based on time, and data with time tags is also called time series data. [0003] Time series data is mainly collected and generated by various types of real-time monitoring, inspection and analysis equipment in the power industry, chemical industry, etc. The typical characteristics of these industrial data are: fast generation frequency, each monitoring point can generate multiple pieces of data; heavily dependent on the collection time, each piece of data requires a unique time; there are many measuring points and a large amount of information, and the conventional real-time monitoring system has thousands of monitoring points, and the monitoring points generate data every second. Dozens of gigabytes of data are generated every day. [0004] see figure 1 as sh...

AI Technical Summary

Problems solved by technology

This method effectively reduces the problem of repeated attribute storage of the same data, but the writing performance of the data after splitting cannot meet the demand, and the efficiency of associated query is low

For example, in the process of equipment operation and maintenance, there are 100 pieces of data reported every 5 minutes by a million-level equipment, and the writing performance of storage needs to reach a very high writing speed. In this case, the data query work cannot be performed.

[0013] In another storage method, a secondary index is performed on the basis of the primary index of the label field based on the data storage structure and method, but this method essentially only reduces the row key (RowKey) of the data point, and does not modify the data. The query is effectively improved, which essentially fails to solve the current data writing pressure

Images