A method for aggregation optimization of time series data

Abstract

A method for aggregation optimization of temporal data includes dividing a temporal database file into a data block file and a block index file; scanning the block index file according to the calculated start and end time period, extracting the data blocks that meet the time period condition and other filtering conditions, and sorting according to the offset of each data block in the data block file; scanning the data files inexpensively by sorted data blocks, after the calculation of each data block, gathering the calculation results. The method can be completed by opening a data file for scanning whether it is for reading a single time series data or for aggregating multiple time series data, which greatly improves the overall performance.

Description

technical field [0001] The invention relates to the field of data processing, in particular to an aggregation optimization processing method for time series data. Background technique [0002] Time series data refers to time series data, which is a data column recorded in chronological order. There is a special time series database to handle the storage and query of time series data. [0003] In the prior art, a time-series database usually contains multiple time-series data. In addition to reading a single time-series data, it is usually necessary to aggregate and calculate multiple time-series data. The usual practice is to process each time series separately, get the results, put them in the cache, and then aggregate the results. [0004] However, compared to the need to scan the database file once for each time series, multiple time series need to open and scan the database file multiple times, which will greatly increase the number of disk IOs and affect system perform...

AI Technical Summary

Problems solved by technology

[0004] However, compared to the need to scan the database file once for each time series, multiple time series need to open and scan the database file multiple times, which will greatly increase the number of disk IOs and affect system performance.

Images