Pre-calculation result generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a pre-calculation result generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a feature vector set, the feature vector set being determined based on a dimension combination queried by a query statement within a preset time; inputting the feature vector set into a query popularity model, obtaining the score of each feature vector in the feature vector set, and obtaining the query popularity model based on data table static features and historical query statements through pre-training; determining an expected score value of each feature vector according to the score of each feature vector; and based on the expected score value of the feature vector, screening out a target feature vector meeting the query expectation from the feature vector set, and performing pre-calculation based on the target feature vector. According to the embodiment of the invention, the calculation efficiency of the pre-calculation result is high, the number of calculation results is small, less storage space is occupied, and the storage overhead is reduced.

Description

technical field [0001] The invention relates to the technical field of multidimensional analysis of big data, in particular to a method, device, electronic equipment and storage medium for generating precalculation results. Background technique [0002] With the increasing amount of data, ad hoc multidimensional analysis of big data is becoming more and more valuable. However, the timeliness of the conventional distributed computing framework is difficult to meet the ad hoc multidimensional analysis. However, based on the computing power and storage capacity of the big data platform, the method of exchanging space for time and pre-computing metrics can meet the needs of big data ad hoc multi-dimensional queries. However, it saves all query conditions, dimension combinations, metric value calculation results, and calculation process results, resulting in a large data capacity of the pre-computation results. After a long period of time, the data capacity overhead of the pre-c...

AI Technical Summary

Problems solved by technology

However, it saves all query conditions, dimension combinations, measurement value calculation results, and calculation process results, resulting in a large data capacity of pre-computation results. After a long period of time, the data capacity overhead of pre-calculation results is getting larger and larger.

In addition, some pre-calculated results are not or rarely queried by users, but these pre-calculated results occupy a large amount of storage space, resulting in a waste of storage space

Images