Stream data-oriented quick indexing method, streaming data-oriented quick indexing device, streaming data-oriented quick indexing equipment and storage medium

Abstract

The invention discloses a streaming data-oriented quick indexing method. The method comprises the following steps: receiving sent data tuples, organizing the data tuples into an indexing structure, and periodically writing the data tuples into a distributed storage data block; carrying out layout on leaf nodes of a B + tree, wherein the layout comprises layout of an index array and layout of a data array; obtaining a query request, obtaining the offset of data tuples satisfying the query request through binary search in the data tuple of the leaf node of the B + tree according to the query request, and obtaining a target data tuple in the data tuple according to the offset; carrying out boundary updating on the B + tree; and performing priority ranking on the target data tuple by using a preset algorithm. High-speed insertion of data can be realized, and the real-time retrieval capability can be improved.

Description

technical field [0001] The present invention relates to the field of indexing, in particular to a fast indexing method, device, equipment and storage medium for flow data. Background technique [0002] The Internet of Things and smart devices are an extremely important part of building a smart city. And with the explosive growth of data generated by IoT sensors and smart devices with location information, the demand for high-speed insertion and real-time retrieval of data is also increasing. The high-speed insertion and real-time retrieval of data is also an extremely important data processing capability for supporting smart manufacturing and smart cities, and system users can quickly find the required historical data and new data. [0003] At present, the performance of existing distributed databases is not good. The key-value storage represented by HBase does not provide a good time range query, and the time series database represented by Druid lacks information other tha...

AI Technical Summary

Problems solved by technology

[0003] At present, the performance of existing distributed databases is not good. The key-value storage represented by HBase does not provide a good time range query, and the time series database represented by Druid lacks information other than time. second dimension index

In the research and practice of the prior art, the inventors of the present invention have found that the distributed database mainly faces the following problems: (1) cannot provide high-speed insertion of stream data; (2) comparison of query delay on key and timestamp high

Images