Memory database storage engine management method

Abstract

The invention particularly relates to a memory database storage engine management method. According to the memory database storage engine management method, on the basis of a Memtable management mechanism of a RocksDB storage engine, an ART index is newly built in the Memtable to replace a skiplist index, a key-value is inquired by adopting a double-index mechanism of the ART index and a Hash index, and the reading performance of the RocksDB storage engine is optimized, so that a RocksDB memory engine mode is realized, disk reading and writing are reduced, and the inquiry speed is increased. According to the memory database storage engine management method, the read performance problem of a RocksDB-based storage engine in an existing distributed NewSQL database is optimized, the efficient read-write performance of the memory is fully played, more data can be cached in the memory, and the read amplification problem of the RocksDB is solved.

Description

technical field [0001] The invention relates to the technical field of database management, in particular to a method for managing a memory database storage engine. Background technique [0002] In recent years, as the capacity of dynamic random access memory (DRAM) has increased and the unit price has decreased, it has become possible to store and process large amounts of data in memory. Compared with disk, the data read and write speed of memory is several orders of magnitude higher. Saving data in memory can greatly improve the performance of applications than accessing it from disk. [0003] CockroachDB is an open source distributed database. It has the ability of NoSQL to store and manage massive data, and maintains the ACID and SQL supported by traditional databases. It also supports cross-region, decentralization, high concurrency, strong consistency of multiple copies, and high availability. and other characteristics. Supports OLTP scenarios and lightweight OLAP sc...

AI Technical Summary

Problems solved by technology

The price in exchange is read performance and increased merge overhead: the Memtable of the RocksDB single-column family has a default size of 64MB. When this size is exceeded, the written data will be converted from Memtable to immutable Memtable, and then a background thread will be started to regularly write the data written into immutable Memtable. Perform Compaction operations to reclaim resources; add immutable Memtable to the flush pipeline and wait for the background thread to flush to the SST, resulting in read amplification problems. To query the Value value corresponding to a Key value, it may be necessary to traverse multiple SSTable files , corresponding to multiple random IO

And the space is enlarged, and the multi-version data will take up more storage space before merging

[0006] In order to optimize the read amplification problem based on the RocksDB storage engine in the existing distributed NewSQL database, and optimize the read performance problem of the memory database based on the ART algorithm, the present invention proposes a management method for the memory database storage engine

Images