Data storage structure conversion method and apparatus

Abstract

The present invention provides a data storage structure conversion method. The method comprises: extracting a first data unit and a second data unit in to-be-converted data based on an original storage structure; combining the first data unit and the second data unit to determine a key in a converted-into storage structure; then based on the original storage structure of the to-be-converted data, determining a value in the converted-into storage structure according to a value setting rule; and according to the converted-into storage structure, storing a data key-value pair consisting of the key and the value into a key-value database. According to the method provided by the present invention, the data based on the original storage structure is converted into and stored as the data based on the converted-into storage structure, so as to store various types of data by using a uniform converted-into storage structure. Meanwhile, with relatively high data storage processing efficiency being ensured, a storage mode limited by a physical memory is converted into a storage mode in which a magnetic disk is the destination of data storage and that is not limited by a physical memory.

Description

technical field [0001] The present invention relates to the technical field of databases, in particular, the present invention relates to a data storage structure conversion method and device. Background technique [0002] With the rapid development of the information age, not only large enterprises, but even some small enterprises have accumulated terabytes of data. Organizations of all sizes have begun to have the need to deal with large-scale and multiple types of data, while the traditional relationship SQL databases (SQL databases) are almost at their limits in terms of data storage scalability. In order to solve the challenges brought by large-scale data collections and multiple data types and the problems of big data applications, non-relational databases (NoSQL databases) emerged as the times require. [0003] Commonly used non-relational databases include KV (Key-Value, key-value) storage databases, where data is organized, indexed, and stored in the form of key-va...

AI Technical Summary

Problems solved by technology

The KV storage method is very suitable for data storage services that do not involve too many data relationships and business relationships. It can effectively reduce the number of reads and writes to the disk, and has better read and write performance than SQL database storage. The KV storage database can be implemented including Redis And LevelDB, etc., LevelDB can support billion-level data volume, and can maintain very high data storage and processing performance at this level. At the same time, LevelDB can store data to disk, which is conducive to permanent storage of data; however, currently The interface of LevelDB only supports key-value (key-value pair) data without deformation, the interface is relatively single, and the scalability to business is poor

Redis is a high-performance key-value storage system that supports a variety of value-type data storage structures, including string (string) type, list (linked list) type, set (set) type, zset (ordered set) ) type and hash (hash) type, however, in order to ensure high data storage and processing efficiency, its data is cached in memory, that is, Redis is an in-memory database, so this storage method is limited by physical memory and cannot Realize big data storage

Images