Method and device for storing data objects

Abstract

The invention provides a method and a device for storing data objects. The method includes the following steps that when the data object writing request is received, data of the data objects is divided into first partial data and second partial data, wherein the first partial data includes data fields of the data object, and the second partial data includes metadata of the data objects; the first partial data is wrote and stored in a first data file of a first external storage device; the location information, in the first external storage device, of the first partial data and the second partial data are wrote and stored in a second data file of a second external storage device. According to the method and the device, data required by retrieval of the data objects and high in access frequency is loaded into a cache region, the cache space occupied by each data object can thus be effectively reduced, and the data retrieval speed is ensured.

Description

technical field [0001] The present invention relates to the technical field of data storage, and more specifically, to a method and device for storing data objects. Background technique [0002] Existing databases or file systems usually use B / B+ trees, hash tables, skip tables, etc. as their cache management methods, and data is usually associated with these structures in the form of objects. Taking the B / B+ tree as an example, each data object will be stored in each tree node. In the B+ tree, the data information is moved to the leaf node, and the internal nodes of the B+ tree are only used to store the relevant Key value range. Because the data nodes are large, they usually take up a lot of memory. When performing big data analysis, it often leads to insufficient memory. At this time, the database or file system needs to use the relevant replacement algorithm (for example, LRU (Least Recently Used) algorithm) ) Discard the memory data page. After the memory page is disca...

AI Technical Summary

Problems solved by technology

[0003] That is, the existing way of storing data objects makes the data objects loaded into the cache area (for example: the leaf nodes of the B / B+ tree) consume too much cache space, thereby swallowing a large part of the memory, and insufficient memory leads to: There are many cache misses in data retrieval, pages are frequently swapped in and out, the memory structure is frequently adjusted, and the system spends a lot of CPU time processing page replacement work, so that customer requests and I / O requests cannot be processed in a timely manner

Moreover, it also requires the database or file system to perform complex and time-consuming operations on cache management, reducing the performance of the database or file system

Images