Storage capacity balancing method and device and object storage method and device

Abstract

The invention discloses a storage capacity balancing method and device and an object storage method and device. The storage capacity balancing method comprises the steps of forming a hash ring, wherein the hash ring is composed of N hash values; dividing the hash ring into M equal parts, wherein each equal part of the hash ring forms a virtual node; allocating X virtual nodes to the to-be-distributed storage nodes according to the real-time capacity information of the storage nodes; hashing the X virtual nodes onto a hash ring; forming a hash index record, wherein the Hash index record comprises VNode _ ID of virtual nodes, corresponding Hash values and OSD _ IDs of the to-be-distributed storage nodes; and repeatedly distributing the virtual nodes to all the storage nodes in the storage system, hashing the virtual nodes and forming a hash index record. When the virtual nodes are allocated to the storage nodes, the real-time capacity information of the storage nodes and all the storagenodes in the storage system is referred to, the write-in amount of each storage node is automatically balanced, and the capacity balance of each storage node is ensured to the maximum extent.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a storage capacity equalization method, an object storage method and a device. Background technique [0002] Object storage is a storage system based on objects as the basic unit of storage. In order to meet the massive object storage requirements, a distributed storage system is generally composed of multiple storage nodes (Object Storage Device, referred to as: OSD) to provide high-performance storage externally. Write and query capabilities. [0003] When faced with the storage requirements of a large number of small files, such as storing pictures collected by video capture devices, the industry usually adopts the method of writing hashes, that is, calculating hash values ​​​​based on object IDs. Different hash values ​​correspond to different storage nodes. In this way, objects can be evenly written to each storage node in quantity. [0004] However, due to the huge n...

AI Technical Summary

Problems solved by technology

[0004] However, due to the huge number of objects and different sizes, ranging from tens of KB (=210 bytes) to several MB (=220 bytes), the storage capacity occupied by the same number of objects on each storage node varies greatly. It is easy to cause the capacity of storage nodes to be unbalanced, causing some storage nodes to withdraw from service due to full capacity, which in turn reduces the overall throughput of the system

Images