A data migration method, device and system for a distributed storage system

Abstract

The invention discloses a method, device and system of data migration of a distributed type storage system. The method includes the steps that a source storage server receives data which are from a client-side and need to be written into, and the data is written into a target storage server by the source storage server when the data are determined to need to be written into the target storage server by the source storage server according to preset routing algorithm. In the implemented cases, in the process of migrating the data to the target storage server by the source storage server, the data which need to be written into are still sent to the source storage server by the client-side, the source storage server acts on behalf of the client-side in a responsibility position and stores the data onto the target storage server provided that the memory address of the data is the target storage server, and thereby the distributed type storage system can achieve smooth linear scale in real time, improve performance of the system, improve usability and reduce deployment complexity.

Description

technical field [0001] This application relates to the field of communication technology, and in particular to a data migration method, device and system for a distributed storage system Background technique [0002] In a distributed storage system (that is, data is stored in multiple independent storage servers), when the data storage requirement reaches the actual physical storage capacity, the distributed storage system needs to be physically expanded to meet the demand. The usual method of physical expansion is Add a new storage server to the storage cluster. [0003] In the prior art, the ways to realize the expansion of the distributed storage system include: [0004] (1) The expansion of the distributed storage system is realized by shutting down the storage server for maintenance, which will affect the availability of the distributed storage system. [0005] (2) Realize the expansion of the distributed storage system through HDFS (Hadoop DistributedFileSystem, dist...

AI Technical Summary

Problems solved by technology

[0010] In the above method (1), since it will affect the availability of the distributed storage system, that is, it will cause the distributed storage system to be unavailable, so it is rarely used in practical applications, especially in large-scale Internet systems

[0011] The above method (2) is the current mainstream distributed storage architecture, but it depends on the Namenode. Every read and write access needs to visit the Namenode, which increases the overhead and reduces the performance; when the Namenode fails, it will cause the entire distribution. The distributed storage system is unavailable; further, if Namenode adopts a highly available distributed solution, it will make the distributed storage system more complicated

[0012] In the above method (3), it is necessary to add a proxy layer for routing selection, which affects system performance; and when migrating data, it is necessary to modify the configuration relationship between storage data and storage nodes, which is complex in logic and may cause load imbalance

Images