Cloud data migration method

Abstract

The invention discloses a cloud data migration method. The cloud data migration method comprises the following specific steps: dividing one file into one or more blocks to store, and distributing and storing on a cluster server by taking the block as a unit, wherein each block is an independent storage unit; taking each block as one partition, and taking the partition as a basic unit for data migration and load monitoring; respectively classifying all storage nodes in an immigration node set and an emigration node set according to the relationship of standardized load values and the total node number of a system; and achieving load balance of the system among various nodes in a cloud environment through a data migration algorithm. Compared with the prior art, the cloud data migration method disclosed by the invention has the advantages that: a data migration strategy is made on the basis of the balance degree; an optimal data migration operation is selected; therefore, the load balance of the system is achieved; the load skew is reduced; and the cloud data migration method is high in practicability and easy to popularize.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a highly practical cloud data migration method. Background technique [0002] Massive data storage with high reliability and scalability is a huge challenge for Internet companies. Traditional databases are often difficult to meet this demand, and most of the searches for specific systems are based on primary keys. In this case, using a relational database will make the efficiency low, and expansion will become a big problem in the future. In such cases, using storage will be a good choice. For storage systems deployed in cloud environments, data migration is a key technology to realize node dynamic expansion and elastic load balancing, mainly including migration plan formulation, routing information synchronization, user request forwarding, and data consistency management. . A large amount of state synchronization accompanying the data migration process will have a certa...

AI Technical Summary

Problems solved by technology

2) The data migration strategy based on the greedy method is mainly aimed at the data migration problem of the Key / Value storage system. The basic idea is to use statistical methods to monitor hotspot partitions online, and preferentially migrate some data of hotspot partitions to neighboring nodes with lighter loads. In order to simplify the complexity of the migration operation, the hash algorithm will maintain the order of the Key values. The main shortcoming of these two parts is that the migration overhead is not considered

Although most of the existing research work focuses on data migration in non-virtualized environments, these methods are often not applicable to cloud storage systems.

Images