Method for adjusting distribution of data fragments and data server

Abstract

The invention discloses a method for adjusting distribution of data fragments. The method comprises the following steps of: when an adjustment moment corresponding to a to-be-processed data fragment arrives, obtaining access frequency information of the to-be-processed data fragment; determining earning data of each data center corresponding to the data fragment; and finally generating optimal distribution information according to the earning data of each data center and the quantity of copies, and adjusting the position, at each data center, of each copy according to the optimal distribution information. According to the method, the distribution condition of data fragments can be dynamically optimized according to the access frequencies and characteristics of the data fragments without additionally arranging an internal memory or a hard disk for storage, so that the requirement for transmission bandwidth between the data center is reduced.

Description

technical field [0001] The invention relates to the field of communication technology, in particular to a method for adjusting the distribution of data slices. The invention also relates to a data server. Background technique [0002] In the environment of cloud computing and big data processing, the distributed file system serves as the underlying storage layer and provides nearly infinitely scalable storage service capabilities to the upper-layer business. However, with the increase in the scale of data centers and the popularity of global deployment, there are more and more accidents where data centers go offline due to various physical problems (such as cutting optical fibers, fires in computer rooms, etc.). Line, will cause serious service availability problems. [0003] In order to improve the availability and continuity of data services, a common practice in the industry is to distribute multiple copies of data to multiple data centers in a certain area, and improve...

AI Technical Summary

Problems solved by technology

However, since the cached data is placed in the memory, which is several orders of magnitude smaller than the capacity of the disk memory (TB vs GB), the effect of the cache will decrease as the amount of data increases, and the cache system puts the cache in The hard disk will occupy the available IO capacity of user data. In addition, it is difficult to cooperate with the cache and the underlying file system.

For example, rewriting the data of a certain file will invalidate the cached data of the entire file, which will affect the efficiency of the cache.

Images