Method for reducing overall overhead of cloud storage in the presence of data dependence relationship

Abstract

The invention discloses a method for reducing the overall overhead of cloud storage in the presence of a data dependence relationship. The method comprises two parts including a data storage decision model and a data storage strategy decision algorithm. In the data storage decision model, a decision variable of the model is a data storage strategy, and comprises non-storage, multi-replica storage and erasure code storage; a constraint condition of the model is that data availability requirements satisfy a given availability index; an optimization target of the model is that the overall overhead of the system is minimized, wherein the data overall overhead can simultaneously comprise storage overhead and calculation overhead; in the calculation overhead, the data generation time is a random variable; an expected calculation method is given in the invention; in the data storage strategy decision algorithm, when new data is generated, the data storage strategy is directly decided; and, when each decision period is ended, the data storage strategy in a communication branch is updated by respectively using a genetic algorithm or a traversal pruning algorithm in manners of taking the communication branch in a dependence relationship graph as a unit and judging whether the node number of the communication branch is greater than a threshold value or not. Compared with the prior art, the method provided by the invention has the advantage that: on the premise that the data availability is ensured, the overall overhead of the system is reduced.

Description

technical field [0001] The invention relates to the field of cloud storage, in particular to a method for reducing the overall cost of cloud storage when data dependencies exist. Background technique [0002] With the rapid development of cloud storage, more and more applications store data in the cloud, how to reduce the storage cost of massive data in the data center has become an important issue. [0003] In cloud storage, there may be dependencies between data. For example, video files with different resolutions can be generated by transcoding the original video file. At this time, there is a dependency relationship between the original file and the transcoded file. [0004] At present, the solution to reduce the overall cost of cloud storage when data dependencies exist is to use an algorithm to decide whether data needs to be stored. At this time, the data is either not stored or stored in multiple copies with a fixed number of copies. For data that is not stored, wh...

AI Technical Summary

Problems solved by technology

[0005] However, there are still two deficiencies in the existing solutions: (1) when storing data, the data storage strategy is fixed

No further consideration is given to the variable data storage strategy

(2) The problem of data unavailability is not considered when the data generation time exceeds the user's allowable response delay time

Images