An Erasure Coding Method with Low Data Reconstruction Overhead

Abstract

The invention discloses an erasure code method with a low data reconstruction overhead. The method comprises the following steps: a data block partition step, a data coding step, a step of checking the data block state, a step of repairing and partitioning based on a failure block and a corresponding failure repair step. The method combines the excellent repair bandwidth property of Butterfly Codes and an intra-group reconstruction method for local reconstruction in network coding, so that the required disk reading and network bandwidth transmission during data reconstruction in a system can be reduced. In the erasure code method, each group has two check blocks, and intra-group coding of Local Reconstruction Codes is used to maintain the optimal repair bandwidth in the group, and then thesystem reliability is guaranteed by using partial check blocks to encode all data blocks. Compared with the Local Reconstruction Codes, the intra-group checking of the method has higher reliability,so that the number of required global check is less; therefore, the method has lower repair bandwidth under the equal storage overhead.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more particularly relates to an erasure code method with low data reconstruction overhead. Background technique [0002] Erasure coding is an encoding method to ensure data redundancy. It first splits the original data into equal-sized data blocks, and then encodes the data blocks into check blocks. When several data blocks or check blocks are lost, the erasure code technology can ensure that the original data can still be recovered. This technology is widely used in distributed storage systems and cloud storage systems to improve system reliability and prevent data inaccessibility caused by disk failure or data loss. [0003] In the traditional encoding method, n is the total number of data blocks and check blocks, k is the number of data blocks, and m=n-k is the number of check blocks. First, the original data of size M is equally divided into k data blocks: D 0 、D 1 ,...,D k-...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the above defects or improvement needs of the prior art, the present invention provides an erasure code method with low data reconstruction overhead, thereby solving the problem that the existing partial reconstruction code has a relatively large storage overhead and the butterfly code can only have two A technical problem with relatively large limitations due to a check block

Images