Method and device for quickly verifying consistency of multi-duplicate storage

Abstract

The invention provides a method and a device for quickly verifying the consistency of multi-duplicate storage. The method includes S1, judging data writing volumes of head files when systems are in idle states, creating snapshot files by the aid of I / O (input / output) redirection technologies when the volumes of data written into the head files reach 30% of the volumes of integral spare files at least, utilizing new head files with continuously written data as null comparison sparse files, computing hash values of the snapshot files and storing the hash values of the snapshot files as extended attributes; S2, carrying out snapshot operation on the head files once at intervals of preset time and comparing hash values of the new head files to the hash values of the snapshot files when the consistency of multi-duplicate data need to be timely verified. The method and the device have the advantages that characteristics of distributed block storage are combined with one another, the method for quickly detecting the consistency of multiple duplicates on the basis of snapshot technologies is proposed, hash value computation tasks are divided to be carried out in multiple time slots, accordingly, influence on normal reading and writing tasks of the systems can be prevented, and the multi-duplicate storage consistency verification speeds can be increased.

Description

technical field [0001] The invention relates to the technical field of distributed data storage and data verification, in particular to a multi-copy storage fast verification consistency method and a device thereof. Background technique [0002] Most of the existing distributed storage systems use multi-copy technology to improve system reliability, availability, performance and scalability. [0003] Multi-copy technology can ensure that when individual hosts and disks in the distributed storage system fail, the system can still provide normal read and write services, improving system reliability, and multiple copies can be used for shunting and speeding up response time. However, the distributed storage system communicates through the network, and the instability of the network may easily cause inconsistency of the back-end data, and the distributed storage system generally includes a large number of server hosts and disks, and the probability of hardware damage is relative...

AI Technical Summary

Problems solved by technology

But for large files, calculating the hash value will consume a lot of CPU and disk bandwidth, seriously affecting the performance of the system

File inconsistencies are often relatively rare, but calculating the file hash value requires reading the contents of the entire file, resulting in a huge waste of resources

Images