Quick data verification method, device and system for cloud storage system

A cloud storage system and data verification technology, applied in the field of cloud storage, can solve problems such as high error risk and slow verification speed

Active Publication Date: 2022-03-08
深圳市木浪云科技有限公司
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Embodiments of the present invention provide a fast data verification method, device and system for a cloud storage system, to at least solve the problem that data needs to go through a longer path than a traditional file system when reading a file from a distributed file system client. The technical problems caused by slow verification speed and high risk of error

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Quick data verification method, device and system for cloud storage system
  • Quick data verification method, device and system for cloud storage system
  • Quick data verification method, device and system for cloud storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0039] According to an embodiment of the present invention, a fast data verification method of a cloud storage system is provided, such as image 3 As shown, the method includes:

[0040] Step S302, receiving a sample verification instruction;

[0041] Step S304, extract the data files required to verify the target file from the sampling verification instruction, obtain the data sets that need to be sampled based on the extracted data files, form the metadata of the data sets that need to be verified, and analyze the data files that need to be verified. Describe the metadata of the data set that needs to be verified;

[0042] Wherein, the extracted data file includes: the target file, a check code file corresponding to the target file, a sampling verification ratio, wherein, the check code file includes a check algorithm, a check code length, a checksum Check code array length, check code array and check code.

[0043] In an exemplary embodiment, based on the extracted data...

Embodiment 2

[0054] According to an embodiment of the present invention, another fast data verification method for a cloud storage system is provided, such as Figure 4 As shown, the method includes:

[0055] Step S402, initiation of the verification task;

[0056] The step of initiating the verification task is that the user sends a sample verification command through the interface of the backup module to sample check whether there is a data error in a disk file of a virtual machine.

[0057] In an exemplary embodiment, combining Figure 5 , the initiation of the verification task may include the following steps:

[0058] 1) The user application initiates a sample verification command to the backup module;

[0059] 2) After receiving the sampling verification instruction, the backup module generates a fixed-format sampling verification task plan file according to the verification requirements of the backup data, that is, the data file required for verifying the target file. The genera...

Embodiment 3

[0087] According to an embodiment of the present invention, another fast data verification method of a cloud storage system is provided, such as Figure 5 As shown, the method includes:

[0088] Step S502, initiate a verification task;

[0089] The step of initiating the verification task is that the user sends a sample verification command through the interface of the backup module to sample check whether there is a data error in a disk file of a virtual machine.

[0090] Step S504, preprocessing the verification task;

[0091] The file system service of the storage system reads the verification task plan file to obtain the necessary information for data verification, including the target file, the corresponding verification code file of the target file, and the sampling verification ratio. The file system of the storage system calls the data sampling module of the metadata service cluster through the sampling verification interface module, and sends the relevant informatio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a quick data verification method, device and system for a cloud storage system. The method comprises the steps that the method is executed in a cloud storage system, and the method comprises the steps that a sampling verification instruction is received; extracting a data file required for checking a target file from the sampling checking instruction, obtaining a data set needing to be sampled based on the extracted data file, forming metadata of the data set needing to be checked, and analyzing the metadata of the data set needing to be checked; dividing a data verification process into a plurality of verification subtasks according to an analysis result of the metadata and the number of service nodes of data blocks of the target file; and executing the plurality of verification subtasks in parallel to verify the data. According to the method and the device, the technical problems of relatively slow verification speed and relatively high error risk caused by the fact that data needs to experience a longer path than that of a traditional file system when a file is read from a client of the distributed file system are solved.

Description

technical field [0001] The present invention relates to the field of cloud storage, in particular to a fast data verification method, device and system for a cloud storage system. Background technique [0002] When computer systems transmit and store data, errors will inevitably occur. In order to avoid this kind of error, on the one hand, start from the hardware to improve the anti-interference ability and reliability of the hardware; on the other hand, take code correction measures in data coding, so that the machine can find errors and even correct them by itself. This kind of data coding with error detection or automatic error correction capability is usually called data check code. The principle is to add some check bits to the data to form a data check code, and to judge whether there is an error or to correct it by checking the validity of the data check code. Commonly used data check codes include parity check code, Hamming check code, cyclic redundancy check code ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F11/10G06F16/182G06F3/06
CPCG06F11/1004G06F16/182G06F3/0608G06F3/067G06F3/0641
Inventor 陈元强蔡涛吴健辉
Owner 深圳市木浪云科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap