Data processing method and device, electronic equipment and readable storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a readable storage medium. The method comprises the steps that when a random write request for first target data isreceived, determining a first target storage unit used for storing the first target data, and writing the first target data into the first target storage unit in a cache region; and when it is detected that the second target storage unit in the local cache region meets the full stripe requirement, transferring the data in the second target storage unit to the storage region of the cloud storage system. The method can improve the read-write performance and the system stability of the cloud storage system.

Description

technical field [0001] The present application relates to distributed data storage technology, and in particular to a data processing method, device, electronic equipment and readable storage medium. Background technique [0002] A distributed cloud storage system generally supports random write operations on storage files stored in a standard file operating system using a standard pwrite interface. In order to avoid reading the entire object file data from the disk during random writing, the general processing method is to store the object file in fragments (such as fragmentation of EC (Erasure Coding, erasure code) stripes). [0003] Compared with file multi-copy storage, EC technology has higher storage space utilization and high reliability, but the encoding method of EC technology is more complicated. For scenarios with frequent random writes, each random write needs to be read from the disk The original data of the entire stripe, and integrate the data to be written w...

AI Technical Summary

Problems solved by technology

[0003] Compared with file multi-copy storage, EC technology has higher storage space utilization and high reliability, but the encoding method of EC technology is more complicated. For scenarios with frequent random writes, each random write needs to be read from the disk The original data of the entire stripe, and integrate the data to be written with the original data in the condition, recalculate the check data, and rewrite the new stripe to the disk

[0004] It can be seen that frequent random writing services will trigger the cloud storage system to perform frequent EC calculations and data reading and writing, which will affect the CPU (Center Process Unit, central processing unit), memory, and disk IO (Input / output, input / output) of the cloud storage system. Output) and other hardware resources consume a lot, which affects the read and write functions and system stability of the cloud storage system

Images