Log-based object storage additional writing implementation method

Abstract

The invention discloses an object storage additional writing implementation method based on logs. The method comprises the steps that a client sends an additional writing request; an object storage Web end processing layer checks the legality of the request parameters; an object storage processing layer performs object legality check, and judges whether a target object exists or not if the check is passed; if the target object exists, the metadata of the target object is analyzed, and to-be-added data is written in a corresponding mode according to the type of the target object; if the targetobject does not exist, the strip information of the target object is initialized, and to-be-added data is written by taking a new strip as a starting position; the metadata is written into a head object of the target object; after metadata updating is completed, the object storage system returns response information to the client. On the premise that the original object file does not need to be uploaded repeatedly, the data additional synthesis is achieved, and meanwhile, logs are recorded during data uploading to avoid data loss caused by accidental interruption.

Description

technical field [0001] The present invention relates to the field of distributed object storage systems, in particular to a log-based Ceph (a unified, distributed file system designed for excellent performance, reliability and scalability) object storage appending and writing implementation method. Background technique [0002] With the development of Internet of Things, cloud computing, big data, and 5G technologies. The rapid development of medical, telecommunications, Internet, government and enterprise, and other industries has led to the rapid growth of massive data, which poses great challenges to traditional storage systems. As an emerging storage technology, object storage is widely used in more and more industries. . Compared with the traditional file system, object storage abandons the complex directory and semantic design of the file system, stores data in a flat key-value manner, greatly simplifies the complexity of metadata management, and has almost no limit o...

AI Technical Summary

Problems solved by technology

However, due to the limitations of the current object storage IO (input and output) characteristics, it does not support additional write operations and only allows overwrite write operations. Therefore, object storage urgently needs a new IO processing method that can support users to read and edit at the same time. Cater to the constant changes in the market and strengthen the competitiveness of the market

Images