Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data storage and data management method and device of distributed storage system

A distributed storage and data storage technology, applied in the transmission system, electrical components, etc., can solve the problem of high write delay glitch rate and achieve the effect of reducing the glitch rate

Active Publication Date: 2020-12-25
ALIBABA GRP HLDG LTD
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, there is a problem of high write delay glitch rate in the existing distributed storage system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data storage and data management method and device of distributed storage system
  • Data storage and data management method and device of distributed storage system
  • Data storage and data management method and device of distributed storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0038] This embodiment provides a data storage method of a distributed storage system, such as figure 2 shown, including:

[0039] Step 110, the client obtains information of a plurality of writable blocks of the first file and information of a plurality of data servers corresponding to each writable block in the plurality of writable blocks from the master server;

[0040] In this embodiment, the client extends the protocol used to obtain the writable block information of the file in terms of the protocol facing the master server, and adds support for a file having multiple writable blocks. The client sends a query request for the writable block information of the first file to the main control server (ie, a request for obtaining the writable block information of the first file), and obtains the writable block information of the first file. The query request may also be triggered by a user request or timing or other methods. After the client obtains the writable block info...

Embodiment 2

[0073] This embodiment relates to a data management method of a distributed storage system, such as Figure 4 shown, including:

[0074] Step 210, the master control server allocates multiple writable blocks for the first file, and allocates multiple data servers for each writable block in the multiple writable blocks;

[0075] In this embodiment, when the main control server assigns a data server to a writable block in the plurality of writable blocks, the data server that is not assigned to any writable block in the plurality of writable blocks is prioritized. Select multiple different data servers to assign to this writable block. The data servers assigned to multiple writable blocks are different, which can maintain the relative independence of each writable block operation, effectively reduce glitches, and avoid the failure of writing operations on multiple writable blocks caused by the failure of one data server. If the number of writable data servers in the system is ...

example 1

[0107] This example describes an exemplary data writing process, configuration process, etc. from a system perspective. Among them, the data writing process includes:

[0108] In the first step, the client (Client) starts, and the user configures the write data parameters of the first file as follows: configure the maximum concurrent number C of writable blocks (Chunk); and configure the time interval T for initiating write operations to multiple Chunks ;

[0109] The maximum concurrent number C of Chunk is greater than or equal to 2, and less than or equal to the number M of writable blocks of the first file.

[0110] In the second step, the Client sends a request to obtain the writable block information of the first file to the master control server (Master);

[0111] Step 3: After receiving the request, the Master finds the information of the existing Chunk in the first file and the configured number of writable blocks M in the file after receiving the request;

[0112] ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to data storage and data management methods and devices for a distributed storage system and equipment. The data storage method comprises: a client obtains information of a plurality of writable blocks of a first file from a master server and information of a plurality of data servers corresponding to each of the plurality of writable blocks; when the client performs a once-writing operation on the first file, writing processing is carried out on the plurality of writable blocks in a concurrence manner and data are written into the corresponding data servers; and if the client detects that all data servers corresponding to any of plurality of writable blocks return writing data successfully, the client determines that the writing operation to the first file is successful. Therefore, the burr rate of the single file writing operation can be reduced effectively.

Description

technical field [0001] The present invention relates to data storage, and more specifically, to a data storage and data management method, a client and a master control server in a distributed storage system. Background technique [0002] Cloud computing technology is becoming more and more popular now, and distributed storage is one of the most basic problems to be solved by cloud computing. The distributed storage system stores data on multiple physically dispersed storage nodes, manages and allocates the resources of these nodes in a unified manner, and provides users with a file access interface, which solves the problems of the local storage system in file size, file quantity, open Limitations on the number of files, etc. [0003] Typical distributed storage systems are now deployed in a three-terminal mode, such as figure 1 As shown, it usually includes the main control server, data server and client. The master control server is also called metadata server, name se...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08
CPCH04L67/1097
Inventor 刘善阳张海勇石超
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com