Data storage method and device based on distributed system and storage node

A distributed system and storage node technology, applied in the field of storage nodes and data storage based on distributed systems, can solve problems such as the inability to guarantee the successful storage of the data to be stored, the failure of the storage of the data to be stored, etc.

Pending Publication Date: 2021-04-27
BIGO TECH PTE LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] At present, distributed systems usually use data sharding to store the primary copy of the same sharded data on different storage nodes, so as to realize disaster recovery of data storage by increasing the number of copies, so that each data to be stored All can be successfully stored in the normal storage node designated when the primary copy is stored under the data fragment corresponding to the data to be stored in the distributed system, but the multi-copy storage in the distributed system will bring a lot of additional storage performance overhead, and When all the storage nodes designated by the primary copy under a certain data shard fail, the storage of the data to be stored corresponding to the data shard will also fail, so that it is impossible to guarantee the storage of each data to be stored in the distributed system. successfully stored

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data storage method and device based on distributed system and storage node
  • Data storage method and device based on distributed system and storage node
  • Data storage method and device based on distributed system and storage node

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0028] Figure 1A It is a flow chart of a data storage method based on a distributed system provided in Embodiment 1 of the present invention. This embodiment can be applied to the situation of storing various data in any distributed system. The data storage method based on the distributed system provided in this embodiment can be executed by the data storage device based on the distributed system provided in the embodiment of the present invention. The device can be realized by software and / or hardware, and integrated in the In the storage node of the method, the storage node may be each distributed node in the distributed system.

[0029] Specifically, refer to Figure 1A , the method may include the following steps:

[0030] S110, if the original storage table fragment pointed to by the data to be stored does not have an available storage node in the distributed system, select a corresponding target storage table fragment from surviving storage table fragments with availabl...

Embodiment 2

[0039] Figure 2A It is a flowchart of a data storage method based on a distributed system provided in Embodiment 2 of the present invention, Figure 2B It is a schematic diagram of the principles of the process of reading and writing data in the distributed system provided by Embodiment 2 of the present invention. This embodiment is optimized on the basis of the foregoing embodiments. Specifically, such as Figure 2A As shown, in this embodiment, the specific storage process of the data to be stored and the specific reading process of the stored data in the distributed system are mainly explained in detail.

[0040] optional, such as Figure 2A As shown, the following steps may be included in this embodiment:

[0041] S210, determine whether the original storage table fragment pointed to by the data to be stored has an available storage node in the distributed system, if yes, perform S250; if not, perform S220.

[0042] Optionally, in order to ensure the accuracy of data...

Embodiment 3

[0054] Figure 3A It is a flowchart of a data storage method based on a distributed system provided in Embodiment 3 of the present invention, Figure 3B It is a schematic diagram of the principle of the process of judging whether the storage node specified by the storage table fragmentation is available in the method provided by the third embodiment of the present invention. This embodiment is optimized on the basis of the foregoing embodiments. Specifically, such as Figure 3A As shown, this embodiment mainly explains in detail the specific judgment process of judging whether the original storage table fragment pointed to by the data to be stored has an available storage node in the distributed system.

[0055] optional, such as Figure 3A As shown, the following steps may be included in this embodiment:

[0056] S310. Periodically detect the working status of each storage node in the distributed system, and determine whether the storage node pointed to by each storage ta...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a data storage method and device based on a distributed system and a storage node. The method comprises the following steps: if an available storage node does not exist in a distributed system in an original storage table fragment pointed by to-be-stored data, selecting a corresponding target storage table fragment from survival storage table fragments with the available storage node; and storing the to-be-stored data through available storage nodes existing in the distributed system through the target storage table fragments. According to the method provided by the embodiment of the invention, comprehensive disaster tolerance of data storage in the distributed system can be realized without increasing the number of storage nodes specified by the storage table fragments during multi-copy storage, so that the normal overhead of data storage in the distributed system is ensured; the problem of data storage failure caused by failure of each storage node specified by the original storage table fragment in the distributed system is avoided, and the success rate and reliability of data storage disaster tolerance in the distributed system are improved.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of data storage, and in particular, to a data storage method, device and storage node based on a distributed system. Background technique [0002] With the vigorous development of Internet technology, there will be a large number of user business requests between rapidly iterative applications and background services, which will also generate a large amount of business data. At this time, a corresponding distributed system will be built to store various business data. To use the stored business data for user behavior analysis and troubleshooting, etc. Therefore, in order to ensure the high-performance and successful reading and writing of data in the distributed system, it is necessary to provide a reliable disaster recovery for the data storage under the distributed system Function. [0003] At present, distributed systems usually use data sharding to store the primary copy of the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F11/14
CPCG06F3/0614G06F3/067G06F11/1464
Inventor 罗佳
Owner BIGO TECH PTE LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap