Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Dynamic load balancing method and system for distributed file system

A distributed file and dynamic load technology, applied in the field of cloud data, can solve the problems of unbalanced load and slow writing speed of distributed file system, and achieve the effect of ensuring load balance and writing speed

Pending Publication Date: 2021-02-26
SHANGHAI BILIBILI TECH CO LTD
View PDF11 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the embodiment of the present application is to provide a dynamic load balancing method, system, computer equipment and computer-readable storage medium for a distributed file system, so as to solve the problems of slow write speed and unbalanced load of the distributed file system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dynamic load balancing method and system for distributed file system
  • Dynamic load balancing method and system for distributed file system
  • Dynamic load balancing method and system for distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0038] figure 2 A flow chart of a dynamic load balancing method for a distributed file system according to Embodiment 1 of the present application is schematically shown. It should be noted that, the following is an exemplary description with the management node 200 as the execution subject.

[0039] Such as figure 2 As shown, the dynamic load balancing method for a distributed file system may include steps S200-S206, wherein:

[0040] Step S200, acquiring multiple heartbeat packets (Heartbeat) provided by multiple data nodes.

[0041] The multiple data nodes are nodes used for data storage in the distributed file system, such as DataNodes 300-308.

[0042] Compared with only being used to report survival information, each heartbeat packet in this embodiment is also used to report load-related information of a corresponding data node. The load-related information includes IOPS, IOUtil, Load, BandWidth, and the like. Among them: (1) IPOS (Input / Output Operations Per Seco...

Embodiment 2

[0094] Figure 8 Schematically shows a block diagram of a dynamic load balancing system for a distributed file system according to Embodiment 2 of the present application, the dynamic load balancing system for a distributed file system can be divided into one or more program modules, one Or multiple program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program modules referred to in the embodiments of the present application refer to a series of computer program instruction segments capable of accomplishing specific functions. The following description will specifically introduce the functions of each program module in this embodiment.

[0095] Such as Figure 8 As shown, the dynamic load balancing system 800 for a distributed file system may include, wherein:

[0096] The first acquiring module 810 is configured to acquire multiple heartbeat packets provided by multiple data nodes, w...

Embodiment 3

[0107] Figure 9 A schematic diagram of a hardware architecture of a computer device 9 suitable for implementing a dynamic load balancing method for a distributed file system according to Embodiment 3 of the present application is schematically shown. The computer device 9 may serve as the management node 200 or a component of the management node 200 . In this embodiment, the computer device 9 is a device capable of automatically performing numerical calculation and / or information processing according to preset or stored instructions. For example, it may be a workstation, a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers) and the like. Such as Figure 9 As shown, the computer device 9 at least includes but is not limited to: a memory 910 , a processor 920 , and a network interface 930 that can communicate with each other through a system bus. in:

[0108] The memory 910 incl...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a dynamic load balancing method for a distributed file system. The method comprises the steps: obtaining a plurality of heartbeat packets provided by a plurality of data nodes,and enabling the plurality of data nodes to be nodes for data storage in the distributed file system; collecting node load information of each data node at least based on the plurality of heartbeat packets; receiving a writing request of a client; and responding to the write-in request: selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node so as to connect the write-in task of the client through the plurality of target data nodes. In the embodiment, the load related information is provided through the heartbeat packet so as to select the appropriate data node for writing, and the writing speed of the client is guaranteed.

Description

technical field [0001] The present application relates to the field of cloud data, and in particular to a dynamic load balancing method, system, computer equipment and computer-readable storage medium for a distributed file system. Background technique [0002] With the development and popularization of computer and information technology, data is growing explosively. At present, the accumulated data is written and stored in the distributed file system. While effectively utilizing and processing these data, ensuring the load balance of the distributed file system has become one of the research hotspots of the distributed file system. [0003] A distributed file system includes multiple data nodes and management nodes. These data nodes are responsible for the storage, reading and writing of real files, and the management nodes are responsible for assigning tasks to these data nodes for execution. Task allocation is a fundamental and challenging problem for distributed file ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/182G06F9/50
CPCG06F16/182G06F9/5083G06F9/505
Inventor 陈世云岑玉海毛剑
Owner SHANGHAI BILIBILI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com