Distributed decision making supporting massive high-concurrency access I/O (Input/output) server load balancing system

Abstract

The invention discloses a distributed decision making supporting massive high-concurrency access I / O server load balancing system, and is used for solving the problems of high performance computers, for example, the high performance computer is lack of file strips with high data concurrency access file level, an allocation method which is used for the strip-oriented files and can fully consider dynamic file access features, and a load balancing which supports the distributed decision making. The system disclosed by the invention provides a novel dynamic load balancing support which has high expandability and fully considers network delay and migration cost. Modularly, the system mainly comprises file level strips oriented to high-concurrency access request, file allocation capable of minimizing IO access conflict and the load balancing based on distributed decision making. The system can meet the application needs of the load balancing system of the high performance computers with massive high concurrency and distributed decision making. Thus, the system has the characteristics of being wide in application foreground and capable of generating remarkable economic benefit, and the like.

Description

technical field [0001] The invention discloses a large-scale IO server load balancing system, in particular to a load balancing system among large-scale high-concurrency access I / O servers in a high-performance computer file system supporting distributed decision-making. I / O refers to input / output output. It belongs to the field of computer technology. Background technique [0002] Parallel input / output (Input / Output) system has always been a hot research direction in the field of computer system architecture, which aims to create multiple data paths between memory and disk to alleviate the I / O performance bottleneck of computer systems. As one of the core software of the parallel input / output system, the parallel file system not only provides the semantics and interfaces required for parallel access to file data, but also enables file segmentation, file allocation, and dynamic load balancing between data servers. means to ensure the aggregated access speed of file data. ...

AI Technical Summary

Problems solved by technology

Load mass data hierarchical storage method (application number: 200710118116.5), metadata service system, metadata synchronization method and write server update method (200810224708.X), a dynamic data distribution method for cluster file system (application number: 201210184965.1), a A method and device for extracting file layout from a parallel file system (application number: 201110003511.5), a dynamic load balancing method for cluster servers (03118565.7), a server load balancing method for weighted minimum connection allocation (03147308.3), etc., However, the existing mature parallel file system server balancing technology and patent research at home and abroad pay more attention to the load balancing of computing nodes, and lack of load balancing among large-scale I / O servers in parallel file systems, especially "large-scale I / O" / O server" and "complex I / O access mode" (such as small file I / O access in PB-level environment, etc.) Examination of high-performance computers, the research on load balancing among large-scale I / O servers in parallel file systems in higher-performance computers is relatively few or even lacking; finally, traditional load balancing methods are not enough to solve the problems in higher-performance computers. Load imbalance problems due to significant increase in the number of servers and complex I / O access patterns, etc.

There is no server load balancing system among large-scale high-concurrency access I / O servers in a high-performance computer file system that supports distributed decision-making based on the basic flow of load balancing processing, especially one that can simultaneously support high-concurrency file access requests. File-level striping, file striping that minimizes I / O access conflicts, and dynamic adaptive load balancing based on distributed architecture Three functions of load balancing methods and systems related research and patents

Images