Distributed file system based IO (input output) request dispatching method and system

Abstract

The invention discloses a distributed file system based IO (input output) request dispatching method. The distributed file system based IO request dispatching method includes receiving an IO request from a client; updating an overall situation process table according to the IO request; adding the IO request to the end of a request line of an overall situation object which the IO request belongs to; judging whether length of the continuous IO request of the overall situation object is smaller than REQ_MAX (L) or notand the L is smaller than maximum series of the multi-grade request line , if yes, deleting the overall situation object from the Lth line, and inserting the same to the end of the Lth+1 line; continuously monitoring the IO request from the client, repeating the above steps to generate a dynamic multi-grade progress request line; extracting all the IO requests of a first progress object in the multi-grade progress request line which is not null currently, and executing and updating status messages and parameters of the progress object. By the method, overall response time of a client request can be greatly reduced when system load is bigger, and performance of the system can be improved.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more particularly relates to a distributed file system-based IO request scheduling method and system. Background technique [0002] With the development of distributed computer clusters, large-scale data-intensive high-performance computing relies on distributed parallel file systems, such as Lustre, GPFS, PVFS, pNFS, etc. However, with the development of hardware, the mismatch between CPU speed and IO device speed has become more and more prominent. IO bottlenecks have become an important factor limiting the high performance of computer clusters. For applications with intensive data I / O requirements This bottleneck is more prominent. [0003] Parallel I / O refers to the parallel I / O access of multiple computer nodes to shared files or files on different disks at the same time. It mainly uses a set of storage devices to store massive data and provides corresponding parallel I / O inter...

AI Technical Summary

Problems solved by technology

[0007] Generally speaking, the request scheduling layer in the traditional distributed file system adopts a simple strategy, such as FCFS. Existing research has proposed some new scheduling methods, such as SJF, WSJF, MLF, etc., but these The scheduling method is based on IO request as a unit, without considering the client process to which the request belongs. For a client process with multiple synchronous IO requests, it is difficult to reduce its average waiting time and improve system performance

Images