Fine-granularity high-performance cloud resource management and scheduling method

Abstract

The invention discloses a fine-granularity high-performance cloud resource management and scheduling method. By means of user login computational node limiting, core-memory same-proportion resource scheduling, limiting of CPU nuclear number used by each GPU, releasing of occupied resources after operation is finished, SSH login using resource management and login node resource limiting, managementof memory and GPU resources when tasks are distributed according to a CPU core in a SLURM operation scheduling system can be improved, Cgroup limiting on resources of user SSH login is realized, a SLURM-based resource scheduling system is improved, and more efficient and reasonable high-performance could platform resource scheduling is realized.

Description

technical field [0001] The invention relates to high-performance computing technology, in particular to a fine-grained high-performance cloud resource management and scheduling method. Background technique [0002] High-performance computing usually uses parallelization technology to run applications efficiently, quickly and reliably, usually using multiple processors (as part of a single machine) or multiple computers organized in a cluster to execute the same job in parallel to achieve Performance that cannot be achieved by a single machine. High-performance computing is mainly used to deal with complex computing problems and can be applied in large-scale scientific computing environments, mainly involving weather forecasting, nuclear weapon research, nuclear material storage simulation, oil exploration and oil storage simulation, bioinformatics and gene sequencing, etc. . [0003] SLURM (Simple Linux Utility for Resource Management, a simple and practical resource manag...

AI Technical Summary

Problems solved by technology

[0008] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a fine-grained high-performance cloud resource management and scheduling method. By logging in to the node resource limit, the node resources are allocated according to the core allocation and memory in the same proportion, and the GPU nodes are allocated according to the number of GPUs requested. Limit the number of CPU applications, limit users to log in to computing nodes, monitor and limit the resources used by users for SSH login, etc., to solve the uneven allocation of user resource applications and resource limitation loopholes caused by the use of SLURM for resource management and scheduling in the existing technology question

Images