Method and device for managing container resources in cluster

Abstract

The invention relates to a method for managing container resources in a cluster. The method comprises the following steps: (1) presetting a seat for each node in the cluster; (2) after each round of scheduling, if the task running on the node needs to be interrupted and suspended, executing forced deletion on the task; (3) performing forced deletion on the task execution of the scheduling program, and enabling all nodes which terminate task operation to serve as idle available nodes to enter a new round of scheduling; (4) acquiring the existence state of a container in a single node every other seat monitoring trigger time, and updating the seat state in real time; (5) applying for resources of a seat when a container is started for a node needing to execute a new training task in a new round of scheduling, and waiting for the seat resources if no seat resources exist; and (6) creating a new container, starting a new training task, and changing the seat state. The efficiency of the scheduling system can be improved, risks caused by coexistence of training containers are eliminated, and meanwhile the independence of the scheduling module is improved.

Description

technical field [0001] The invention relates to the field of computing clusters, in particular to a method and device for managing container resources in a cluster. Background technique [0002] With the development of cloud computing technology, complex models need to be trained using large-scale computing clusters, and Kubernetes container technology is usually used for large-scale deployment of training environments. The process of model training needs to consider the problem of computing power allocation and task scheduling. The invention application (patent application number: 2021114594024) discloses a method for cluster management task scheduling. Allocate available nodes for tasks with running permission. For tasks that have not obtained running permission: if they are in a non-running state, suspend them and continue to wait; if they are already running, they need to be suspended and the node resources occupied by them need to be released. . However, in the prior ...

AI Technical Summary

Problems solved by technology

However, in the existing technology, for nodes that use forced deletion to release resources, there is a certain lag time from when the forced deletion occurs to when the real container is deleted and the resources are released.

During system resource scheduling, this lag time will lead to container coexistence problems. Newly started containers need to compete for resources with containers that have not been actually deleted, resulting in low operating efficiency and may even cause serious problems such as insufficient resources.

If the container is actually deleted before scheduling, it will also lead to low efficiency of the scheduling system and lower resource utilization.

Images