Tensorflow-based multi-task elastic scheduling method and tensorflow-based multi-task elastic scheduling system

Abstract

The invention discloses a tensorflow-based multi-task elastic scheduling method and system, and the method comprises the following steps: carrying out the preprocessing of tasks entering a cluster through a task management system, and allowing a plurality of tasks in the cluster to be carried out at the same time; obtaining the number of all GPU resources in the cluster from a configuration file of the task management system, dividing a data set for the training task by utilizing the number of available GPUs, and dividing each part of data set to a specified GPU; elastically allocating video memory resources to the tasks on each GPU, and elastically expanding idle GPUs to improve the training speed; completing gradient descent on each part of data set to obtain the current gradient of eachpart; accumulating the gradients by using the communication among the clusters to obtain a current total gradient; broadcasting the total gradient to each GPU, and then performing next gradient descent. According to the method, multiple tasks can be elastically scheduled to enter the cluster, and distributed computing is efficiently completed by utilizing existing resources of the cluster.

Description

technical field [0001] The invention belongs to the technical field of computers, and in particular relates to a multi-task flexible scheduling method and system based on tensorflow. Background technique [0002] As an important branch of the field of machine learning, deep learning has received high attention from industry and academia in recent years, and has achieved remarkable development, and has been widely used in machine vision, speech recognition and other fields. However, massive training data and ultra-large-scale models have brought increasingly severe challenges to deep learning. Distributed deep learning has emerged as the times require and has gradually become an effective means to deal with this challenge. [0003] Google has developed the MapReduce system, which has achieved certain results in the distributed training of processing large-scale data. This framework realizes the ability to efficiently process large-scale data, but its ability in deep learning ...

AI Technical Summary

Problems solved by technology

It introduces the idea of ​​multi-tenancy and flexible scheduling, but the way it expands from the inside of the cluster will change the composition of the cluster, so that it is difficult for users to make in-depth adjustments, and at the same time, some programmatic errors may occur, adding unnecessary trouble , less available

Images