Heterogeneous GPU distribution system and method for multiple deep learning tasks in distributed environment

Abstract

The invention belongs to the field of deep learning under artificial intelligence, and discloses a heterogeneous GPU distribution system and method for multiple deep learning tasks in a distributed environment, and the system comprises a GPU Profile module, a task information collection module, a GPU selection module and a deep learning training module. According to the heterogeneous GPU allocation method for the multiple deep learning tasks in the distributed environment, the GPUs with different computing capabilities can be allocated to the tasks with corresponding requirements, and the tasks with complex model levels and large batch data volume are adapted to the GPU with the best performance and the nodes with video memories sufficient for storage to run; tasks needing longer time to do deep learning training are accelerated, so that the multi-task execution efficiency in a heterogeneous environment is obviously improved; and when multiple deep learning tasks are executed concurrently, multiple deep learning can be integrally completed more quickly, so that the time for a programmer or a user to wait for a result can be saved.

Description

technical field [0001] The invention belongs to the field of deep learning under artificial intelligence, and in particular relates to a heterogeneous GPU allocation system and method for multiple deep learning tasks in a distributed environment. Background technique [0002] Today, deep neural networks are trained with large-scale data to obtain very accurate models, which promotes the continuous application of deep neural networks in the fields of image classification, speech recognition, and unmanned driving. These trends have led to more and more complex deep neural network models, and also prompted the emergence of devices to accelerate deep neural network training, such as GPUs, FPGAs, and TPUs. Therefore, how to use heterogeneous acceleration devices in a distributed environment more efficiently has gradually become an important hot issue. [0003] It has gradually become a common phenomenon for multiple tasks to perform deep learning training concurrently in a distr...

AI Technical Summary

Problems solved by technology

[0004] At present, the traditional GPU allocation method of the deep learning training framework is generally to statically specify the GPU parameters when starting multi-tasks in the distributed environment, and the GPU selection parameters provided by the deep learning training framework are used to schedule tasks with different requirements to the corresponding GPU for in-depth processing. Learning and training; the deep learning training framework also provides the use of all available GPU allocation methods, which allows the batch data of each task to be allocated to all GPUs for deep learning training. Due to the strong computing power of the GPU, the amortized small batches are quickly trained Data, this GPU allocation scheme causes the GPU with strong computing power to be idle for a long time, and the impact is that the utilization rate of the GPU with strong computing power is not high

[0005] Because the traditional GPU allocation scheme for multiple deep learning tasks in a distributed environment does not consider the characteristics and requirements of the tasks, and does not make full use of heterogeneous GPU performance to meet the concurrent operation of different deep learning training tasks, making multiple deep learning tasks The overall execution efficiency is not high

Images