Deep learning job operation method and device

Abstract

The invention provides an operation method and device for deep learning jobs. The method comprises the steps that deep learning job information to be processed is acquired; wherein the to-be-processeddeep learning job information comprises the type of the to-be-processed deep learning job; matching in a preset operation strategy according to the type of the deep learning job to be processed to obtain a scheduling strategy corresponding to the type of the deep learning job to be processed; wherein the scheduling strategy comprises operation priorities of the deep learning jobs; and finally, running the deep learning job to be processed according to the running priority of the deep learning job. Deep learning homework and an operation strategy are separated; the two parts are independent; only when the deep learning job needs to be operated, a scheduling strategy corresponding to the type of the deep learning job is obtained through matching in a preset operation strategy; wherein the scheduling strategy comprises operation priorities of the deep learning jobs, so that the purpose of unifying priority strategies of all operators to facilitate subsequent extension and maintenance isachieved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a method and device for running a deep learning job. Background technique [0002] With the rapid development of computer technology, more and more applications of Kubernetes, Kubernetes is an open source platform for automatic deployment, expansion and operation and maintenance of container clusters, which can be abbreviated as K8s; Kube-bath is a batch of K8s The processing scheduler provides a mechanism for applications that want to run K8s batch jobs; kubeflow is a glue project that combines many support for machine learning, such as model training, hyperparameter training, model deployment, etc., into containers Deploy in a standardized way to provide high availability and convenient expansion of each system in the entire process. Users who deploy kubeflow can use it to perform different machine learning and deep learning tasks. Different types of deep learning jo...

AI Technical Summary

Problems solved by technology

[0003] At present, there are priority options for deep learning jobs (jobs) when scheduling in Kube-bath, while Kubeflow contains a variety of operators, and each job will have a priority setting, but there has never been a unified The standard, and it has not been fully implemented, which leads to great inconvenience in the process of subsequent expansion and maintenance

Images