Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Resource scheduling method and device, equipment and storage medium

A technology of resource scheduling and connection mode, which is applied in the field of deep learning and computer, and can solve the problems of slowing down the speed of distributed training and the inability to ensure reasonable scheduling of GPU resources, etc.

Active Publication Date: 2021-09-10
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF12 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Although the current resource scheduling algorithm considers different scheduling strategies when selecting the target computing node, it cannot guarantee that the selected target computing node is the computing node with the best performance. In turn, the reasonable scheduling of GPU resources cannot be guaranteed, resulting in slower distributed training

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Resource scheduling method and device, equipment and storage medium
  • Resource scheduling method and device, equipment and storage medium
  • Resource scheduling method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0024] The current resource adjustment methods on the public cloud usually include the following two implementation solutions.

[0025] Solution 1. The scheduler perceives the GPU resource requirements of the training task, selects the target computing node through multiple scheduling strategies, such as fragmentation (binpack) strategy, priority strategy, etc., and use...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a resource scheduling method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of deep learning. According to the specific implementation scheme, the method comprises the steps of obtaining a GPU topological relation of a cluster according to GPU connection information of each computing node in the cluster; and under the condition that a task request which aims at a target task and is used for applying for GPU resources is received, determining a target computing node of the target task and a target GPU in the target computing node according to the task request and the GPU topological relation so as to complete GPU resource scheduling of the target task. According to the invention, resource scheduling can be optimized.

Description

technical field [0001] The present disclosure relates to the field of computer technology, in particular to the field of deep learning technology, and further relates to a resource scheduling method, device, device, and storage medium. Background technique [0002] With the development of artificial intelligence technology, it is necessary to build deep learning applications in various scenarios. At the same time, as the deep learning model becomes more complex, with more and more parameters, and the amount of user data increases, distributed training has become the primary choice for deep learning training. In order to use massive data to train a deep learning model with better performance, it is necessary to efficiently call and manage a large-scale graphics processing unit (Graphics Processing Unit, GPU). From a platform perspective, how to use fewer GPU resources to complete more user training tasks has become a challenge and a development direction. [0003] Although ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/48G06F9/50G06T1/20
CPCG06F9/4881G06F9/5027G06T1/20G06F9/5044G06F9/5066G06F2209/509G06N3/098Y02D10/00G06F9/5033
Inventor 徐彬彬唐亮赵颖李曙鹏施恩钱正宇谢永康
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com