Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A task allocation method and system based on a resource management platform

A resource management platform and task allocation technology, applied in the field of task allocation method and system based on resource management platform, can solve problems such as time and resource waste, and achieve the effect of avoiding time and resource waste

Active Publication Date: 2022-07-15
GUANGDONG INSPUR BIG DATA RES CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Based on the above problems, this application provides a task allocation method and system based on a resource management platform, which can solve the problem in the prior art when processing artificial intelligence model training or testing tasks on a resource management platform due to a large number of nodes needing to download data sets time and resource waste

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A task allocation method and system based on a resource management platform
  • A task allocation method and system based on a resource management platform
  • A task allocation method and system based on a resource management platform

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0048] see figure 1 , which is a flowchart of a task allocation method based on a resource management platform provided by an embodiment of the present application.

[0049] It should be noted that the methods provided by the embodiments of the present application can be applied to a distributed resource management platform, such as a resource management platform such as kubernetes.

[0050] In this embodiment, the method can be implemented, for example, by the following steps S101-S104.

[0051] S101: Accept the AI ​​model training and / or testing tasks and the names of the datasets required to process the tasks.

[0052] It should be noted that the types of artificial intelligence models are not limited in the embodiments of the present application. The artificial intelligence model mentioned in the embodiments of the present application may be an algorithm model in deep learning or machine learning.

[0053] It should be noted that, the task in the embodiment of the prese...

no. 2 example

[0067] In the above-mentioned first embodiment, a node having a required data set whose size meets the preset requirements is selected for task allocation, and it is difficult to quantify the required data set size of a node. Moreover, in the case of actually assigning tasks, when there are a few nodes in the cluster that have the required data set size that meets the preset requirements, according to the above assignment method, it is easy to cause a large number of tasks to be assigned to a small number of nodes, resulting in The load on each node assigned to the task is too large.

[0068] In view of the above situation, the present application provides another task allocation method based on the resource management platform, figure 2 This is a flowchart of another method for task allocation based on a resource management platform provided by the embodiment of the present application.

[0069] This embodiment will be described in detail below.

[0070] S201: Receive arti...

no. 3 example

[0083] In the above two exemplary embodiments, the task assignment to the nodes takes into account the required data sets on the nodes. When assigning tasks, the nodes with the larger required data sets are selected for task assignment, as follows: It is guaranteed to reduce the download of datasets when processing tasks. However, in some cases, nodes with larger required data sets may not be able to meet the needs of processing tasks in terms of their own performance, etc. Only considering whether the nodes have data sets may not be able to select the most efficient node for processing tasks under conditions . In view of the above situation, the present application provides another embodiment of a task allocation method based on a resource management platform.

[0084] The main difference between this embodiment and the previous two embodiments is that, for selecting a node for assigning tasks, not only the required data set owned by the node, but also the computing capabili...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present application discloses a task allocation method and system based on a resource management platform. The method includes: receiving artificial intelligence model training and / or testing tasks and the names of data sets required for processing the tasks; acquiring data set distributions of multiple nodes information; according to the name of the data set in the node, determine whether the node has the required data set; if the node has the required data set, select the required data set according to the size of the required data set in the node The nodes whose size of the data set meets the preset requirements are assigned tasks. It can be seen that the present application takes into account the data sets possessed by nodes when assigning tasks, and selects nodes whose size of the required data sets satisfies the preset conditions for task allocation, so that nodes do not need to download the required data sets or reduce the amount of data downloaded by nodes. The frequency of sets increases the efficiency of processing tasks.

Description

technical field [0001] The present application relates to the field of artificial intelligence technology, and in particular, to a method and system for task allocation based on a resource management platform. Background technique [0002] With the rapid development of artificial intelligence technology and containerization technology, more and more artificial intelligence models are trained and tested on distributed resource management platforms. Existing artificial intelligence models are trained and tested using datasets, and different datasets are selected according to the application scenarios of different models. Correspondingly, the distributed resource management platform needs to download the corresponding data set according to the application scenario of the model to train and test the model. When the training task or testing task of the artificial intelligence model needs to be processed on the resource management platform, the resource management platform first ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4881G06F9/5066G06F9/5033G06F2209/509G06N20/00G06F9/5088G06F2209/5017G06F2209/5022
Inventor 王德奎
Owner GUANGDONG INSPUR BIG DATA RES CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com