Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

DAG task scheduling method and device based on offline computing platform

An offline computing and task scheduling technology, applied in computing, multi-programming devices, program control design, etc., can solve problems such as inability to provide, clearly display upstream and downstream dependencies, and inability to find the source of tasks, and achieve task scheduling. Simple and effective to improve the efficiency of task scheduling

Inactive Publication Date: 2018-12-11
杭州比智科技有限公司
View PDF7 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, it is not possible to provide information such as which task changes caused the status change of a task (backtracking upstream), and which tasks will change due to a task status change (backtracking downstream), that is, it is impossible to find the source of the task that caused the task change.
Moreover, neither of the above two processing methods can clearly display the upstream and downstream dependencies between different task nodes, providing users with a visual display of task processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • DAG task scheduling method and device based on offline computing platform
  • DAG task scheduling method and device based on offline computing platform
  • DAG task scheduling method and device based on offline computing platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0031] figure 1 A flow chart of a DAG task scheduling method based on an offline computing platform according to an embodiment of the present invention is shown. like figure 1 As shown, the DAG task scheduling method based on the offline computing platform specifically includes the following steps:

[0032] Step S101, constructing a plurality of task nodes according to the pre-stored metadata information, and constructing a depen...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a DAG task scheduling method and device based on an offline computing platform. The method comprises the following steps: a plurality of task nodes are constructed according tothe metadata information stored in advance, and constructing the dependencies among the task nodes according to the dependencies among the metadata; topologically sorting the task nodes to generate the DAG task relationship diagram; according to the triggered scheduling request to the task node, carrying out the task of the corresponding task node, and modifying the display style of the corresponding task node in the DAG task relation diagram in real time. The invention utilizes DAG task relation diagram to visually display the process of task scheduling to users, realizes real-time monitoring of task scheduling, facilitates tracing the task abnormality by using DAG task relation diagram, and searches the source of task abnormality. Furthermore, DAG task diagram makes task scheduling easier and improves task scheduling efficiency.

Description

technical field [0001] The invention relates to the field of software, in particular to a DAG task scheduling method and device based on an offline computing platform. Background technique [0002] Offline calculation (Offline calculation) is a data calculation technology oriented to the field of data processing. It is a solution for batch acquisition of data, batch transmission of data, periodic batch calculation of data, and data display. Offline computing is the ability to perform complex batch calculations on a large amount of data, save for a long time, the data is completely in place before calculation and will not change, and the results of batch calculations can be queried quickly. At present, most offline calculations use Sqoop to acquire data in batches, HDFS to store data in batches, MapReduce to calculate data in batches, Hive to calculate data in batches, and Hadoop’s own task scheduling scheme. With the rapid development and application of information technolo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48
CPCG06F9/4806
Inventor 徐慧慧王洁
Owner 杭州比智科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com