Task scheduling method and device

Abstract

The embodiment of the present application provides a task scheduling method and device, wherein the method includes: obtaining a task scheduling request; the task scheduling request carries the task path corresponding to the task to be executed; according to the task path corresponding to the task to be executed and the task configuration information, determine the Execute the result output path and execution method corresponding to the task respectively; send the task path, result output path, and execution method corresponding to the task to be executed to the task execution device to execute the task; after determining that the task to be executed is successfully executed, the The result output path corresponding to the execution task is re-used as the task path corresponding to the task to be executed; the above process is repeated until the cyclic scheduling and execution of the task ends. Through the embodiment of the present application, the maintenance cost of task scheduling can be reduced, and the efficiency of task execution can be improved.

Description

technical field [0001] The present application relates to the field of computer equipment, in particular to a task scheduling method and device. Background technique [0002] In big data processing, the efficiency of task scheduling directly affects the efficiency of big data processing, and also affects the ability of the data platform to execute tasks submitted by users. [0003] At present, there are two main ways of task scheduling, one is based on DAG graph (Directed Acyclic Graph, non-loop directed graph) for task scheduling, for example, divide a job into multiple tasks, and create a DAG graph according to the dependencies between tasks , and perform task scheduling according to the DAG graph. The other is a scheduled task scheduling mechanism, specifically, configure a time interval, schedule tasks according to the configured time interval and trigger task execution. [0004] In the above-mentioned first method, a complex DAG graph needs to be created in advance be...

AI Technical Summary

Problems solved by technology

[0004] In the above first method, it is necessary to pre-create a complex DAG graph before task scheduling, and if the scheduling order changes during the task scheduling process, the DAG graph needs to be modified, so this method has the problem of high maintenance cost of task scheduling

In the above-mentioned second method, when scheduling tasks according to the preset time interval and triggering task execution, if there is data delay and the data has not been transmitted to the task execution module, it will lead to waste of task scheduling and execution, so this method Has the problem of low task execution efficiency

Images