A Workflow Task Scheduling Method in Distributed Environment

Abstract

The invention discloses a workflow task scheduling method in a distributed computing environment, which uses a directed acyclic graph to describe workflow tasks and resources, and calculates the average task execution time and average task according to node weight information and edge weight information Transfer time, so as to judge the workflow task type and perform classification model processing, and divide the tasks with data dependencies into several independent task sets, each task set contains one or several tasks, and each task set Multiple tasks of have data dependencies, thus turning workflow tasks into "independent tasks". The task set gathers tasks with a relatively large amount of computing data or data transmission together, and at the same time reduces the number of tasks that need to be allocated independently, and improves the performance of later task scheduling as a whole. In addition, when the number of workflow tasks increases, it is only necessary to strengthen the gathering operation of the tasks, and there is no need to process all the tasks, which has better scalability.

Description

technical field [0001] The invention belongs to workflow task scheduling technology and workflow task modeling technology under distributed computing, in particular to a workflow task scheduling method under distributed computing environment. Background technique [0002] Task scheduling is the most important part in a distributed computing environment, and it also plays an irreplaceable role in the entire distributed infrastructure. Task scheduling in a distributed computing environment requires finding the best solution for task resource allocation in consideration of time, cost, reliability, availability, throughput, and resource utilization. [0003] Task scheduling in a distributed computing environment is mainly divided into independent task scheduling and workflow task scheduling. A workflow task contains several tasks, and these several tasks have data dependencies. Existing methods aim to reduce the cost of task execution when scheduling workflow tasks. For exampl...

AI Technical Summary

Problems solved by technology

However, none of the above algorithms considers the impact of the data dependencies between workflow tasks on the task scheduling effect, resulting in high communication costs for workflow tasks, long execution completion time, and low efficiency of workflow task scheduling.

Images