Task allocation method and device

Abstract

The invention provides a task allocation method and device. The method includes the steps that a current computational node is determined as a computation priority type computational node or a storage priority type computational node according to resource information corresponding to each computational node in a distributed computational model; a target computational task is acquired and analyzed into at least two computational processes with the time sequence; the computational processes are analyzed, and the current computational process is analyzed into at least two parallel tasks; each parallel task is analyzed in order to determine attribute information corresponding to the parallel task; the current parallel task is determined as a data-intensive parallel task or a compute-intensive parallel task according to the attribute information corresponding to each parallel task; the compute-intensive parallel task is allocated to the computation priority type computational node, and the data-intensive parallel task is allocated to the storage priority type computational node. By means of the technical scheme, the computational duration of the target computational task can be shortened.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a task allocation method and device. Background technique [0002] With the advent of the big data era, pure high-performance single-node computing has been difficult to meet the needs of big data computing, and distributed computing models such as MapReduce, Spark, and Storm have emerged as the times require. [0003] At present, in the distributed computing model, multiple computing nodes are used to build a distributed computing model, and a target computing task with a large amount of computing is split into a serial computing process in chronological order. When the serial computing process can When further splitting into multiple parallel computing processes, when further splitting the current serial computing process into multiple parallel tasks, each parallel task under the current serial computing process is randomly assigned to the distributed computing model ...

AI Technical Summary

Problems solved by technology

[0004] Since the hardware performance of each computing node in the distributed computing model is not the same, when the computing node with lower hardware performance cannot meet the computing requirements or storage requirements of the parallel task assigned to the current computing node, it needs to be killed on the current computing node Drop the parallel task and reassign the parallel task to other computing nodes. Frequent killing and restarting the parallel task will consume a lot of time and increase the calculation time of the target computing task.

Images