Message-based distributed task distribution scheduling method and device

Abstract

The invention provides a message-based distributed task distribution scheduling method and device. The method comprises the steps: extracting the attributes of a task, and segmenting the task into a plurality of atomic tasks; distributing the atomic tasks to corresponding working nodes for execution; and according to the execution condition of the atomic tasks, performing limited times of distribution retry on the atomic tasks which fail to be executed. By the adoption of the technical scheme, the problems that in the prior art, a task distribution scheduling technology lacks a task segmentation function and task distribution scheduling control is not flexible are effectively solved.

Description

technical field [0001] The invention belongs to the field of the Internet and distributed systems, and relates to a message-based distributed task dispatching method. Background technique [0002] At present, distributed detection technology is generally adopted in the field of Internet information detection. Through distributed deployment of working nodes, the distribution scheduling server comprehensively considers the number of tasks, computing resources of working nodes and network resources, and distributes scheduling of working nodes to perform Internet information detection tasks. Task distribution scheduling includes system load balancing, task priority and distribution rate control, failed task retry mechanism and other task distribution scheduling strategies. System load balancing refers to adjusting the task distribution and scheduling strategy according to the system load such as the resource usage of the working nodes, the number of system sessions, and the occu...

AI Technical Summary

Problems solved by technology

[0003] However, the existing task distribution and scheduling methods generally have the following problems: First, the lack of accurate task segmentation function

Due to the various input parameters of Internet information detection plug-ins, the existing task distribution and scheduling methods do not divide tasks, resulting in complex detection plug-in function logic and duplication of different detection plug-in functions, which reduces the overall efficiency of the system; flexible

Although the existing task distribution and scheduling methods take into account the adjustment of the distribution and scheduling strategy according to the resource usage of the working nodes, there is no task distribution and scheduling strategy that distinguishes between different priorities. At the same time, it also lacks a management and control mechanism for failed tasks, resulting in high priority tasks. Or emergency tasks cannot be executed first, the results of failed tasks are missing or become zombie tasks and cannot be ended normally, which reduces the reliability of task scheduling and also affects the user experience

Images