Task scheduling method based on distributed scheduling system, server and memory medium

Abstract

The invention provides a task scheduling method based on a distributed scheduling system, a server and a memory medium. The method comprises the steps of receiving mass task scheduling requests, establishing timed tasks in batches according to the requests, and storing related information of each task to a message queue; receiving the tasks from the message queue, issuing each task to a corresponding host for performance according to a preset mode, and monitoring a performance state of each task in real time; automatically triggering the host which performs the task to reperform the task according to a preset task restart rule if certain task is performed unsuccessfully, writing performance state identification information to the task by a load balancing device if the task is still reperformed unsuccessful, forwarding the task to the other host for performance, until the task is performed successfully, and storing a task performance result to a database in a log form. Through application of the method, the server and the memory medium, the scheduling of mass distributed scheduling tasks can be realized, and the throughput of the distributed system is improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a task scheduling method based on a distributed scheduling system, a server and a computer-readable storage medium. Background technique [0002] In the Internet, cloud computing, and big data era, task processing has the characteristics of big data and high concurrency. The main purpose of the distributed task scheduling system is to avoid repeated processing of data when coordinating multiple nodes to process the same task. The nodes of the cluster share the processing of large batches of tasks to improve the processing efficiency of batch tasks. [0003] The existing distributed scheduling system implements task scheduling based on the database. The nodes executing the tasks in the system write the relevant information of the tasks into the database one by one. The scheduler of each node abides by an operation rule based on database locks to ensure the uniqueness of the ope...

AI Technical Summary

Problems solved by technology

[0003] The existing distributed scheduling system implements task scheduling based on the database. The nodes executing the tasks in the system write the relevant information of the tasks into the database one by one. The scheduler of each node abides by an operation rule based on database locks to ensure the uniqueness of the operation. , but this scheduling strategy has certain limitations: the cluster feature works well for tasks with high CPU usage, but for a large number of short tasks, each node will seize the database lock, so there will be a large number of threads waiting for resources

And this situation will become more and more serious with the increase of nodes

Images