Distributed task scheduling method and device, electronic equipment and readable storage medium

Abstract

Embodiments of the invention provide a distributed task scheduling method and device, electronic equipment and a readable storage medium. The method comprises the following steps of: receiving task pulling requests sent by a plurality of task execution nodes at a preset frequency by a main node; distributing a task to a target task execution node by the main node according to the received task pulling request, wherein the target task execution node is a task execution node in the plurality of task execution nodes; receiving a task execution result sent by the target task execution node after executing the distributed task by the main node; and when the fact that the target task execution node has a fault is perceived, transferring an in-execution task queue of the target task execution node to other survived task execution nodes. According to the method and device, the electronic equipment and the readable storage medium, single-point faults can be avoided and the stability and availability of systems can be improved.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of distributed scheduling, and in particular, to a distributed task scheduling method and device, electronic equipment, and a readable storage medium. Background technique [0002] In business systems, background tasks are generally used to periodically process some batch jobs (tasks) to improve user experience and increase system scalability, so distributed task scheduling is widely used. [0003] In the prior art, Quartz is commonly used as a distributed task scheduling framework. Quartz has two ways to use: one is to integrate tasks into the business system, and execute a pre-configured task periodically or regularly, task details, trigger time Such information is stored in the local memory, and only one node is executing tasks at the same time, thereby realizing distributed task scheduling. This method has disadvantages such as single point of failure and does not support failov...

AI Technical Summary

Problems solved by technology

[0003] Quartz is commonly used as a distributed task scheduling framework in the prior art. There are two ways to use Quartz: one is to integrate the task into the business system, periodically or regularly execute a pre-configured task, task details, trigger time Such information is stored in the local memory, and only one node is executing tasks at the same time, thereby realizing distributed task scheduling. This method has disadvantages such as single point of failure and does not support failover.

The other is the clustered deployment of Quartz. The details of multiple tasks are stored in a database or other distributed storage system and shared by all Quartz task processes (also called nodes). In this way, the system availability is not high. All Quartz nodes Each Quartz node communicates through a shared database. When a network failure occurs, each Quartz node thinks that other Quartz nodes are offline, and thus becomes the master node to take over the entire system. Therefore, a "split brain" phenomenon occurs

Images