Job scheduling method and apparatus

Abstract

The invention provides a job scheduling method and apparatus. The job scheduling method comprises the steps of obtaining priority information of job entities carried by the job entities and the minimum resource quantity required by the job entities; dividing the multiple job entities with the same priority into the same job queue according to the priority information of the job entities to obtain at least a job queue; and if the resource quantity allocated to the job entities in each job queue is less than the minimum resource quantity required by the job entities, obtaining available resource from a resource queue, and allocating the available resource to the job entities. By adoption of the scheme of the job scheduling method and apparatus, service level agreement guarantee for different job entities when multiple job entities are under concurrent execution can be realized, including resource allocations for job entities of different priorities or the job entities of the same priority; the balance of resource sharing and well-aligned guarantee is realized; and therefore, the service level agreement for each job entity is realized, and the utilization ratio of the overall cluster resource is improved as well.

Description

technical field [0001] The invention relates to the technical field of computer networks, in particular to a job scheduling method and device. Background technique [0002] Existing big data or distributed computing systems use a simple queue mechanism for job scheduling, which lacks guarantees for different service level agreements (SLAs) for different jobs. [0003] The current job scheduling of big data or distributed computing generally adopts the following queue mechanisms: [0004] 1. Single queue mode [0005] (1) First-in-first-out queue: That is, various jobs are queued indiscriminately according to the time sequence of applying for resources, and the job scheduler allocates resources according to the queuing sequence of jobs. [0006] (2) The queuing strategy of the earliest completion time: According to the estimated execution time of each job, compare the situation of idle resources, and select the job with the shortest execution time to be executed first. [...

AI Technical Summary

Problems solved by technology

[0009] However, the existing job scheduling methods for big data or distributed computing systems cannot meet the scheduling requirements in the case of concurrent multiple jobs and multiple SLAs, such as:

[0010] 1. When a low-priority job is running, a new high-priority job preempts the resources of the existing job to run;

[0011] 2. When resource contention occurs, the minimum resource quota for the job cannot be guaranteed;

[0012] 3. Resource waste when utilization is unbalanced among multiple resource queues

Images