A Hybrid Scheduling System for Improving Online Service Quality and Cluster Resource Utilization

Abstract

The present invention proposes an online and offline hybrid scheduling system that improves online service quality and cluster resource utilization, including an information collection module, an online AM module, an offline AM module, a client module, and a scheduler module; The call relationship between online services is stored in Redis; when a user submits an online application, the online AM module analyzes the dependencies of each component, and deploys the components in the cluster in sequence according to the dependencies ; When a user submits an offline job, the offline AM module applies for resources from the RM, and schedules the offline job to run in the cluster; the client module parses and processes the job submitted by the user, and converts the job into a corresponding request to The RM applies for resources, and is responsible for pulling up the online AM module to show the application running status to the client; the scheduler module is located in the RM, and the scheduler module will regularly calculate the criticality of each container according to the criticality of each container reported by the online AM module The criticality of the server node, and schedule according to the calculated server criticality when scheduling offline jobs.

Description

technical field [0001] The invention relates to the field of cluster resource task scheduling, and mainly relates to an online and offline hybrid scheduling system for improving online service quality and cluster resource utilization. Background technique [0002] With the vigorous development of cloud computing technology, more and more calculations and applications are hosted on public clouds. Through the cloud platform, users can quickly and easily expand simple applications into large-scale complex applications, and pay on demand, at each time node, they only pay costs comparable to their scale. Suppliers can gain economies of scale by building large-scale data centers and sharing resources with multiple tenants. [0003] However, cluster resource utilization in most cloud facilities is very low, which greatly reduces cost efficiency. The researchers conducted a resource utilization analysis for a production cluster of thousands of machines on Twitter. This cluster was...

AI Technical Summary

Problems solved by technology

However, the existing technology only supports scheduling offline jobs, which cannot meet the needs of modern data centers to simultaneously schedule online services and offline jobs; for multiple components deployed in the system, some are very busy, while others are very idle, lacking an accurate The method of locating the key components that have the greatest impact on user experience in the system; in addition, the system does not consider giving preferential treatment to these key components, and lacks a method to ensure the quality of online services

Images