Service scheduling method, device and system

Abstract

The invention discloses a service scheduling method, device and system. A specific embodiment of the method comprises the steps of: monitoring whether a waiting queue has service requests, wherein the waiting queue is used for storing service requests to be executed; in response to the determined service requests in the waiting queue, exchanging a running queue with the waiting queue, wherein the running queue is used for storing the current executed service request; combining the service requests in the exchanged running queue to generate a to-be-processed request set and sending the to-be-processed request set to a service processing interface, so that the service processing interface processes the to-be-processed request set in batch; and after the service requests in the to-be-processed request set are processed, emptying the exchanged running queue. The embodiment can adaptively adjust the batch processing quantity according to the service demand and the hardware processing capability, thereby improving the service processing efficiency.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to the field of computer resource allocation and scheduling technology, especially to a business scheduling method, device and system. Background technique [0002] With the development of computer network technology, the data scale of various network services continues to grow, and the service scheduling mechanism affects the processing efficiency of services to a large extent. The DNN (Deep Neural Network, deep neural network) model can be used for large-scale data prediction business. This type of business requests has high concurrency, high real-time requirements, and a large amount of calculation. Heterogeneous hardware is usually used to batch process business requests. In this scheduling architecture, business requests are combined according to the pre-configured number of batches and processed by heterogeneous hardware. The pre-configured number of batches is re...

AI Technical Summary

Problems solved by technology

[0003] However, due to the hardware architecture limitations of FPGA (Field-Programmable Gate Array) / GPU (Graphics Processing Unit, Graphics Processing Unit), current heterogeneous hardware cannot perform multiple tasks at the same time, that is, multi-threading of business Does not improve hardware efficiency

However, the method of processing forecasted service requests based on the above-mentioned pre-configured batch quantity has the following disadvantages: each type of service needs to add a scheduling layer, which is used to combine service requests and initiate requests to heterogeneous hardware uniformly, which increases the number of service requests. The difficulty of use; when the QPS (Query PerSecond, query rate per second) is small, the number of batches is fixed so that the number of requests must reach a certain number or a certain delay can be processed, and the hardware is idle during the process of waiting for the request. It leads to waste of resources; when the QPS is large, the maximum number of business requests that can be processed at the same time is the number of batches, and the efficiency of heterogeneous hardware to merge large-scale requests is much higher than the efficiency of sequentially processing each request, so pre-configure the batch The way of processing quantity reduces the calculation efficiency to a certain extent, which is not conducive to maximizing the use of hardware computing power and adapting to the real-time requirements of the business

Images