Method and system for providing gpu service

Abstract

The disclosure relates to the technical field of deep learning services, and discloses a method and system for providing GPU services. The method includes the following steps: start a container in a container cluster management system; read configuration information and load the container in the container according to the configuration information Inference server; receiving request information from the client, and sending a calculation instruction to the inference server according to the request information, the calculation instruction is used to instruct the inference server to invoke a model deployed based on the Caffe framework to perform inference on the GPU; receiving the calculation result returned by the reasoning server; and sending the calculation result to the client. Some technical effects of the present disclosure are: by loading the inference server in the container, and invoking the model deployed by the Caffe framework to perform inference on the GPU, the service request of the client can be satisfied with high efficiency.

Description

technical field [0001] The present disclosure relates to the technical field of deep learning services, in particular to a method and system for providing GPU services. Background technique [0002] Deep learning (DL, Deep Learning) has developed extremely rapidly in recent years. Various researches based on convolutional neural networks (CNNs, Convolutional Neural Networks) are in full swing, and academic literature emerges in endlessly. The emergence of these research results is inseparable from the support of hardware. It is the super computing power of the image processing unit (GPU, Graphics Processing Unit) that makes it possible for shallow neural networks to rapidly develop into deep neural networks. Therefore, GPU is an indispensable computing resource for deep learning research and application. [0003] The application fields of deep learning are extremely wide, such as image processing, speech recognition, target detection and other fields are involved. The proc...

AI Technical Summary

Problems solved by technology

But TensorFlow Serving does not provide support for deep learning framework Caffe

[0013] The essence of the deployment of the Caffe model can be understood as the deployment of GPU services. In order to improve the utilization of GPU resources in TensorFlow Serving, Batcher is used to combine multiple reasoning requests into one batch processing request. This mode is not applicable to Caffe. Caffe Although the model under the framework can also load multiple pieces of data at one time for processing, its internal code logic is still executed sequentially rather than in parallel. If batch mode is used, it will not only occupy more video memory resources, but also slow down the processing speed. Improvement, so it is necessary to additionally consider the deployment method of the Caffe model

Images