Heterogeneous cluster-orientated task processing method and device

Abstract

The invention discloses a heterogeneous cluster-orientated task processing method and device. The method comprises the following steps that: receiving a task request and a basic execution environment;according to the task request, dispatching heterogeneous equipment; compiling the basis execution environment into an execution environment corresponding to the dispatched heterogeneous equipment, and deploying to the dispatched heterogeneous equipment; and enabling the dispatched heterogeneous equipment to execute the task request. By use of the method, a user only needs to provide the basic execution environment by aiming at a task, the user does not need to independently compile one version of execution environment by aiming at each type of hardware platform, the quick development of the code of the heterogeneous equipment is realized, and development and maintenance cost is lowered.

Description

【Technical field】 [0001] The invention relates to computer application technology, in particular to a heterogeneous cluster-oriented task processing method and device. 【Background technique】 [0002] With the development of big data and deep learning technology, the method of training and learning massive data through deep learning methods, and finally learning a set of accurate deep models has been paid more and more attention by various technology companies. A more complex and powerful in-depth model can deeply reveal the complex and rich information carried in massive data, and make more accurate predictions for future or unknown events. Deep learning applications include speech recognition, image recognition, natural language processing, search advertising CTR estimation, etc. Since deep learning applications usually rely on huge computing and communication, using a heterogeneous cluster of CPU+ heterogeneous accelerators (GPU, FPGA, etc.) for training and online has be...

AI Technical Summary

Problems solved by technology

[0005] 1. The development cost is huge: In order to write hardware device programs, project developers need to learn certain hardware knowledge, which is not convenient for rapid iteration of software

[0006] 2. Huge maintenance cost: Since there is a version of the code for each hardware platform, it is necessary to maintain multiple codes with unified functions

Images