Tensor calculation operation method, tensor calculation operation device and tensor calculation operation system

Abstract

The embodiment of the invention provides a tensor calculation operation method, a tensor calculation operation device and a tensor calculation operation system. The method comprises the steps of forming a plurality of calculation instructions, wherein the calculation instructions are used for representing tensor calculation modes; aiming at each calculation instruction, determining one of the plurality of equipment ends as the equipment end corresponding to the calculation instruction; for each calculation instruction, packaging the calculation instruction in a packaging mode which can be analyzed by a preset operation environment of the equipment end corresponding to the calculation instruction to obtain task information corresponding to the equipment end; and for each piece of task information, sending the task information to the device end corresponding to the task information, so that the plurality of device ends carry out tensor calculation in parallel according to the tensor calculation mode represented by the plurality of calculation instructions. The advantages of large-scale operation can be fully utilized in a parallel calculation mode, tensor calculation is accelerated, and the tensor calculation efficiency can be effectively improved.

Description

technical field [0001] The invention relates to the technical field of machine learning, in particular to a tensor calculation operation method, device and operation system. Background technique [0002] In some application scenarios, due to actual needs, electronic devices with computing capabilities can perform calculations between one or more tensors (hereinafter referred to as tensor calculations), such as the process of using a trained convolutional neural network for image recognition In , the convolutional layer, pooling layer, activation layer, etc. in the neural network can be realized by tensor calculation. [0003] The complexity of tensor calculation is high, and general-purpose host-side devices (hereinafter referred to as the host-side) often do not have the hardware conditions to perform high-complexity calculations. For example, the host-side often does not have a high-performance GPU, such as P4 ( A GPU model), T4 (a GPU model), TX1 (a GPU model) type GPU. ...

AI Technical Summary

Problems solved by technology

[0005] However, there is often a lack of unified standards on the device side. For example, some devices can use P4-type GPUs, other parts can use T4-type GPUs, and the rest can use TX1-type GPUs.

The same computing instruction is difficult to be recognized by all devices, so it is difficult for developers to use multiple devices with different types of GPUs to perform tensor calculations together, resulting in low efficiency of tensor calculations

Images