Low-latency collaborative adaptive CNN inference system and low-latency collaborative adaptive CNN inference method

Abstract

The invention discloses a low-latency collaborative adaptive CNN inference system and method, which are applied to an edge computing network. The system comprises a master node and a plurality of slave nodes, the master node is used for monitoring the running state of each slave node in real time and periodically updating inference tasks of the slave nodes, and the slave nodes are used for inferring specific execution of the tasks. According to the invention, a CNN cross-layer data range deduction mechanism is realized in the system, so that the master node and the slave node can deduce the dependency relationship between any two pieces of data in multiple layers of the CNN, thereby realizing efficient user-defined dynamic scheduling; meanwhile, a balanced synchronous scheduler is designedas a built-in scheduler of the system, the load of the slave nodes is adjusted in a dynamic sensing mode, and efficient recovery of a memory is achieved.

Description

technical field [0001] The invention relates to the fields of edge computing and deep learning, in particular to a low-latency cooperative adaptive CNN inference system and method. Background technique [0002] The past decade has witnessed the rise of deep learning. As its typical representative, Convolutional Neural Network (CNN) is widely used in computer vision and video analysis. Using a dedicated CNN, applications can detect and classify objects from images / videos more precisely than before. Despite these advantages, it is worth noting that CNN inference is computationally resource intensive. Taking VGG-16 as an example, classifying an image with a size of 224*224 requires 15.5G multiplication and addition operations. Therefore, traditional solutions tend to transfer images or videos to the cloud with powerful computing power. However, user data is generated at the edge of the network. Long-distance transmission over a WAN introduces latency and jitter that can ad...

AI Technical Summary

Problems solved by technology

[0006] Purpose of the invention: Aiming at the deficiencies of the prior art, the present invention proposes a distributed collaborative adaptive CNN inference system for CNN tasks under the edge network, which at least solves the problems of fixed scheduling strategy and high synchronization overhead, so as to speed up the deployment of CNN at the edge of the network. inference speed

Images