Deep neural network multi-path reasoning acceleration method for edge intelligent application

Abstract

The invention discloses a deep neural network multi-path reasoning acceleration method for edge intelligent application, comprising the following steps of: firstly analyzing the classification capability and calculation cost of quitting a branch outlet in advance of each layer of a deep neural network, and selecting an outlet combination with the maximum reasoning benefit to be added to an original model; setting a threshold unit between the exit and a trunk layer, training, and judging whether the task can exit from the current exit or not; compressing intermediate feature data for a task which cannot exit in advance at a terminal layer but must be transmitted to an edge layer; and finally, monitoring and analyzing the network load and the computing capabilities of the terminal and the edge equipment on line in an edge computing environment, cutting a multi-path model by taking the minimized reasoning time delay as a target, and respectively deploying model blocks in a terminal layerand an edge layer to finally form a multi-path reasoning acceleration framework. According to the method, the reasoning flexibility can be improved, the accuracy is ensured, the total reasoning time delay is reduced, and the real-time and high-precision requirements of edge intelligent application are met.

Description

technical field [0001] The present invention belongs to the fields of edge intelligence and deep learning, and specifically relates to a method for implementing inference optimization and acceleration of deep learning models on which applications depend under the scenario of deploying intelligent applications in an edge computing environment. Background technique [0002] During the rapid development of artificial intelligence, Deep Neural Network (DNN), with its powerful learning ability, has achieved excellent results in classic task scenarios such as computer vision or natural language processing. At the same time, with the development of the Internet of Things era, the rapid popularization of smart terminals such as smart cameras, smart sensors, and various Internet of Things devices has made deep learning algorithms successfully applied to some practical deployments, such as face recognition, smart security and other scenarios. Terminal intelligence has become an inevit...

AI Technical Summary

Problems solved by technology

However, such methods directly ignore the selection of the location and number of outlets. The simple preset outlets do not consider the complex and variable computing resource loads of the terminal layer and the edge layer, and the thresholds set manually do not have a good effect on the newly collected data. Decision-making ability leads to a decrease in the accuracy of model reasoning, which cannot meet the high-precision requirements of applications such as autonomous driving

[0007] Therefore, the existing deep learning model reasoning acceleration method still has great limitations when applied to the scenario of edge computing and artificial intelligence applications, and cannot meet the low-latency and high-precision operation requirements of edge intelligence applications.

Images