Memory management method based on deep learning network

Abstract

The invention relates to a memory management method based on a deep learning network. The method comprises the following steps: loading network model parameters; loading a network model; receiving network operation data input by a user; comparing the network input size with a default network input size or a last network input size; if the network input size is large, recalculating the temporary memory of each layer and the inter-layer transfer memory, and redistributing the temporary memory and the inter-layer transfer memory; and enabling the network to operate according to the operation equipment and parameters set by the user.

Description

technical field [0001] The invention belongs to the field of deep learning, and in particular relates to memory management based on a deep learning network. Background technique [0002] With the rapid development of deep learning in recent years, products in computer vision, data processing, etc. are increasing day by day, and chips that can deploy deep learning are also diversified. The existing deep learning deployment framework has the following problems: [0003] 1. The network architecture cannot use different processing units (such as DSP and GPU) to run different layers of the same network at the same time. [0004] 2. On embedded devices, the use of memory is very limited. In the past network (such as NCNN, MACE) operation, the memory allocation is invisible to the user, the user can only view and monitor the memory usage of the network through the system's memory detection system, and there is an unpredictable memory allocation Possibility, causing network insta...

AI Technical Summary

Problems solved by technology

[0003] 1. The network architecture cannot use different processing units (such as DSP and GPU) to run different layers of the same network at the same time

[0004] 2. On embedded devices, the use of memory is very limited

In the past network (such as NCNN, MACE) operation, the memory allocation is invisible to the user, the user can only view and monitor the memory usage of the network through the system's memory detection system, and there is an unpredictable memory allocation possibility, causing network instability on embedded devices

[0005] 3. In the previous network framework, the utilization rate of GPU is uncontrollable, and all network architectures will maximize the use of GPU

When multiple networks call the GPU at the same time, there will be competition for resources, and the running time of each network will cause uncertainty

Images