Rapid deployment method of convolutional neural network on FPGA based on pytorch framework

Abstract

The invention discloses a method for quickly deploying a convolutional neural network on an FPGA based on a Pytorch framework, including establishing a model rapid mapping mechanism, reconfigurable computing unit construction, and an adaptive processing flow based on rule mapping. When defining a convolutional neural network under the Pytorch framework, a fast mapping mechanism for the model is established through the construction of naming rules. The optimization strategy calculation is performed under the constraints of hardware resources, and the template library based on the hardware optimization strategy is established, and the reconfigurable computing unit is constructed on the FPGA side. Finally, in the adaptive processing flow based on rule mapping, the complex network model file is decomposed on the FPGA side, the network is abstracted into a directed acyclic graph, and finally a neural network accelerator is generated to realize the integration from the model file of the Pytorch framework to FPGA deployment. process. This method can establish a directed acyclic graph of the network through the model rapid mapping mechanism. In the FPGA deployment process, only the hardware design variables need to be input to complete the FPGA deployment process. The method is simple and has strong versatility.

Description

technical field [0001] The invention belongs to the technical field of convolutional neural network hardware acceleration, and relates to a rapid deployment method of a convolutional neural network based on a Pytorch framework on an FPGA. Background technique [0002] In recent years, convolutional neural networks have been widely used in natural language processing, computer vision and other fields. A neural network generally includes two stages: training and testing. Training is the process of extracting model parameters from training data and neural network models (ResNet, RNN, etc.) using CPU or GPU. The test is to run the test data with the trained model (neural network model + model parameters) and check the results. And caffe, Pytorch, and tensorflow are to unify and abstract the link data involved in the training process to form a usable framework. [0003] At present, the industry deploys deep learning models on FPGAs mostly by analyzing the caffe prototxt to obt...

AI Technical Summary

Problems solved by technology

Since the Pytorch model does not contain network topology information, the model trained in Pytorch needs to be converted into a caffe model through tools such as onnx before it can be deployed. However, the onnx tool only supports the regular caffe layer. If it is a custom layer in Pytorch, then unable to convert

Images