Deep learning model tuning method and computing device

Abstract

A computing device includes a memory, a scheduling unit, and an acceleration unit, wherein the acceleration unit is used for executing each quantitative model; the memory stores instructions; the scheduling unit reads the instruction to perform the following steps: creating a plurality of configuration combinations for the deep learning model, each configuration combination specifying a value combination of a plurality of quantitative configuration parameters; based on each configuration combination, performing quantization operation on the deep learning model to obtain a plurality of models after quantization operation; sequentially deploying the plurality of quantized models to an acceleration unit, and receiving precision data corresponding to the plurality of quantized models from the acceleration unit; and on the basis of the respective precision data of the plurality of models after the quantification operation, obtaining an optimal model of which the precision loss meets a set condition. According to the embodiment of the invention, the neural network acceleration unit and the scheduling unit are matched with each other, so that the optimal model with small precision loss can be quickly obtained.

Description

technical field [0001] The present disclosure relates to the field of neural networks, and in particular, relates to a deep learning model tuning method and computing device. Background technique [0002] In the field of neural networks, inference refers to pushing a pre-trained deep learning model to be used in actual business scenarios. Because inference is directed at the user, inference performance is critical, especially for enterprise-grade products. [0003] Regarding inference performance, in addition to optimization at the hardware level, at the algorithm level, model quantization (Quantized Model) is one of the important means to improve inference performance. There are many methods for model quantization at present, and converting the model from a 32-bit single-precision floating-point number (FP32) model to an 8-bit integer data (INT8) model is one of the methods. Usually we use 32-bit single-precision floating-point numbers when building deep learning models. ...

AI Technical Summary

Problems solved by technology

[0004] However, converting the model from the FP32 model to the INT8 model will result in a loss of data accuracy, so how to minimize the loss of accuracy during quantization is a key issue

Images