A design method of a full-fixed-point neural network

Abstract

The invention discloses a design method of a full-fixed-point neural network. The method comprises the following steps of designing a neural network framework, and selecting a saturation activation function as a neural network activation function; selecting an initial overall fixed point bit width of the data according to an application scenario of the neural network; determining an initial decimal part bit width and an initial integer part bit width according to the precision requirement and the data characteristics of the neural network; carrying out binary conversion on the decimal part andthe integer part; taking the converted fixed point format data as input, carrying out neural network training, and recording a training result; Recording a training test result; and repeating the step 2 to the step 6 until the overall fixed-point bit width, decimal bit width and integer bit width meeting the requirements are found to serve as the final fixed-point architecture of the neural network. According to the present invention, the design of the neural network is calculated by adopting the fixed point number, the occupied resources are few, and the power consumption and the cost are low.

Description

technical field [0001] The invention relates to the technical field of artificial intelligence neural networks, in particular to a design method of a fully fixed-point neural network. Background technique [0002] An artificial neural network is a computational model designed by humans to mimic the way biological neural networks work. Neuron (Neuron) is the basic unit of neural network, also known as node (Node), it receives input (Input) from the outside or other nodes, and calculates output (Output) through an activation function (Activation Function); each The input corresponds to Weight, which is the relative importance of each input received by this node; Bias can be understood as a special input. [0003] Deep learning is a field of machine learning that studies the algorithms, theory, and applications of complex artificial neural networks. Since it was proposed by Hinton in 2006, deep learning has been greatly developed and has been successfully applied to many fiel...

AI Technical Summary

Problems solved by technology

However, compared with fixed-point arithmetic units, floating-point arithmetic has problems such as occupying more resources, large area, high power consumption, and high cost.

In particular, for FPGA and application-specific integrated circuit (ASIC) chips, there is also the problem of poor timing convergence when it comes to hardened neural networks

Images