Neural network processing method and device, storage medium, and electronic apparatus

Abstract

The invention provides a neural network processing method and device, a storage medium, and an electronic apparatus, wherein the method comprises: determining an objective function in a convolution neural network, wherein each layer of the objective function corresponds to an initial network weight value, and the convolution neural network is applied to at least one of face recognition, vehicle detection, and object recognition; processing the initial network weights according to preset regular constraint coefficients to obtain target network weights; using the target network weights and input values to an output value in the objective function. The invention solves the technical problems of slow convergence and low precision of convolution neural network in the prior art.

Description

technical field [0001] The present invention relates to the communication field, in particular, to a neural network processing method and device, a storage medium, and an electronic device. Background technique [0002] In the convolutional neural network in the prior art, as the number of layers of the convolutional neural network deepens, the complexity of the network becomes larger and larger. For example, the number of convolutional layers of the popular resnet neural network can exceed 1000 In addition, the calculation of all convolutional layers accounts for almost 80% of the calculation of the entire network. As a result, similar convolutional neural networks cannot be run on embedded devices such as surveillance cameras. In order to reduce the computational complexity of the convolutional layer, existing techniques operate by directly binarizing the floating-point weights and floating-point activations of the neural network. [0003] figure 1 is a schematic diagra...

AI Technical Summary

Problems solved by technology

[0003] figure 1 is a schematic diagram of the weight Gaussian distribution of the convolutional neural network in the prior art of the present invention, such as figure 1 As shown, the weight w of each layer of the convolutional neural network l The distribution of is generally around 0, which is close to the Gaussian distribution, so if the binarization operation is directly performed on the weight, it will lead to the binarized weight and the original weight w l The difference is large, resulting in oscillation when using the stochastic gradient descent algorithm to optimize the convolutional neural network, resulting in slower convergence and lower accuracy

In the same way, the output activation value of each layer of the convolutional neural network is also similar to the Gaussian distribution. If binary quantization is forcibly performed, it will cause a large difference in the value before and after quantization.

[0004] Aiming at the above problems existing in the prior art, no effective solution has been found yet

Images