Brain-like binary neural network automatic structure learning method

Abstract

The invention relates to a brain-like binary neural network automatic structure learning method, and belongs to the field of neural networks, and the method comprises the following steps: S1, carrying out the binaryzation of a reference model, including the binaryzation of weight and activation; s2, establishing a plurality of parallel feed-forward subnets based on the binarization reference model, and connecting different subnets through a channel-level weighted transverse path to obtain an initial CBiNet model structure; s3, according to the loss functions of different application tasks, a total loss function is established in combination with grouping sparse regularization terms based on transverse path connection; s4, after the learning rate is set, training the CBiNet on the training set, and when the loss function shows convergence, ending the training to obtain the CBiNet with sparse transverse path connection; and S5, testing the CBiNet on the test set. According to the CBiNet constructed by the method, the feature expression capability of the binary neural network can be improved, and the image processing task can be higher in precision and higher in accuracy.

Description

technical field [0001] The invention belongs to the field of deep neural networks, and relates to an automatic structure learning method of a brain-like binary neural network. Background technique [0002] As the ultimate version of the compression model, the binary neural network (Binary Neural Network, BNN) constrains the full-precision weight and activation of the deep neural network (Deep Neural Network, DNN) to two discrete values, namely {-1,+1 }. Therefore, theoretically, the memory capacity required for DNN inference can be reduced by up to 32 times. By utilizing BNNs, energy-intensive operations such as floating-point multiplication can be replaced by efficient bit operations, which reduce computational latency and power consumption by orders of magnitude compared to high-precision comparable models. However, BNNs usually cause severe performance degradation. [0003] Recent research suggests using multiple binarized sub-networks in BNN to better approximate the o...

AI Technical Summary

Problems solved by technology

[0005] In view of this, the object of the present invention is to provide a brain-like binary neural network automatic structure learning method, through the cellular binary neural network (Cellular Binary Neural Network, CBiNet), to solve the problem of low image processing precision and low accuracy

Images