Dense connection-based deep neural network structure improvement method and system

Abstract

The invention discloses a deep neural network structure improvement method based on dense connection. The method comprises the steps of establishing a full-connection deep neural network, and determining the number of layers of the full-connection deep neural network, the number of neurons of each layer and parameters, dividing the full-connection deep neural network into a plurality of dense connection blocks with moderate sizes according to a sequence, realizing dense connection in each dense connection block, and combining the dense connection blocks into the deep dense neural network according to a full connection mode. According to the inventionmethod, the dense connection idea is introduced, the full-connection deep neural network structure is improved, the gradient problem can be effectively reduced, the feature utilization rate is improved, the training time is shortened, and the training accuracy is improved.

Description

technical field [0001] The invention relates to the technical field of deep learning algorithms, in particular to a densely connected deep neural network structure improvement method. Background technique [0002] With the substantial improvement of computer processing capabilities, the research and development of deep learning algorithm technology has developed rapidly. The layers of deep neural networks are getting deeper and deeper, and with the increasing amount of data, deep neural networks have shown great advantages compared with traditional machine learning algorithms. [0003] However, with the increase in the number of deep neural network layers, some problems have also appeared. For example, the transmission of training signals is becoming more and more difficult, the gradient disappears and the gradient explodes, and the learning and training process becomes slow and difficult to converge. At the same time, the traditional fully connected neural network structur...

AI Technical Summary

Problems solved by technology

[0003] However, with the increase in the number of deep neural network layers, some problems have also appeared. For example, the transmission of training signals is becoming more and more difficult, the gradient disappears and the gradient explodes, and the learning and training process becomes slow and difficult to converge.

At the same time, the traditional fully connected neural network structure cannot add richer feature combinations, and the feature utilization rate is low.

Images