Bayesian structure learning method and device of deep neural network

Abstract

The embodiment of the invention provides a Bayesian structure learning method and device of a deep neural network. The method comprises the steps that a deep neural network comprising a plurality of learning units with the same internal structure is constructed, each learning unit comprises a plurality of hidden layers, a plurality of calculation units are included between the hidden layers, the network structure is the relative weight of each calculation unit, and parameterized variational distribution is adopted to model the network structure; a training subset is extracted, and a network structure is sampled by adopting a re-parameterization process; an evidence lower bound is calculated; and if the change of the evidence lower bound exceeds the loss threshold, the network structure andthe network weight are optimized, and new training is started. According to the embodiment of the invention, a deep neural network comprising a plurality of learning units with the same internal structure is constructed; and the relative weight of each calculation unit between each hidden layer in the learning unit is trained through the training set to obtain an optimized network structure, thereby bringing comprehensive improvement to the prediction performance and prediction uncertainty of the deep neural network.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a Bayesian structure learning method and device for a deep neural network. Background technique [0002] Bayesian deep learning aims to provide accurate and reliable uncertainty assessment for flexible and efficient deep neural networks. Traditionally, the Bayesian network introduces uncertainty into the network weights, which can often prevent the model from being easily over-fitted, and also brings efficient prediction uncertainty to the model. However, there is a problem with introducing uncertainty on the network weights. First of all, the prior distribution of artificially set weights is often unreliable, which can easily lead to problems such as over-pruning, which greatly limits the model fitting ability; secondly, introducing flexible variational distributions on weights can easily lead to difficulties in reasoning. This is because of complex dependencies in the...

AI Technical Summary

Problems solved by technology

However, there is a problem with introducing uncertainty on the network weights

First of all, the prior distribution of artificially set weights is often unreliable, which can easily lead to problems such as over-pruning, which greatly limits the model fitting ability; secondly, introducing flexible variational distributions on weights can easily lead to difficulties in reasoning. Th

Images