Structure learning in convolutional neural networks

Abstract

The present disclosure provides an improved approach to implement structure learning of neural networks by exploiting correlations in the data / problem the networks aim to solve. A greedy approach is described that finds bottlenecks of information gain from the bottom convolutional layers all the way to the fully connected layers. Rather than simply making the architecture deeper, additional computation and capacitance is only added where it is required.

Description

technical field [0001] The present disclosure relates to computing networks, and more particularly, to neural networks configured to learn hierarchical representations from data. Background technique [0002] Neural networks involve computational methods loosely modeled after the neural structures of biological brain processing that can be used to solve complex computational problems. Neural networks are often organized as a set of layers, where each layer includes groups of interconnected nodes that contain various functions. Weighted connections enable processing within the network to perform various analytical operations. Learning methods can be employed to construct and modify the network and the associated weights of the connectors within the network. By modifying connector weights, this allows the network to learn from past analyses, over time, to improve future analyses. [0003] Neural networks can be employed to perform any suitable type of data analysis, but are...

AI Technical Summary

Problems solved by technology

However, a disadvantage of this approach is that the number of layers remains static, where only known individual layers within the static number of layers are augmented by structural learning methods to be more or less complex

Therefore, the method fails to identify any new layers that may be required to optimize the structure

Images