Implementation method of gradient descent width learning system

Abstract

The invention provides an implementation method of a gradient descent width learning system. The gradient descent width learning system is composed of three dense layers, namely a feature layer, an enhancement layer and an output layer, and a merging layer. The feature layer maps the input data into mapping features by using random mapping to form feature nodes; the enhancement layer enhances themapping features output by the feature layer to form enhancement nodes; the merging layer merges the outputs of the feature nodes and the enhancement nodes and inputs the merged outputs into the output layer as a whole; and the output layer maps the output of the merging layer into the final output of the network, a small batch of training samples are continuously input into the gradient descent width learning system during training, and the gradient descent width learning system adopts a gradient descent method to update the weight of the network, so that the loss of a mean square error MSE loss function is gradually reduced. When the gradient descent width learning system implemented by the method continuously performs small-batch training sample training, the regression performance during batch training can be obviously improved, and the method can be applied to regression tasks.

Description

technical field [0001] The invention relates to the technical field of width learning, in particular to a method for realizing a gradient descent width learning system. Background technique [0002] The Broad Learning System (BLS) is a model that can replace the deep structure neural network (deep structure neural network). It was proposed by Chen et al. and can be effectively used in classification and regression tasks. Different from the currently popular Deep Neural Network (DNN), BLS is proposed on the basis of Random Vector Functional Link Neural Network (RVFLNN) and has a flat network architecture. . Its basic idea is: First, generate mapped features from input data to form feature nodes, and one mapped feature can form multiple feature nodes. Second, the mapped features are enhanced into enhancement nodes (enhancementnodes) with randomly generated weights. Finally, all mapped features and enhancement nodes are directly connected to the output, and the required conn...

AI Technical Summary

Problems solved by technology

[0004] For the existing BLS processing regression tasks, for the case of small batch input data increments, that is, small batch training samples continue to enter the network, the loss value of the loss function may increase continuously, which may lead to network regression performance degradation technical problem, the present invention provides a gradient descent width learning system implementation method

Images