Method for self-adaptively adjusting learning rate by tracking and controlling neural network

Abstract

The invention provides a method for self-adaptively adjusting the learning rate by tracking and controlling a neural network. The method comprises the following steps of building a control system; unitizing all weights of the neural network according to layers; introducing a training sample set to obtain an error signal e(n) and a training cost function epsilon(n); obtaining a linearized activation function s(x); determining an induction local region and nerve cell output of each nerve cell; solving each local gradient function delta j(n) and each linearization expression delta jL(n); adjusting the learning rate selectively and self-adaptively; training the weight of a synapse of each nerve cell; adding one to the number of cycles until stopping criterions are met, and outputting tracking and controlling signals. According to the method for self-adaptively adjusting the learning rate on the basis of the activation function and the multiple error signals, a step length iteration mean value is not affected by noise, a smooth iteration curve is obtained, and the error signals are fully used; thus, the learning rate can be updated in real time, and computation complexity is reduced.

Description

technical field [0001] The invention relates to the technical field of neural network optimization, in particular to an adaptive learning rate adjustment method for neural network tracking control. Background technique [0002] The artificial neural network is a network system composed of interconnected artificial neurons. It abstracts and simplifies the human brain from the perspective of microstructure and function. It can be regarded as a large-scale highly parallel processor composed of simple processing units. Nature has the property of storing experiential knowledge and making it available. The similarity between the neural network and the human brain is that the knowledge acquired by the neural network is learned from the external environment, and the connection weights between the interconnected neurons are used to store the acquired knowledge. In terms of processing and computing, although the function of each processing unit seems simple, the parallel activities o...

AI Technical Summary

Problems solved by technology

Scholars at home and abroad have also achieved certain results in the research of LMS variable step size. Aboulnasr algorithm proposed by Aboulnasr et al. is a representative type of LMS method that uses error signal to iterate. The cross-correlation function control of the error at the next adjacent time, so that the mean value of the step size iteration will not be affected by the noise, and the iterative curve will be smoother, but the iterative process of this kind of method has a certain complexity, and the use of the error signal is also difficult. Restricted to the instantaneous error and its previous error

Images