Recurrent neural network architecture search method and system based on an improved evolutionary algorithm, and medium

Abstract

The invention discloses a recurrent neural network architecture search method and system based on an improved evolutionary algorithm, and a medium. The method comprises the following steps: training aplurality of recurrent neural network sub-models to update a shared weight; initializing a generation population and a historical record table for recording the performance of all the recurrent neural network models; randomly sampling from the population to generate a sample, selecting a sample optimal model to perform mutation operation, removing the oldest or worst model in the population witha specified probability, and adding mutated child nodes into the population and the historical record table; and judging whether a preset ending condition is met or not, if not, continuing to carry out sample variation, and otherwise, outputting an optimal model in the historical record table. According to the method, the search process of a recurrent neural network architecture can be accelerated, the performance and the search time are considered simultaneously when the population is updated in each step, and the search efficiency of the recurrent neural network architecture can be greatly improved.

Description

technical field [0001] The present invention relates to architecture design and optimization technology of deep neural network, in particular to a search method, system and medium of cyclic neural network architecture based on improved evolutionary algorithm. Background technique [0002] Over the past few years, deep neural networks have achieved great success in many challenging applications, such as speech recognition, image recognition, and machine translation. Yet until recently, most of these advanced neural network architectures were manually designed by human experts. To speed up the process, the researchers looked at automated methods, now commonly referred to as neural network architecture search (NAS), which traditionally used evolutionary algorithms. [0003] The basic steps of the evolutionary algorithm include: firstly initialize the population, then select the optimal model from the population as the parent model to perform a mutation operation to generate a ...

AI Technical Summary

Problems solved by technology

However, ENAS needs to train for 150 cycles to continuously update the shared weights in the sub-models, and needs to train LSTM as a controller to search for the best architecture, which will undoubtedly increase the time and complexity of the search

Images