Progressive differentiable network architecture search method and system based on Bayesian optimization

Abstract

The invention discloses a progressive differentiable network architecture search method and system based on Bayesian optimization. The method is applied to the search of a neural network structure of automatic machine learning, and is characterized by comprising the following steps of selecting a part of operations by adopting Bayesian optimization when operation selection between nodes is carried out based on PDARTS; sampling channels connected between the nodes through Bayesian optimization, and performing operation search; and after the Bayesian optimization of the channel connected with each node is finished, introducing an attention mechanism to carry out weight superposition, and realizing the search of the network architecture. According to the present invention, the occupied storage resources and calculation consumption during network architecture search can be greatly reduced, and the search time is greatly shortened.

Description

technical field [0001] The invention relates to the technical field of automatic machine learning, in particular to a progressive differentiable network architecture search method and system based on Bayesian optimization. Background technique [0002] Neural Architecture Search (NAS) is one of the hot spots in the field of automatic machine learning (Auto-ML). By designing a cost-effective search method, it can automatically obtain a neural network with strong generalization ability and friendly hardware requirements, and a large number of liberated Researcher creativity. In the classic NAS method (that is, the network structure search method), it mainly includes the following three aspects: search space, search strategy, and evaluation evaluation. Differentiable Architecture Search (DARTS) and Progressively Differentiable Architecture Search (PDARTS) are generally used in the prior art for network structure search. [0003] However, there are two main disadvantages of di...

AI Technical Summary

Problems solved by technology

But this greatly affects the effect of

(2) When the method starts to search, the operation it takes is random, that is, affected by random initialization, the search results are not controllable and difficult to reproduce

The main disadvantages of progressively differentiable network architecture search (PDARTS) are: (1) Although the problem of resource occupation is solved to a certain extent, the memory and calculation loss required by it are still very large

Moreover, on a large data set, many non-parameter connections (skip-connect) will be caused. Although there are certain restrictions in the text, it is still not accurate enough, resulting in low accuracy and time-consuming of the searched network.

Images