Deep learning hyper-parameter tuning improvement method based on Bayesian optimization

Abstract

The invention discloses a deep learning hyper-parameter tuning improvement method based on Bayesian optimization. The method comprises the steps that a fast optimal Latin square experimental design algorithm is used to generate an initial point set, and the generation rate of effective evaluation points is improved; parallel computing is used to acquire the response of the initial point set in a computer, and the response of multiple points on an objective function is calculated at the same time to construct a data set; and a Bayesian optimization iterative process is initiated, and a parallelcomputing method is used in the computing process to accelerate the whole optimization process. According to the invention, the defects of long time consumption and large performance fluctuation of atraditional Bayesian optimization algorithm can be effectively solved; the optimization speed is accelerated through parallel computing; and the optimization efficiency is significantly improved.

Description

technical field [0001] The present invention relates to an optimization and improvement method, in particular to an optimization and improvement method for deep learning hyperparameters based on Bayesian optimization. Background technique [0002] Deep learning is a method based on data representation learning in machine learning, which is mainly manifested in building models and using data for learning. Hyperparameters are model parameters whose values ​​are set before starting the learning process, rather than the parameter data obtained through training. The selection of hyperparameters in deep learning is difficult and irregular, and there are unpredictable effects between different hyperparameters. Its debugging is very time-consuming, and the evaluation of each hyperparameter combination requires a large number of iterative calculations. For such problems, some classic optimization algorithms such as particle swarm optimization algorithm, simulated annealing algorithm...

AI Technical Summary

Problems solved by technology

However, the random experiment design method or the artificially set initial point cannot guarantee the rationality of the initial point distribution, which has a great impact on the simulation model of the objective function; the orthogonal sampling method can ensure the rationality of the sampling distribution, but increases the Calculation cost; compared with the random sampling method, the traditional Latin hypercube experimental design method has a more reasonable data distribution, but the distribution of sampling points still has randomness; the initial point sampling methods of the above methods are as follows: Figure 1-3 shown

Therefore, applying the traditional Bayesian optimization algorithm to solve the hyperparameter optimization problem of the deep learning model has high computational costs, and due to the flaws in the initial point generation method, the optimization results fluctuate greatly, which is not conducive to engineering applications

Images