Resource scheduling method and device in hyper-parameter optimization process, equipment and medium

Abstract

The invention provides a resource scheduling method and device in a hyper-parameter optimization process, equipment and a medium. A plurality of hyper-parameter tuning strategies are fused, each hyper-parameter tuning strategy is used for selecting a hyper-parameter combination for a machine learning model based on a hyper-parameter selection strategy corresponding to the hyper-parameter tuning strategy, and each round of iteration process comprises the step that currently available resources are determined; allocating currently available resources to the plurality of hyper-parameter tuning strategies; and obtaining one or more hyper-parameter combinations generated by the hyper-parameter tuning strategies allocated to the resources based on the allocated resources. Therefore, according tothe resource scheduling scheme fusing the plurality of hyper-parameter tuning strategies, the risk of poor effect or convergence to local optimum caused by using a single strategy can be eliminated,and the hyper-parameter optimization effect can be further improved by allocating resources to the plurality of hyper-parameter tuning strategies.

Description

technical field [0001] The present invention generally relates to the field of machine learning, and more specifically, relates to a resource scheduling method and a resource scheduling device in the hyperparameter optimization process of a machine learning model, as well as a computing device and a non-transitory machine-readable storage medium. Background technique [0002] Before training a machine learning model, it is necessary to determine the hyperparameters of the machine learning model. Hyperparameters can be regarded as the framework parameters of the machine learning model, which are parameters that describe the machine learning model from a higher level. For example, hyperparameters can describe various parameters such as the learning rate of the machine learning model, the dropout rate of nodes, and the batch size. [0003] At present, most hyperparameter optimization schemes of machine learning models are implemented based on a single strategy. These strateg...

AI Technical Summary

Problems solved by technology

[0004] The advantage of the non-model-oriented search strategy is that when there are enough hyperparameters sampled, it can find better hyperparameters. The disadvantage is that this unguided search strategy can find better hyperparameters when the dimension of the hyperparameter space is very high. very inefficient

Model-oriented strategies usually search for high-quality hyperparameters more efficiently than non-model-oriented strategies when the fitting effect is good. However, for any hyperparameter tuning problem, there is no guarantee that this fitting is effective

Images