A machine learning hyperparametric optimization system and method based on asynchronous Bayesian optimization

Abstract

The invention relates to a machine learning super-parameter optimization system and a machine learning super-parameter optimization method based on asynchronous Bayesian optimization, comprising a Bayesian optimization module, a model parameter pool model, a Kmeans clustering module, a task scheduling module and an adaptive determination model parallelism module. The invention automatically adjusts the parameters of machine learning under the big data environment efficiently, effectively utilizes the parallel computing ability of multiple computers, and efficiently adjusts the parameters of the big data machine learning automatically, so that people can better use the big data machine learning in the production practice.

Description

technical field [0001] The invention relates to a machine learning hyperparameter optimization system and method based on asynchronous Bayesian optimization. It belongs to the field of computer artificial intelligence. Background technique [0002] With the development of cloud computing and big data technology, machine learning technology has become a hot spot in academia and business circles. However, machine learning involves a lot of theoretical knowledge, and the machine learning model contains a large number of parameters. It requires rich experience to design an efficient model. In order to promote the wider application of machine learning technology and effectively reduce the threshold for machine learning applications, Automatic Machine Learning (AutoML for short) technology emerged as the times require, that is, by providing automation technology for each link of machine learning, beginners can also Machine learning model training and application can be carried o...

AI Technical Summary

Problems solved by technology

The problem with Random search is that if two parameter points are relatively close (for example, the Euclidean distance in space is small), then these two parameter points are mutually redundant, which will reduce the search efficiency. For high-dimensional features Space (when there are many parameters), it is easy to fall into a certain local area

[0008] The shortcomings of classical Bayesian optimization are: the optimization process is a serial process, which cannot effectively utilize the parallel computing power of multiple machines. In the big data environment, there is still the problem of low efficiency, which makes it difficult for big data machine learning to perform automatic tuning. reference, unable to effectively deal with the big data environment

[0010] The asynchronous Bayesian optimization proposed by Kandasamy K et al. is a parallel method of classical Bayesian optimization, but the method in the paper uses each computing node to be responsible for the evaluation of a model, which makes it impossible for a single model to be effectively used for large data. Training, but also unable to cope with the scene where multiple models converge at the same time

[0011] Bayesian optimization in the above-mentioned big data environment has problems such as low efficiency, which makes the availability of machine learning automatic parameter adjustment technology in the big data environment low

Images