Regression model hyperparameter optimization method

Abstract

The invention discloses a regression model hyperparameter optimization method comprising the first step of sequentially training regression models with a hyperparameter being p1 on an entire data set,and obtaining L well-trained candidate hyperparameter models; the second step of obtaining errors of each candidate hyperparameter model on each sample (xi, yi); the third step of calculating a direction similarity matrix; the fourth step of calculating the symmetry similarity of each parameter; and the fifth step of searching a hyperparameter with the smallest symmetry similarity, and taking thehyperparameter as an optimal parameter for returning. The parameter optimization process provided by the method does not need the artificial setting of other parameters, and the optimization processhas no subjective interference; the optimization process does not require data division and has high efficiency and certainty; and main calculation parts of the method are relatively independent and can be processed in parallel on large data. The efficiency of the method is 6-8 times that of a cross validation method.

Description

technical field [0001] The invention belongs to the field of machine learning optimization modeling, and in particular relates to a regression model-oriented hyperparameter optimization method. Background technique [0002] In the context of the rapid growth of data and information, machine learning, as the core driving force of data mining, has become an indispensable key link in the process of knowledge extraction. Inferring the correspondence between input data and output data based on empirical data is an important class of problems to be solved by machine learning. This type of problem is a regression prediction problem when both the input data and the output data are numerical. For example, predict precipitation based on conditions such as temperature and humidity, predict electricity load based on the population, GDP and consumer index of a certain place, and use the closing price, increase and trading volume of the stock index futures index on the previous day to pr...

AI Technical Summary

Problems solved by technology

However, the training and verification process requires random partitioning of the data, which brings uncertainty in error estimation and non-uniqueness in parameter selection; in addition, the data partitioning, training and verification steps increase the computational complexity

When the computing power is limited and the actual demand is relatively urgent (such as real-time prediction of short-term traffic flow), this parameter optimization method seems to be unable to do what it wants.

Images