Data processing method and system

Abstract

The invention relates to a data processing method and system. The processing method comprises the steps of configuring a to-be-configured model structure through multiple groups of preset hyper-parameters to obtain multiple to-be-tested models; sending the plurality of to-be-tested models, the training data and the verification data to a cloud computing platform, so as to enable the cloud computing platform to train each to-be-tested model through the training data, and obtaining convergence information obtained by verifying each trained to-be-tested model through the verification data by thecloud computing platform; and according to all the received convergence information, determining a target hyper-parameter corresponding to the to-be-tested model with the highest convergence speed, and taking the to-be-tested model with the highest convergence speed as a final model. By adopting the method in the embodiment, the obtained to-be-tested model can be trained through the cloud computing platform, the final model with the highest convergence rate and the target hyper-parameter are selected from the to-be-tested model, and the hardware building cost of a local end can be reduced.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, in particular to a data processing method and system. Background technique [0002] With the development of artificial intelligence technology, the application scenarios and scope of deep learning are becoming more and more extensive. In deep learning, some parameters that have an important impact on the results must be manually specified in advance. In machine learning, this class is a parameter that sets the value before starting the learning process, rather than the parameter data obtained through training. Hyperparameters: for example: Learning rate; however, hyperparameters are currently obtained based on the experience of algorithm personnel. Because of the complexity of deep learning models, manual specification is often time-consuming and laborious. [0003] The work of manual parameter adjustment first has extremely high requirements on the skills of algorithm ...

AI Technical Summary

Problems solved by technology

However, with this method, since a large number of parameter values ​​need to be combined and traversed during implementation, and a large amount of training is required for the model, it will occupy a large amount of computing resources. In order to obtain the optimal hyperparameters as soon as possible, it is necessary to build computing power locally. Strong hardware equipment, which will cause high hardware construction costs and seriously affect the control of the company's operating costs

[0005] Aiming at the problem in related technologies that building hardware devices for automatic parameter adjustment at the local end will cause high operating costs for enterprises, no effective solution has been provided so far

Images