Distributed data processing method and device, computer equipment and storage medium

Abstract

The invention relates to a distributed data processing method and device, computer equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the steps that model training information sent by at least two edge node devices is acquired, wherein the model training information is transmitted in a plaintext form, and the model training information is obtained by training a sub-model by the edge node equipment in a differential privacy mode; on the basis of the model training information sent by the at least two pieces of edge node equipmentrespectively, sub-models obtained by training the at least two pieces of edge node equipment respectively are obtained; and on the basis of a specified model integration strategy, model integration iscarried out on the sub-models obtained by training the at least two pieces of edge node equipment to obtain a global model. By means of the scheme, on the premise that data privacy safety is guaranteed, the model integration mode is expanded, and the model integration effect is improved.

Description

technical field [0001] The embodiments of the present application relate to the technical field of artificial intelligence, and in particular to a distributed data processing method, device, computer equipment, and storage medium. Background technique [0002] With the continuous development of artificial intelligence and the continuous improvement of user privacy requirements, the application of machine learning model training based on distributed systems is becoming more and more extensive. [0003] Federated learning is a machine learning method for distributed systems based on cloud technology. In the federated learning architecture, it includes a central node device and multiple edge node devices, and each edge node device stores its own training data locally. Federated learning includes horizontal federated learning. Horizontal federated learning is to obtain respective model gradients based on local training data in multiple edge node devices, encrypt each model grad...

AI Technical Summary

Problems solved by technology

[0004] In the above technical solution, in order to protect the security of the training data, the model gradient needs to be encrypted, which leads to the central node device can only use the security aggregation algorithm for model fusion, which affects the model integration effect of federated learning

Images