Hybrid federated learning method based on knowledge transfer

Abstract

The invention discloses a hybrid federated learning method based on knowledge transfer. The method comprises the following steps: introducing an incentive mechanism in federated learning, firstly uploading, by each piece of equipment, data distribution conditions, and making, by a server, a decision according to demand information of current data, so that a data selection algorithm based on mutual information is performed, and corresponding rewards are given to the equipment uploading the data; then performing training in a deep learning model which is the same as each piece of local equipment by utilizing collected shared data to obtain an auxiliary model; and transferring general knowledge of the auxiliary model to an aggregation model. According to the invention, in different federated training rounds, different transfer methods are used to transfer the general knowledge of the auxiliary model to the aggregation model according to the equipment aggregation model and the auxiliary model, so that an optimized global model is obtained; and the ability to distinguish general features can be provided for the aggregation model in a few rounds, so that local rounds of an equipment model are reduced, and rapid convergence and high accuracy of the global model are realized.

Description

technical field [0001] The invention belongs to the technical field of deep learning, and in particular relates to a hybrid federated learning method. Background technique [0002] In recent years, as the importance of privacy protection has gradually emerged, federated learning has become a popular machine learning paradigm in which a large number of terminal devices jointly train deep learning models while ensuring that data does not flow out of the local area. At present, horizontal federated learning integrates the advantages of terminal localization without requiring a large amount of data transmission and massive IoT devices, and sinks the inference process of the model from the cloud to the edge close to the user, while enhancing data privacy and avoiding unstable network conditions. Impact, improving the response time of services has become an ideal research direction. However, considering the heterogeneous data distribution of devices and unstable network connectio...

AI Technical Summary

Problems solved by technology

The disadvantage of the existing technology is that it only tries to weaken the data heterogeneity of devices, but does not fundamentally solve the problem of data heterogeneity

Images