Personalized federal learning method based on multi-head attention mechanism

Abstract

The invention discloses a personalized federal learning method based on a multi-head attention mechanism. The method is characterized in that the method comprises the following steps: 1, building a federated learning local model multi-head attention mechanism model: enabling the multi-head attention mechanism to be used in a most classical convolutional neural network, retaining key information through the multi-head attention mechanism, and carrying out feature extraction and selection better to improve the recognition accuracy; and 2, building a federal learning global model multi-head attention mechanism model: considering the influence of the personalized problem of each local model on the global model, and making corresponding changes according to the personalized characteristics of the model when global model parameters are sent to the local models. The correlation of extracted features and the data difference brought by reasonable individuation of each client can be considered, so the individuation degree of the data can be increased on the basis of ensuring the improvement of the accuracy.

Description

technical field [0001] The invention belongs to the technical field of privacy protection and data security, and in particular relates to a personalized federated learning method based on a multi-head attention mechanism. Background technique [0002] In the field of artificial intelligence, people's attention to privacy protection and data security is also increasing. As a distributed machine learning / deep learning framework that protects data privacy, federated learning It provides a good solution to problems such as structure and unbalanced data distribution. At this stage, machine learning and deep learning have also achieved great success in various fields, laying the foundation for the better performance of the federated learning algorithm model. [0003] However, for the sake of convenience, the conventional federated learning algorithm only averages the parameters of each local model, and neither considers the correlation of extracted features nor considers the data...

AI Technical Summary

Problems solved by technology

[0003] However, the conventional federated learning algorithm only averages the parameters of each local model for convenience, and neither considers the correlation of extracted features nor considers the data differences brought about by the reasonable personalization of each client.

The local model parameters of local data after a certain number of iterations of local model training will have an impact on the global model. Similar participants may have similar local training data, while randomly selected participants may have different training data. Similar data in a certain iteration There are differences between the local model trained by the number of times and the local model trained by different training data, so the data with large training differences will produce very different model parameter updates. Therefore, its existence does not take into account the correlation of extracted features, It also does not take into account the problem of data differences brought about by the reasonable personalization of each client

Images