Privacy protection method and system based on knowledge transfer under collaborative learning framework

Abstract

This disclosure proposes a privacy protection method and system based on knowledge transfer under the collaborative learning framework, including: dividing the local privacy data set into several privacy subsets that are mutually disjoint, and training a corresponding privacy model based on each privacy subset ;Collaborative learning: Submit public data to the privacy model, the privacy model uses the aggregation mechanism, and uses knowledge transfer to label the public data set; after obtaining enough labeled data, train the local interaction model, in each round of training Upload some parameters to the server, the server updates and maintains the global parameters and provides the latest parameters for all parties to download, and the participants download the latest parameters to optimize the local interaction model. The multi-parameter interaction of simultaneous collaborative learning in the present disclosure can also ensure that even if the interaction model acquires less labeled data, it can still maintain a high accuracy after multiple rounds of training.

Description

technical field [0001] The present disclosure relates to the technical field of information processing, in particular to a privacy protection method and system based on knowledge transfer under a collaborative learning framework. Background technique [0002] As one of the important fields of artificial intelligence, deep learning has developed rapidly in recent years with the help of the depth and parameter advantages of the model. Especially in the face of large-scale data volume, compared with traditional machine learning methods, deep learning models can often achieve better results, and they have also achieved great success in image recognition, natural language processing and other fields. However, the need for large amounts of data for training has also become a drawback of deep learning. First of all, in real life, tens of thousands of data are generated every day. These data are often unlabeled and cannot be directly used as training sets for deep learning. They ar...

AI Technical Summary

Problems solved by technology

First of all, in real life, tens of thousands of data are generated every day. These data are often unlabeled and cannot be directly used as training sets for deep learning. They are also mixed with many erroneous or low-value samples. The data set spends a lot of manual screening and labeling

2. For some small organizations, due to economic and other reasons, giant companies such as Google and Apple cannot obtain sufficient and comprehensive data to train models.

With limited data, it is difficult for the deep learning model to obtain high accuracy, and the generalization ability of the model is also curbed

3 Even for large companies, the traditional training method needs to upload the local data of each user to the central server, and the central server will integrate data resources, use user data to train a global model, and then distribute it to each user, but this will also bring corresponding problems. Privacy issues. These companies hold users’ private data for a long time. Once attacked, a large amount of private information will be leaked. For example, hackers stole 50 million user account information through program loopholes in 2018. In addition, personal private information may also be illegally obtained by large companies. use, trade

For example, Hitiaj and others pretended to be honest participants in the collaborative training process to obtain the uploaded parameters of the victim, and then restored the original data of the victim through the local generation of confrontation network GAN, even if some privacy protection technologies such as Differential privacy cannot defend against such attacks

There are also some attacks such as "membership attack" member reasoning attack, which also collects relevant information by analyzing the internal parameters of the model or malicious repeated queries to determine whether a piece of data is used to train the model, which will also lead to a certain degree of privacy leakage

Images