Privacy protection method and system based on knowledge migration under collaborative learning framework

Abstract

The invention provides a privacy protection method and system based on knowledge migration under a collaborative learning framework. The method comprises the following steps: segmenting a local privacy data set into a plurality of disjoint privacy subsets, and training a corresponding privacy model based on each privacy subset; collaborative learning: submitting the public data to a privacy model,using an aggregation mechanism by the privacy model, and migrating knowledge to a public data set to label a label; after enough labeled data is obtained, a local interaction model is trained, part of parameters are uploaded to a server in each round of training, the server updates and maintains global parameters and provides latest parameters for all parties to download, and participants download the latest parameters to optimize the local interaction model. According to the invention, multiple parameter interactions of collaborative learning can also be ensured, and even if the interactionmodel obtains less annotation data, high accuracy can still be maintained after multiple rounds of training.

Description

technical field [0001] The present disclosure relates to the technical field of information processing, in particular to a privacy protection method and system based on knowledge transfer under a collaborative learning framework. Background technique [0002] As one of the important fields of artificial intelligence, deep learning has developed rapidly in recent years with the help of the depth and parameter advantages of the model. Especially in the face of large-scale data volume, compared with traditional machine learning methods, deep learning models can often achieve better results, and they have also achieved great success in image recognition, natural language processing and other fields. However, the need for large amounts of data for training has also become a drawback of deep learning. First of all, in real life, tens of thousands of data are generated every day. These data are often unlabeled and cannot be directly used as training sets for deep learning. They ar...

AI Technical Summary

Problems solved by technology

First of all, in real life, tens of thousands of data are generated every day. These data are often unlabeled and cannot be directly used as training sets for deep learning. They are also mixed with many erroneous or low-value samples. The data set spends a lot of manual screening and labeling

2. For some small organizations, due to economic and other reasons, giant companies such as Google and Apple cannot obtain sufficient and comprehensive data to train models.

With limited data, it is difficult for the deep learning model to obtain high accuracy, and the generalization ability of the model is also curbed

3 Even for large companies, the traditional training method needs to upload the local data of each user to the central server, and the central server will integrate data resources, use user data to train a global model, and then distribute it to each user, but this will also bring corresponding problems. Privacy issues. These companies hold users’ private data for a long time. Once attacked, a large amount of private information will be leaked. For example, hackers stole 50 million user account information through program loopholes in 2018. In addition, personal private information may also be illegally obtained by large companies. use, trade

For example, Hitiaj and others pretended to be honest participants in the collaborative training process to obtain the uploaded parameters of the victim, and then restored the original data of the victim through the local generation of confrontation network GAN, even if some privacy protection technologies such as Differential privacy cannot defend against such attacks

There are also some attacks such as "membership attack" member reasoning attack, which also collects relevant information by analyzing the internal parameters of the model or malicious repeated queries to determine whether a piece of data is used to train the model, which will also lead to a certain degree of privacy leakage

Images