Multi-source multi-modal data preprocessing method and system for shared learning

Abstract

The invention relates to a data processing technology, and aims to provide a multi-source multi-modal data preprocessing method and system for shared learning. The method comprises the following steps: performing data communication across servers; encrypting multi-source data; aligning privacy protection samples; and filtering the multi-source data to form final unified data. According to the invention, a preprocessing stage before training of the shared learning model is abstracted into a system, and a preprocessing process is completed step by step by utilizing a plurality of subsystems, sothat cross-server data communication, multi-source data encryption, privacy protection sample alignment and multi-source data filtering are realized, the shared learning system preprocessing stage issystematic and more specific and complete, the multi-source data communication, encryption, alignment and filtering method is provided, the execution efficiency is higher, the security is higher, andthe data privacy can be ensured; and the method is independent of a specific shared learning task, and can be integrated in any multi-source data processing task as a single module.

Description

technical field [0001] The invention relates to data processing technology, in particular to a multi-source and multi-modal data preprocessing method and system for shared learning. Background technique [0002] With the rise of artificial intelligence algorithms, data privacy issues are gradually being mentioned, and countries around the world are establishing and improving relevant laws to protect data security and privacy. In this context, the shared learning technology based on the principles of cryptography has emerged. It mainly guarantees the security of data in the case of multi-source data joint modeling through security proof. [0003] Artificial intelligence is driven by big data. AlphaGO used a total of 300,000 game data to achieve such results. So people began to expect such big data-driven AI to be applied in all aspects of life. However, the actual situation may be pessimistic: due to the limitations of various external and internal conditions, it is quite ...

AI Technical Summary

Problems solved by technology

Although these technologies can solve the problem of data preprocessing in the process of multi-source data joint modeling to a certain extent, each has certain shortcomings: some patents do not use encryption technology to easily expose data privacy, and some patents only use sample alignment instead of Other steps in the preprocessing link such as data screening are ignored, and some encryption techniques are time-consuming, which makes practical application difficult

Document 2 reduces the complexity of model calculation and the overhead of communication between participants, but its data preprocessing process is relatively simple, and the sample alignment process does not mention encryption technology, so it may cause user information leakage

Images