A training method for horizontally federated xgboost decision trees

Abstract

The invention discloses a training method for a horizontal federated xgboost decision tree, which relates to the technical field of machine learning and federated learning decision trees. A horizontal federated decision tree scheme is constructed by using threshold homomorphic encryption; The state-of-the-art encryption technology improves the strength of privacy protection, reduces the number of communication rounds and traffic, and improves the performance of the solution running in the Internet environment. The main technical scheme of the present invention is: constructing a system structure including an aggregator and a plurality of participants in advance, and then using the threshold homomorphic encryption technology to construct a horizontal federated learning, and performing the gradient data of the aggregator and each participant in the training process. Encryption, and then jointly decrypt the aggregated ciphertext data to obtain a jointly trained decision tree model while protecting the data privacy of all parties.

Description

technical field [0001] The invention relates to the technical field of federated learning and machine learning decision trees, in particular to a training method for horizontal federated xgboost decision trees. Background technique [0002] When the data is scattered in different data sources, and multiple parties need to jointly conduct xgboost decision tree training to improve the accuracy of the model, usually, multiple data owners will share the data and then conduct regular decision tree training; but if In this way, in the case of plain text, the data cannot be effectively protected; when data privacy and confidentiality are involved, methods such as privacy calculation can be used to complete the joint decision tree training method of multi-party data, for example, combined with the concept of federated learning, multiple Data owners jointly train machine learning models under the premise of protecting their data privacy. [0003] At present, among the existing horiz...

AI Technical Summary

Problems solved by technology

[0002] When the data is scattered in different data sources, and multiple parties need to jointly conduct xgboost decision tree training to improve the accuracy of the model, usually, multiple data owners will share the data and then conduct regular decision tree training; but if In this way, in the case of plain text, the data cannot be effectively protected; when data privacy and confidentiality are involved, methods such as privacy calculation can be used to complete the joint decision tree training method of multi-party data, for example, combined with the concept of federated learning, multiple Data owners jointly train machine learning models under the premise of protecting their data privacy

[0003] At present, among the existing horizontal federated learning technical solutions, one class uses only uploading plaintext gradients to the aggregator, which is often used for joint training of linear models and neural networks. The assumption of this type of method to protect data privacy is that gradient learning cannot reveal the original data, but the training process of the decision tree will involve the gradient information of a large number of sub-data sets, so the joint training of this type of method for the decision tree will leak too much original information of the data owner; the other is the use of secure multi-party computing (secure multi-party computation, MPC) technology performs secret sharing of data and then executes the corresponding training circuit, but the number of communication rounds in the training process of this type of method is too high

As mentioned above, whether it is horizontal federated learning based on plaintext gradient or horizontal federated learning based on secure multi-party computing, there are unavoidable deficiencies or defects in decision tree training based on multiple data sources.

Images