Method and system for testing robustness of artificial intelligence model

Abstract

The invention belongs to the technical field of artificial intelligence model robustness judgment, and discloses a method for effectively attacking an unknown model under the condition of not knowing an artificial intelligence model and a training data set. The method comprises the following steps: capturing shallow shared features of an original data set, and generating a corresponding target sample according to an input condition, wherein the artificial intelligence model to be tested is used for predicting the output of the target sample, and the output is compared with the formal label to obtain the corresponding loss, and the generated target sample is used for training the substitution model and obtaining corresponding output; comparing the output of the same sample under the condition of two different models to obtain comparison loss so as to ensure that the substitution model well learns all functions of the artificial intelligence model to be tested; employing some mature attack algorithms for attacking the substitution model to obtain corresponding adversarial samples, and employing the adversarial samples for attacking an unknown artificial intelligence model so as to judge the robustness of the artificial intelligence model.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence and machine learning, and more specifically relates to a method and system for testing the robustness of an artificial intelligence model. Background technique [0002] The artificial intelligence model is widely used in the daily life of the present invention, for example, face recognition, speech recognition, target tracking, image segmentation and so on. Despite the great success of deep neural networks, current research has shown that they are vulnerable to attacks that lead to wrong predictions of the final system, with serious consequences. Therefore, how to evaluate and improve the robustness of artificial intelligence models is of great significance. [0003] The current test scheme for the robustness of artificial intelligence models includes two implementations based on white-box systems and black-box systems, that is, attacking white-box or black-box systems, and counti...

AI Technical Summary

Problems solved by technology

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a method and system for testing the robustness of artificial intelligence models, the purpose of which is to solve the existing method of robustness testing of network models based on black boxes Due to the need to build a complex special algorithm to imitate the real data set close to the black box system, but the design of this algorithm is quite difficult, time-consuming and laborious technical problems, and the attack success rate of the black box attack algorithm is relatively low, so it cannot be better The technical problem of judging the robustness of the black-box model, and because the original training data set used to train the surrogate model is unknown or only partially known, the surrogate model constructed by this method cannot completely imitate the function of the black-box model , and will affect the technical issues of the accuracy of the final robustness test results,

Images