Continuous learning framework and continuous learning method of deep neural network

Abstract

The embodiment of the invention provides a continuous learning framework and a continuous learning method for a deep neural network, and the framework comprises a condition generator network which isused for generating generation data of the same category as training data of a current task, and distributing a specific parameter subspace for a current task during training; a discriminator networkwhich is used for supervising the generation process of the generated data to enable the generated data to gradually approach the training data of the old task, and taking the approximate generated data as equivalent training data of the old task; and a classifier network which comprises an independent classifier network and an auxiliary classifier network of the discriminator network, and is usedfor selectively keeping parameters of the coded old task by using a weight consolidation mechanism, and continuously updating and jointly training the current task by using the training data of the current task and the equivalent training data of the old task. According to the embodiment of the invention, disastrous forgetting of old tasks in the continuous learning process can be effectively relieved, and the continuous learning ability is improved.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, and more specifically, to a continuous learning framework and a continuous learning method of a deep neural network. Background technique [0002] Acquiring the ability to continuously learn new information is one of the fundamental challenges facing deep neural networks, since continuous acquisition of information from dynamically distributed data often leads to catastrophic forgetting. That is, in the process of learning new tasks, deep neural networks tend to adjust the parameters learned in old tasks, resulting in catastrophic forgetting of the ability to perform old tasks. [0003] According to the setting of continuous learning, the number of tasks learned by the deep neural network is continuously increasing, and the training data of each task cannot be obtained again after the training of the task is completed. To solve the problem of continuous learning tasks, th...

AI Technical Summary

Problems solved by technology

However, this method still has the above-mentioned problem of not being able to generate a large number of categories of complex data, and this method uses an auxiliary classifier to generate an adversarial network as the basic architecture, and the discriminator network and the auxiliary classifier share other layers of networks except the output layer. Under the setting of continuous learning, this structure has the problem of not being able to achieve the optimum at the same time, so it does not perform well on more complex image data sets

Images