Text classification method and system for lifelong learning

Abstract

The invention provides a lifelong learning text classification method and system, and the method comprises the steps: taking a pre-training language model as a text classification model, and carrying out the training of the text classification model through a task sequence, and obtaining a lifelong learning model; setting a storage memory; the method comprises the following steps of: firstly, carrying out sparse experience playback on a lifelong learning model to store memory, carrying out fine tuning on the lifelong learning model by using a sparse experience playback mode according to playback frequency and a storage rate, carrying out fine tuning on the lifelong learning model again, and then carrying out prediction output on an input text by using the lifelong learning model, thereby realizing the following purposes: firstly, reducing time complexity so as to save training time; the text classification accuracy is improved; and disastrous forgetting is relieved.

Description

technical field [0001] The invention belongs to the technical field of text data processing and character processing, and in particular relates to a text classification method and system for lifelong learning. Background technique [0002] Lifelong learning is a continuous learning process, and its research fields include lifelong supervised learning, lifelong reinforcement learning, open learning, etc. This paper mainly studies in the field of lifelong supervised learning, when the model is trained on new task data after completing a series of N supervised learning tasks, the neural network usually forgets the knowledge learned in the previous task, which means that the weight of the new task The weights of previous tasks may be overwritten, thereby reducing the performance of the model on previous tasks and causing catastrophic forgetting. After learning the knowledge of the first N categories, the lifelong learning text classification model has the ability to continue le...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to propose a text classification method and system for lifelong learning, to solve one or more technical problems in the prior art, and at least provide a beneficial choice or create conditions

Images