Sequence classification method based on hierarchical multi-scale recurrent neural network

Abstract

The invention discloses a sequence classification method based on a hierarchical multi-scale recurrent neural network, which comprises the following steps: inputting a section of sequence, and dividing the sequence into a plurality of sub-sequences with equal lengths; constructing a plurality of pyramid structures according to the sequence of the subsequences, receiving a section of subsequence asan input by each pyramid, generating a hidden state located at the tower bottom and a hierarchical aggregation state located at each level, and taking the aggregation state of the tower top of each pyramid as an input of the tower bottom of the next subpyramid; iteratively aggregating the aggregation states of all pyramid tops through jump connection to obtain the output of the layer; constructing a multi-layer recurrent neural network by using hierarchical aggregation state sequences with different scales generated by all low-layer pyramids as input of a high layer, and generating output ofeach layer; aggregating the output of each layer to obtain a multi-scale fusion feature; and finally, on the basis of the feature, using a Softmax layer to classify the sequence. According to the method, relatively high accuracy in sequence classification is achieved.

Description

technical field [0001] The invention relates to the technical field of sequence classification in deep learning, in particular to a sequence classification method based on a hierarchical multi-scale recurrent neural network. Background technique [0002] In the field of deep learning, Recurrent Neural Network (RNN) is often used to model sequence data, which can capture time dependence in sequence data by using cyclic feedback connections. It has achieved good performance in many applications, such as time series classification, text classification and speech recognition, etc. [0003] RNN is usually trained using the Back Propagation Trough Time (BPTT) method. However, in practice, it is difficult for RNN to pass BPTT training and the problem of gradient disappearance or gradient explosion will occur for long sequences, and it is difficult to capture sequence data. long-term dependence. The gradient explosion can be alleviated by simple gradient clipping, but the gradient...

AI Technical Summary

Problems solved by technology

Unfortunately, hierarchical RNNs always have a multi-layer structure, which is harder to train and more prone to vanishing gradients than single-layer RNNs

Images