A Sequence Classification Method Based on Hierarchical Multiscale Recurrent Neural Network

Abstract

The invention discloses a sequence classification method based on a hierarchical multi-scale cyclic neural network, comprising the following steps: inputting a sequence and dividing it into a plurality of equal-length subsequences; constructing a plurality of pyramid structures according to the sequence of subsequences, each A pyramid receives a sub-sequence as input, and generates a hidden state at the bottom of the pyramid and a hierarchical aggregation state at each level. The aggregation state at the top of each pyramid is used as the input at the bottom of the next sub-pyramid; The aggregation state is iteratively aggregated through the skip connection to obtain the output of this layer; the hierarchical aggregation state sequence of different scales generated by all the pyramids in the lower layer is used as the input of the upper layer to construct a multi-layer cyclic neural network to generate the output of each layer; The output of one layer obtains multi-scale fusion features; finally, based on this feature, a Softmax layer is used to classify the sequence. The invention achieves higher accuracy rate in sequence classification.

Description

technical field [0001] The invention relates to the technical field of sequence classification in deep learning, in particular to a sequence classification method based on a hierarchical multi-scale recurrent neural network. Background technique [0002] In the field of deep learning, Recurrent Neural Network (RNN) is often used to model sequence data, which can capture time dependence in sequence data by using cyclic feedback connections. It has achieved good performance in many applications, such as time series classification, text classification and speech recognition, etc. [0003] RNN is usually trained using the Back Propagation Trough Time (BPTT) method. However, in practice, it is difficult for RNN to pass BPTT training and the problem of gradient disappearance or gradient explosion will occur for long sequences, and it is difficult to capture sequence data. long-term dependence. The gradient explosion can be alleviated by simple gradient clipping, but the gradient...

AI Technical Summary

Problems solved by technology

Unfortunately, hierarchical RNNs always have a multi-layer structure, which is harder to train and more prone to vanishing gradients than single-layer RNNs

Images