A convolutional echo state network time sequence classification method based on a multi-head self-attention mechanism

Abstract

The invention discloses a convolutional echo state network time sequence classification method based on a multi-head self-attention mechanism. The method is realized based on two components of an encoder and a decoder. The encoder is composed of an echo state network of a multi-head self-attention mechanism. An input time sequence is firstly subjected to high-dimensional mapping coding through a plurality of echo state networks to generate original echo state characteristics, and then re-coding of echo state characteristic global space-time information is achieved based on a self-attention mechanism, so that the re-integrated high-dimensional characteristic representation has higher discrimination capability; And finally, taking a shallow convolutional neural network as a decoder to realize high-precision classification. The classification model established by the method inherits the characteristic of high training efficiency of the echo state network, realizes the re-coding of the space-time characteristic information by introducing a self-attention mechanism model without additional parameters, achieves a high-precision time sequence classification effect, and is a simple and efficient model.

Description

technical field [0001] The invention relates to the technical fields of reserve pool calculation and neural network research, in particular to a method for time series classification of convolutional echo state networks based on a multi-head self-attention mechanism. Background technique [0002] As one of the recurrent neural networks, the echo state network plays a very important role in the field of time series prediction. The general architecture of traditional echo state networks mainly consists of three components: input layer, hidden layer and output layer. Thanks to the memory capability of the high-dimensional reserve pool, for simple time series forecasting, the echo state network can usually achieve good forecasting performance from simple ridge regression. [0003] As the application of echo state network in time series prediction continues to expand, some researchers try to extend echo state network to time series classification tasks. It is worth noting that ...

AI Technical Summary

Problems solved by technology

Although these works have made some progress, the echo state features generated by the traditional echo state network can only capture short-term information representation, and the loss of feature information in the global space will affect the performance of these models in time series classification to a certain extent. Performance in

Therefore, classification methods based on traditional echo state networks cannot achieve satisfactory performance in some complex time series modeling tasks

Images