354results about How to "Avoid vanishing gradients" patented technology

The invention relates to a code recommendation method based on a long short-term memory (LSTM) network. For the problem that low recommendation accuracy rates, low recommendation efficiency and the like are ubiquitous in existing code recommendation technologies, the method firstly extracts source code to form an API sequence, utilizing the long short-term memory network to build a code recommendation model to learn relationships between API calls, and then carries out code recommendation. A dropout technology is used to prevent model overfitting. At the same time, using a ReLu function to instead a traditional saturation function is provided, the gradient vanishing problem is solved, a model convergence speed is accelerated, model performance is improved, and advantages of the neural network are fully exerted. The technical scheme of the invention has the characteristics of simpleness and quickness, and can better improve an accuracy rate and recommendation efficiency of code recommendation.

The invention discloses a text feature extraction method based on a fusion model, a device, a chat robot and a storage medium. The extraction method comprises the following steps: S1, adopting Word2vec model to extract words, utilizing Skip-gram model to construct a word vector, and then maps the sentences composed of words into a sentence matrix; S2, using convolution neural network to extract the local features of the word vector; S3, extracting the global features related to the context of the word vector by using the BLSTM model; S4, extracting the deeper information features of the localfeatures and the global features through Attention mechanism, and fusing the extracted features; S5, extracting the text feature vector extracted from the network layer by layer by using soft-Max classifier for text classification. The invention not only solves the problem that the single convolution neural network ignores the semantic information of the words in the context, but also effectivelyavoids the problem that the gradient of the traditional circulating neural network disappears or the gradient is dispersed.

The invention discloses a fast magnetic resonance imaging method based on residual U-net convolutional neural network and the method comprises three steps of preparing training data, carrying out training based on the residual U-net convolutional neural network, and carrying out image reconstruction based on the residual U-net convolutional neural network. By adding the residual module into the U-net convolutional neural network, the problems of gradient disappearance, overfitting, low convergence speed and the like of the U-net convolutional neural network can be solved, and the quality of rapid MRI imaging based on the U-net convolutional neural network is improved.

An urban scale taxi track prediction method based on an attention mechanism comprises the following steps that 1, GPS data are collected, and noise appearing in the GPS data collection process is subjected to data cleaning; step 2, map matching is carried out on the cleaned GPS data to obtain experimental data required by the model; step 3, model training; an LSTM network is adopted at an encodingend of the encoder, an encoding vector C is used as the LSTM network of the decoding end at a decoding end, an attention mechanism is applied to a hidden layer vector of the encoder, and a predictedvalue at the previous moment is used as an input value at the current moment each time and is sent into the decoder. According to the method, the embedded vector is adopted to represent the road section information in the urban area, the encoder is used for encoding the track of the taxi, the decoder containing the attention mechanism is used for predicting the track, the correlation in the tracksequence is fully mined, and the track prediction accuracy can be effectively improved.

The invention relates to the field of wind power plant wind speed prediction, and discloses a wind speed prediction method and system based on a long-term and short-term memory time neural network, and the method comprises the steps: A) collecting existing historical wind speed data and actually measured wind speed data; b) analyzing the bad data, carrying out data preprocessing to obtain a wind speed sample, and constructing a wind speed sample database; c) dividing the wind speed sample database into a training sample database, a verification sample database and a detection sample database;d) constructing a long-term and short-term memory neural network model; and E) predicting the wind speed by using the trained long-short-term memory neural network model to obtain a wind speed prediction value. Under the condition of completely utilizing historical data, the wind energy prediction is realized by establishing the long-short-term memory time neural network model and optimizing parameters of the long-short-term memory neural network model, and the prediction precision is high.

The invention relates to the technical field of computer image processing, specifically to a remote-sensing image super-resolution reconstruction method based on a generative adversarial network. Theremote-sensing image super-resolution reconstruction method comprises the following steps: constructing a remote-sensing image super-resolution reconstruction model consisting of a generator network and a discriminator network; introducing a scene constraint sub-network into the generator network to solve the problem of scene change, introducing an edge enhancement sub-network to solve the problemof edge transition smoothness of a generated image, introducing TV loss for noise suppression, and introducing a content fidelity to deal with the problems of instability and gradient disappearance in the training process; and introducing spectrum normalization into a discriminator network to control the performances of a discriminator so as to promoting better learning of the generator. The remote-sensing image super-resolution reconstruction method provided by the invention has the following advantages: a high-quality high-resolution remote-sensing image can be generated based on a low-resolution remote-sensing image; the precision of the low-resolution remote-sensing image in classification detection is effectively improved; the problems of edge transition smoothness and scene change in super-resolution of the remote-sensing image are solved; meanwhile, the problems of training instability and gradient disappearance under the GAN network are solved.

The invention relates to the in-depth learning field, and especially relates to a Convolution Neural Network (CNN) structure improving method; the method comprises the following steps: a, using a fractional max pooling (FMP) principle to change a maximum value pooling layer in a conventional CNN structure into fractional orders, thus realizing sampling dimension reduction under a random dimension; b, ensuring a shallow network structure, continuously widening the network structure, combining the fractional order maximum value pooling layer, and thus improving the network performance. The method uses the fractional order maximum value pooling principle to ensure the layer at a shallow level, can continuously widens the network structure, thus preventing a deep network to have gradient vanish and weight failure phenomenon in a training process, and causing the CNN structure hard to train. The method can realize equal or better performance with the deep CNN structure, and uses less network parameters, thus providing obvious performance advantages.