Traffic flow prediction parallel method based on chaos and reinforcement learning

A technology of reinforcement learning and traffic flow, applied in the field of traffic flow prediction based on chaos and reinforcement learning, can solve problems such as insufficient parallelization and complex structure, achieve fast learning and adjustment, best prediction results, strong interpretability and the effect of the ability to adjust online

Pending Publication Date: 2022-05-10
HARBIN UNIV OF SCI & TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, since most of the parallel computing frameworks are aimed at the problems in the background of big data, the learning of reinforcement learning models and frequent parameter updates present problems such as complex structures and insufficient parallelization.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Traffic flow prediction parallel method based on chaos and reinforcement learning
  • Traffic flow prediction parallel method based on chaos and reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0099] Example 1: Preprocessing of traffic flow data.

[0100] Step1_1, set the embedding dimension m=3 and time delay k=2 of the initialized chaotic time series;

[0101] Step1_2, calculate the maximum Lyapunov exponent of the reconstructed traffic flow time series data by Wolf method, and analyze the chaotic characteristics of the traffic flow time series;

[0102] Step1_3, initialize the chaotic model and generate a comparative chaotic time series, the chaotic model is X i+1 =4X i (1-X i ), where X 1 = 0.1;

[0103] Step1_4, initialize reconstruction and compare the embedding dimension md=3 and time delay kd=2 of chaotic time series data;

[0104] Step1_5, standardize the traffic flow time series and comparative chaotic time series, set the traffic flow time series data and comparative chaotic time series as follows:

[0105] T = [3,6,8,5,7,10,5];

[0106] Td = [0.3600, 0.9216, 0.2890, 0.8219, 0.5854, 0.9708, 0.1133];

[0107]Among them, the mean value of traffic fl...

example 2

[0118] Example 2: Constructing a reinforcement learning environment.

[0119] Step2_1, the data of the preprocessed traffic flow data training set is used as the state space in the environment and sequenced according to time

[0120] Arranged in order, the settings are as follows:

[0121] S 1 =T1=[-0.67,0.34,0.14],

[0122] S 2 =T2=[-0.06,-0.27,0.74],

[0123] S 3 =T3=[0.34,0.14,-0.27],

[0124] …,

[0125] S n =Tn=[0.64,0.25,-0.56];

[0126] Step2_2, perform difference operation on the last one-dimensional data of the adjacent state space in turn to obtain the range of the action space, that is, set:

[0127] T=[3,6,8,5,7,10,5,…,9,10],

[0128] t c1 =10-7=3,

[0129] t c2 =5-10=-5,

[0130] …,

[0131] t cn =10-9=1,

[0132] Then set the action space range as: [-5,3], and the standard deviation of the difference is 0.36;

[0133] Step2_3, with t ci As the center, the reward of the action space corresponding to the state Si is distributed according to the n...

example 3

[0153] Example 3: Initialize the neural network model structure and update method.

[0154] Step3_1, initialize the actor network structure. Since the actor network is used to estimate the action strategy of the agent and the strategy is continuous, the number of input neurons of the actor network is m, which is the state dimension of the environment. When the environment is a training environment, m is the training environment. The state dimension of the environment. When the environment is a comparison environment, m is the state dimension of the comparison environment. The middle layer network structure uses a neural network with a CRU structure, and the output is [d min , d max ] and use the softmax activation function to build the model, for example, if the reconstructed state is set to S=[-0.59,0.42,0.38,0.81], and the action space is [-2,3], then the input neuron The number is 4, the optional action is [-2, -1, 0, 1, 2, 3], and the probability distribution of the corre...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a traffic flow prediction parallel method based on chaos and reinforcement learning. Comprising the following steps: 1, carrying out preprocessing and chaotic analysis on data, generating a comparison chaotic time sequence through a chaotic model, and carrying out reconstruction, standardization and data set division; 2, performing reinforcement learning training and comparison environment construction by using the preprocessed data; 3, constructing an act-critic neural network model to learn an intelligent agent strategy and judge a behavior value; and 4, generating a plurality of training models by the slave process under the parallel framework to interact with the environment, realizing parallel updating through dispersion comparison and reward with the central neural network model of the host process, and finally performing prediction verification by the host process. According to the method, the traffic flow is predicted by adopting reinforcement learning and a chaotic time sequence, and compared with a traditional statistical prediction method, the method has higher interpretability and online adjustment learning ability; parallel reinforcement learning can learn and adjust more quickly, and an optimal prediction result is generated.

Description

technical field [0001] The invention belongs to the fields of chaos theory, reinforcement learning and parallel computing, and in particular relates to a parallel method for predicting traffic flow based on chaos and reinforcement learning. Background technique [0002] With the development of the economic level, the ownership of private cars in China has increased sharply, and the road traffic situation has become more complex, so the prediction of traffic flow has become very important. Through the traffic flow prediction, the road conditions can be planned in advance and the traffic conditions can be guaranteed to the greatest extent. With the development of big data environment and smart city system, the collection of traffic flow prediction data and the dynamic adjustment of prediction methods are becoming more and more important. increasingly real-time. At present, one kind of traffic flow prediction is to use traditional statistical methods to predict the probability...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G08G1/065G06N3/02G06N3/08
CPCG08G1/065G06N3/02G06N3/08
Inventor 刘嘉辉杜金仇化平
Owner HARBIN UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap