Heterogeneous Internet of Vehicles user association method based on multi-agent deep reinforcement learning

Abstract

The invention discloses a heterogeneous Internet of Vehicles user association method based on multi-agent deep reinforcement learning, and the method comprises the steps: firstly modeling a problem into a partially observable Markov decision process, and then employing the idea of decomposing a team value function, and specifically comprises the steps: building a centralized training distributed execution framework, the team value function is connected with each user value function through summation so as to achieve the purpose of implicit training of the user value functions; and then, referring to experience playback and a target network mechanism, performing action exploration and selection by using an epsilon-greedy strategy, storing historical information by using a recurrent neural network, selecting a Huber loss function to calculate loss and perform gradient descent at the same time, and finally learning the association strategy of the heterogeneous Internet of Vehicles users. Compared with a multi-agent independent deep Q learning algorithm and other traditional algorithms, the method provided by the invention can more effectively improve the energy efficiency and reduce the switching overhead at the same time in a heterogeneous Internet of Vehicles environment.

Description

technical field [0001] The invention relates to the field of wireless communication technology, in particular to a user association method of heterogeneous vehicle networking based on multi-agent deep reinforcement learning. Background technology [0002] With the rapid development of economy in recent years, the number of cars in the world is increasing day by day. While cars bring convenience to people, the probability of traffic congestion and traffic accidents is also greatly increased. Therefore, vehicular ad hoc networks (VANETs) are born. VANETs uses advanced wireless communication technology and perception technology to connect vehicles, pedestrians and roads within a certain range, comprehensively perceive traffic and roads, and form a special mobile ad hoc network (Cao s, Lee V c.an accurate and complete performance modeling of the ieee802.11p MAC sublayer for VANET [J]. Computer communications, 2020149:107-120). [0003] Due to the high mobility of the Internet of vehi...

AI Technical Summary

Problems solved by technology

However, the above-mentioned single-agent method requires a large amount of or almost complete information for central decision-making, which not only makes it difficult to realize due to the large computational dimension, but also usually generates a lot of unnecessary communication overhead, so further research is needed on how to use less information and resources for better performance

Images