Collaborative edge caching algorithm based on deep reinforcement learning in ultra-dense network

Abstract

The invention discloses a collaborative edge caching algorithm based on deep reinforcement learning in an ultra-dense network. The collaborative edge caching algorithm comprises the following specificsteps: 1, setting each parameter of a system model; and 2, making an optimal cache decision for each SBS by adopting a Double DQN algorithm so as to maximize the total content cache hit rate of all the SBSs. According to the algorithm, a DQN algorithm and a Double Q-learning algorithm are combined, so that the over-estimation problem of the DQN algorithm on a Q value is effectively solved. In addition, the algorithm adopts a priority experience playback technology, so that the learning speed is increased. The method further comprises a step 3, making an optimal bandwidth resource allocation decision for each SBS by adopting an improved branch and bound method so as to minimize the total content downloading delay of all user equipment. According to the method, the content downloading delayof all users in the ultra-dense network can be effectively reduced, the content cache hit rate and the spectrum resource utilization rate are improved, and the method has good robustness and expandability and is suitable for the large-scale user-intensive ultra-dense network.

Description

technical field [0001] The invention relates to a cooperative edge caching algorithm based on deep reinforcement learning in an ultra-dense network, and belongs to the field of edge caching in an ultra-dense network. Background technique [0002] In the 5G era, with the popularity of smart mobile devices and mobile applications, mobile data traffic is experiencing explosive growth. In order to meet the requirements of high capacity, high throughput, high user experience rate, high reliability, and wide coverage of 5G networks, ultra-dense networks (Ultra-Dense Networks, UDN) came into being. UDN densely deploys low-power small base stations (Small Base Stations, SBS) in indoor and outdoor hotspot areas (such as office buildings, shopping malls, subways, airports, tunnels, etc.) within the coverage of MBS (Macro Base Station, MBS) to improve Network capacity and spatial multiplexing, while making up for blind areas that cannot be covered by macro base stations (MBS). [000...

AI Technical Summary

Problems solved by technology

However, the traditional DQN algorithm usually overestimates the Q value, so a DoubleDQN algorithm is used, which is based on the Double Q-learning algorithm, which can effectively solve the overestimation problem of the DQN algorithm

In addition, the traditional DQN algorithm usually uses random uniform sampling to extract experience samples from the experience playback memory to update the Q network, that is, each experience sample has the same probability of being selected, resulting in very few but particularly high-value experience samples. is efficiently utilized, so a Prioritized Experience Replay technique is used to solve the sampling problem, thus accelerating the learning speed

Images