Internet of Things information freshness optimization method based on dual-network deep reinforcement learning

Abstract

The invention discloses an Internet of Things information freshness optimization method based on dual-network deep reinforcement learning. The method comprises the steps: enabling a sensor to operate a specified experience number, selecting actions according to a strategy in each time step of each period of experience, executing the actions, observing rewards and states, and storing the rewards and states in an experience playback pool; then taking out a batch of experiences from the experience playback pool by the model, calculating a loss function by using a target value calculation formula in an average cost form, updating the current network parameters, and updating the target network parameters every certain time step number; and cyclically executing the steps until the end of the period of experience. According to the method, the sensor can be helped to make an optimal decision in each time step, so that the weighted sum of the average information change age and the energy consumption is minimized. The purpose of maximizing the service life of equipment while the average information change age of the system is minimized is achieved. The Internet of Things information freshness optimization method based on dual-network deep reinforcement learning can be applied to the field of deep reinforcement learning.

Description

technical field [0001] The invention relates to the field of deep reinforcement learning, in particular to a method for optimizing the freshness of Internet of Things information based on dual-network deep reinforcement learning. Background technique [0002] With the deepening of the application scenarios of the Internet of Things system and the implementation of facilities such as the Internet of Vehicles, the real-time performance of the Internet of Things system is becoming more and more important. On the other hand, most of the existing real-time indicators only consider the change of information in the time dimension, and lack the consideration of the change of information in the content dimension, while the age of information change measures the change of information in both time and content dimensions. Variety. Since the age of information change is a random process that changes over time, its average value is often used as a performance indicator. Similar to avera...

AI Technical Summary

Problems solved by technology

At present, most of the common reinforcement learning optimization algorithms are discounted reinforcement learning algorithms, which cannot be directly applied to the optimization of the average cost problem, while the traditional reinforcement learning algorithm suitable for the average cost problem faces the curse of dimensionality problem, there are huge limitations

Images