Age of Information-Aware Radio Resource Management in Vehicular Networks

In this paper, we investigate the problem of age of information (AoI)-aware radio resource management for expected long-term performance optimization in a Manhattan grid vehicle-to-vehicle network. With the observation of global network state at each scheduling slot, the roadside unit (RSU) allocates the frequency bands and schedules packet transmissions for all vehicle user equipment-pairs (VUE-pairs). We model the stochastic decision-making procedure as a discrete-time single-agent Markov decision process (MDP). The technical challenges in solving the optimal control policy originate from high spatial mobility and temporally varying traffic information arrivals of the VUE-pairs. To make the problem solving tractable, we first decompose the original MDP into a series of per-VUE-pair MDPs. Then we propose a proactive algorithm based on long short-term memory and deep reinforcement learning techniques to address the partial observability and the curse of high dimensionality in local network state space faced by each VUE-pair. With the proposed algorithm, the RSU makes the optimal frequency band allocation and packet scheduling decision at each scheduling slot in a decentralized way in accordance with the partial observations of the global network state at the VUE-pairs. Numerical experiments validate the theoretical analysis and demonstrate the significant performance improvements from the proposed algorithm.

Chen Xianfu, Wu Celimuge, Chen Tao, Zhang Honggang, Liu Zhi, Zhang Yan, Bennis Mehdi

Publication type:
A1 Journal article – refereed

Place of publication:

deep reinforcement learning, long short-term memory, Markov decision process, multi-user resource scheduling, Q-function decomposition, Vehicular communications


Full citation:
X. Chen et al., “Age of Information Aware Radio Resource Management in Vehicular Networks: A Proactive Deep Reinforcement Learning Perspective,” in IEEE Transactions on Wireless Communications, vol. 19, no. 4, pp. 2268-2281, April 2020. doi: 10.1109/TWC.2019.2963667


Read the publication here: