Age of Information-Aware Radio Resource Management in Vehicular Networks

In this paper, we investigate the problem of age of information (AoI)-aware radio resource management for expected long-term performance optimization in a Manhattan grid vehicle-to-vehicle network. With the observation of global network state at each scheduling slot, the roadside unit (RSU) allocates the frequency bands and schedules packet transmissions for all vehicle user equipment-pairs (VUE-pairs). We model the stochastic decision-making procedure as a discrete-time single-agent Markov decision process (MDP). The technical challenges in solving the optimal control policy originate from high spatial mobility and temporally varying traffic information arrivals of the VUE-pairs. To make the problem solving tractable, we first decompose the original MDP into a series of per-VUE-pair MDPs. Then we propose a proactive algorithm based on long short-term memory and deep reinforcement learning techniques to address the partial observability and the curse of high dimensionality in local network state space faced by each VUE-pair. With the proposed algorithm, the RSU makes the optimal frequency band allocation and packet scheduling decision at each scheduling slot in a decentralized way in accordance with the partial observations of the global network state at the VUE-pairs. Numerical experiments validate the theoretical analysis and demonstrate the significant performance improvements from the proposed algorithm.

Authors:
Chen Xianfu, Wu Celimuge, Chen Tao, Zhang Honggang, Liu Zhi, Zhang Yan, Bennis Mehdi

Publication type:
A1 Journal article – refereed

Place of publication:

Keywords:
deep reinforcement learning, long short-term memory, Markov decision process, multi-user resource scheduling, Q-function decomposition, Vehicular communications

Published:

Full citation:
X. Chen et al., “Age of Information Aware Radio Resource Management in Vehicular Networks: A Proactive Deep Reinforcement Learning Perspective,” in IEEE Transactions on Wireless Communications, vol. 19, no. 4, pp. 2268-2281, April 2020. doi: 10.1109/TWC.2019.2963667

DOI:
https://doi.org/10.1109/TWC.2019.2963667

Read the publication here:
http://urn.fi/urn:nbn:fi-fe202002034244