Ultra-reliable millimeter-wave communications using an artificial intelligence-powered reflector

In this paper, a novel framework for guaranteeing ultra-reliable millimeter-wave (mmW) communications using a smart, artificial intelligence (AI)-powered mmW reflector is proposed. The use of an AI-powered reflector allows changing the propagation direction of mmW signals and, thus, improving coverage particularly for non-line-of-sight (LoS) areas. However, due to the possibility of stochastic blockage over mmW links, designing an intelligent phase shift-control policy for the mmW reflector to guarantee ultra-reliable mmW communications becomes very challenging. In this regard, first, based on the framework of risk-sensitive reinforcement learning, a parametric risk-sensitive episodic return is proposed to maximize the expected bit rate while mitigating the risk of non-LoS mmW link in the presence of future stochastic blockage over the mmW links. Then, a closed-form approximation for the gradient of the risk- sensitive episodic return is analytically derived. To \emph{directly} find the optimal policy for the proposed phase-shift controller, a parametric functional-form policy is implemented using a deep recurrent neural network (RNN). Then, based on the derived closed-form gradient of risk-sensitive episodic return, the deep RNN-based parametric functional-form policy is trained. The efficiency of the proposed AI-powered reflector is evaluated in an office environment. Simulation results show that the root-mean- square errors between the optimal and approximate phase shift-control policies of the proposed deep RNN is 1.35% in the worst case. Moreover, on average, the mean value and variance of the achievable rates resulting from the deep RNN-based policy are only 1% and 2% less than the optimal solution for different unknown mobile users’ trajectories, respectively.