Deep Contextual Bandits for Fast Neighbor-Aided Initial Access in mmWave Cell-Free Networks

Access points (APs) in millimeter-wave (mmWave) user-centric (UC) networks will have sleep mode functionality. Initial access (IA) is a challenging problem in UC networks due to the coherent serving of the users. In this letter, a novel deep contextual bandit (DCB) learning-based instantaneous beam selection method is proposed as a complementary tool to current IA schemes. In the proposed approach, the DCB model at an AP uses beam selection information from the neighboring active APs as the input to solve the beam search problem of the host AP. The proposed fast beam selection scheme enables APs to be in energy-saving modes while maintaining the ability to serve users without any delay when restored. Simulations are carried out with realistic channel models generated using a ray-tracing tool. The results show that the proposed system with the 5G IA scheme can respond to dynamic throughput demands with negligible latency compared to the 5G IA scheme without the proposed scheme.