Reinforcement Learning-Based Load Balancing Satellite Handover Using NS-3

Abstract

The Fifth-Generation of Mobile Communications (5G) is intended to meet users' growing needs for high-quality services at any time and from any location. The unique features of Low Earth Orbit (LEO) satellites in terms of higher coverage, reliability, and availability, can help expand the reach of 5G and beyond technologies to support those needs. However, because of their high speeds, a single LEO satellite is unable to provide continuous service to multiple User Equipments (UEs) spread over a large (potentially worldwide) area, resulting in the need for LEO satellite constellations with a high number of satellites and a consequent high amount of satellite handovers (HOs). Moreover, UEs can only acquire partial information about the satellite system and compete for the limited available communication resources of the satellites, requiring the implementation of a decentralized satellite HO strategy to avoid network congestion. In this paper, we propose a decentralized Load Balancing Satellite HO (LBSH) strategy based on multi-agent reinforcement Q-learning, implemented within the software Network Simulator 3 (NS-3). LBSH aims to reduce the total number of HOs and the blocking rate while balancing the load distribution among satellites. Our results show that the proposed LBSH method outperforms the state-of-the-art methods in terms of a 95% drop in the average number of HOs per user and an 84% reduction in blocking rate.