Joint Service Migration and Resource Allocation in Edge IoT System Based on Deep Reinforcement Learning

Multi-access Edge Computing (MEC) provides services for resource-sensitive and delay-sensitive Internet of Things (IoT) applications by extending the capabilities of cloud computing to the edge of the networks. However, the high mobility of IoT devices (e.g., vehicles) and the limited resources of edge servers (ESs) affect the service continuity and access latency. Service migration and reasonable resource (re-)allocation consequently become needed to ensure quality of service (QoS). However, service migration results in additional latency. In addition, different mobile IoT users have different resource requirements and different resource allocation policies of target edge servers also determine whether service migration is necessary. Subsequently, how to jointly optimize service migration and resource allocation is a challenge that needs to be carefully addressed. To this end, this paper investigates the joint optimization problem of service migration and resource allocation (SMRA) in MEC environments to minimize the access delay of IoT users. It proposes a joint SMRA algorithm based on deep reinforcement learning (DRL), which takes into account the mobility of IoT users and decides whether to migrate services, where to migrate, and how to allocate resources through the long short time memory (LSTM) algorithm and the parameterized deep Q-network (PDQN) algorithm. Moreover, the PDQN algorithm effectively solves the discrete-continuous hybrid action space challenge in the SMRA problem. Finally, we conduct evaluation using a real-world dataset of Beijing cab trajectories to verify the effectiveness and superiority of our proposed SMRA solution.