Enhanced Online Q-Learning Scheme for Resource Allocation with Maximum Utility and Fairness in Edge-IoT Networks

Internet of Things (IoT) is experiencing an explosion in the data traffic due to the increase in the number of heterogeneous applications. The existing cloud computing models will not be capable to support the IoT applications that are delay-sensitive and using high bandwidth. The Edge-IoT systems represented by shared edge clouds support a wide range of IoT applications. Edge clouds provide resources closer to the IoT devices to tackle the delay sensitivity and bandwidth issues. However, the allocation of these resources with guaranteed application's utility in the context of Edge-IoT with multiple heterogeneous IoT applications, various resource demands, and limited resource availability is challenging. In this paper, we propose a novel enhanced online Q-learning scheme to allocate resources from edge clouds to IoT applications to maximize their utility and maintain allocation fairness them. To achieve these goals, the developed scheme is characterized by a high convergence rate and requires less computation through approximation of Q-value. It is implemented using two settings: centralized through a dedicated controller at the edge cloud and decentralized where each edge server plays the role of a learning agent. Extensive numerical results demonstrate the capability of the proposed scheme in improving applications' utilities and allocation fairness.