PROJECT TITLE :

MAGNETIC: Multi-Agent Machine Learning-Based Approach for Energy Efficient Dynamic Consolidation in Data Centers

ABSTRACT:

Two of the most significant challenges for effective resource management in large-scale cloud infrastructures are increasing the energy efficiency of data centers while simultaneously guaranteeing Quality of Service (QoS), and identifying performance variability in servers caused by either hardware or software failures. Both of these challenges must be overcome. Previous research in the field of dynamic Virtual Machine (VM) consolidation has focused the majority of its attention on finding solutions to the energy problem; however, these studies have not been successful in proposing comprehensive, scalable, and low-overhead methods that simultaneously address energy efficiency and performance variability. In addition to this, they typically make the assumption of overly simplistic power models and fail to take into account all of the delays and power costs that are associated with the transition from one power mode on the host to another. These assumptions are no longer valid in modern servers because they execute workloads that are heterogeneous, and as a result, the results are either unrealistic or inefficient. In this paper, we propose a centralized-distributed low-overhead failure-aware dynamic VM consolidation strategy for large-scale data centers, with the goal of reducing the amount of energy those centers use. Our method uses a centralized heuristic to select the most appropriate power mode and frequency for each host during runtime by employing a distributed multi-agent Machine Learning (ML) based strategy. This strategy then migrates the VMs in accordance with the selected power mode and frequency. Our Multi-agent machine learning-based approach for Energy efficient dynamic Consolidation (MAGNETIC) is implemented in a modified version of the CloudSim simulator. This approach takes into account the energy and delay overheads associated with host power mode transition and VM migration, and it is evaluated using power traces collected from various workloads running on real servers as well as resource utilization logs from cloud data center infrastructures. The findings demonstrate that our approach is superior to other works in the state-of-the-art (SoA) in that it lowers the amount of energy used in data centers by up to 15 percent while maintaining the same level of quality of service and cutting the number of virtual machine migrations and host power mode transitions by up to 86 and 90 percent, respectively. In addition to this, it demonstrates superior scalability in comparison to any other approach, requiring an execution time overhead of less than 0.7% for a data center that contains 1,500 VMs. Finally, our solution has the ability to detect host performance variability caused by failures, automatically migrate virtual machines (VMs) away from failing hosts, and relieve those hosts of their workload.