ImpalaE: Towards an optimal policy for efficient resource management at the edge
Article Sidebar
Main Article Content
Abstract
Edge computing is an extension of cloud computing where physical servers are deployed closer to the users in order to reduce latency. Edge data centers face the challenge of serving a continuously increasing number of applications with a reduced capacity compared to traditional data center. This paper introduces ImpalaE, an agent based on Deep Reinforcement Learning that aims at optimizing the resource usage in edge data centers. First, it proposes modeling the problem as a Markov Decision Process, with two optimization objectives: reducing the number of physical servers used and maximize number of applications placed in the data center. Second, it introduces an agent based on Proximal Policy Optimization, for finding the optimal consolidation policy, and an asynchronous architecture with multiple workers-shared learner that enables for faster convergence, even with reduced amount of data. We show the potential in a simulated edge data center scenario with different VM sizes based on Microsoft Azure real traces, considering CPU, memory, disk and network requirements. Experiments show that ImpalaE effectively increases the number of VMs that can be placed per episode and that it quickly converges to an optimal policy.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
Accepted 2022-11-20
Published 2022-11-21
References
Choo, K.K.R., Lu, R., Chen, L. and Yi, X., 2018. A foggy research future: Advances and future opportunities in fog computing research. DOI: https://doi.org/10.1016/j.future.2017.09.014
Du, B., Wu, C. and Huang, Z., 2019. Learning resource allocation and pricing for cloud profit maximization. Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp.7570–7577. DOI: https://doi.org/10.1609/aaai.v33i01.33017570
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I. and Others, 2018. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures. International Conference on Machine Learning. PMLR, pp.1407–1416.
Fan, Q. and Ansari, N., 2017. Cost aware cloudlet placement for big data processing at the edge. 2017 IEEE International Conference on Communications (ICC). IEEE, pp.1–6. DOI: https://doi.org/10.1109/ICC.2017.7996722
Hadary, O., Marshall, L., Menache, I., Pan, A., Greeff, E.E., Dion, D., Dorminey, S., Joshi, S., Chen, Y., Russinovich, M. and Others, 2020. Protean: VM Allocation Service at Scale. 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). pp.845–861.
Khan, W.Z., Ahmed, E., Hakak, S., Yaqoob, I. and Ahmed, A., 2019. Edge computing: A survey. Future Generation Computer Systems, 97, pp.219–235. DOI: https://doi.org/10.1016/j.future.2019.02.050
Li, Y. and Wang, S., 2018. An energy-aware edge server placement algorithm in mobile edge computing. 2018 IEEE International Conference on Edge Computing (EDGE). IEEE, pp.66–73. DOI: https://doi.org/10.1109/EDGE.2018.00016
Liu, J., Mao, Y., Zhang, J. and Letaief, K.B., 2016. Delay-optimal computation task scheduling for mobile-edge computing systems. 2016 IEEE International Symposium on Information Theory (ISIT). IEEE, pp.1451–1455. DOI: https://doi.org/10.1109/ISIT.2016.7541539
Lorido-Botran, T. and Bhatti, M.K., 2021. ImpalaE: Towards an optimal policy for efficient resource management at the edge. CEUR Workshop Proceedings, 2850, pp.71–82. Available from: http://ceur-ws.org/Vol-2850/paper5.pdf.
Market Insights Reports, 2021-02. Global Edge Computing Market Size, Status And Forecast 2020-2026.
Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D. and Kavukcuoglu, K., 2016. Asynchronous methods for deep reinforcement learning. International conference on machine learning. PMLR, pp.1928–1937.
Mondal, S., Das, G. and Wong, E., 2018. CCOMPASSION: A hybrid cloudlet placement framework over passive optical access networks. IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, pp.216–224. DOI: https://doi.org/10.1109/INFOCOM.2018.8485846
Puthal, D., Obaidat, M.S., Nanda, P., Prasad, M., Mohanty, S.P. and Zomaya, A.Y., 2018. Secure and sustainable load balancing of edge data centers in fog computing. IEEE Communications Magazine, 56(5), pp.60–65. DOI: https://doi.org/10.1109/MCOM.2018.1700795
Qiu, X., Liu, L., Chen, W., Hong, Z. and Zheng, Z., 2019. Online deep reinforcement learning for computation offloading in blockchain-empowered mobile edge computing. IEEE Transactions on Vehicular Technology, 68(8), pp.8050–8062. DOI: https://doi.org/10.1109/TVT.2019.2924015
Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O., 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M. and Others, 2016. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), pp.484–489. DOI: https://doi.org/10.1038/nature16961
Tao, Z., Xia, Q., Hao, Z., Li, C., Ma, L., Yi, S. and Li, Q., 2019. A survey of virtual machine management in edge computing. Proceedings of the IEEE, 107(8), pp.1482–1499. DOI: https://doi.org/10.1109/JPROC.2019.2927919
Trace, 2021-02. Azure Public Dataset. Available from: https://github.com/Azure/AzurePublicDataset.
Wang, Z., Gwon, C., Oates, T. and Iezzi, A., 2017. Automated cloud provisioning on aws using deep reinforcement learning. arXiv preprint arXiv:1709.04305.
Zhang, S., Wu, T., Pan, M., Zhang, C. and Yu, Y., 2020. A-SARSA: A Predictive Container Auto-Scaling Algorithm Based on Reinforcement Learning. 2020 IEEE International Conference on Web Services (ICWS). IEEE, pp.489–497. DOI: https://doi.org/10.1109/ICWS49710.2020.00072
Zhu, T., Shi, T., Li, J., Cai, Z. and Zhou, X., 2018. Task scheduling in deadline-aware mobile edge computing systems. Ieee internet of things journal, 6(3), pp.4854–4866. DOI: https://doi.org/10.1109/JIOT.2018.2874954