ImpalaE: Towards an optimal policy for efficient resource management at the edge

Main Article Content

Tania Lorido-Botran
https://orcid.org/0000-0002-9132-4435
Muhammad Khurram Bhatti
https://orcid.org/0000-0002-1974-8268

Abstract

Edge computing is an extension of cloud computing where physical servers are deployed closer to the users in order to reduce latency. Edge data centers face the challenge of serving a continuously increasing number of applications with a reduced capacity compared to traditional data center. This paper introduces ImpalaE, an agent based on Deep Reinforcement Learning that aims at optimizing the resource usage in edge data centers. First, it proposes modeling the problem as a Markov Decision Process, with two optimization objectives: reducing the number of physical servers used and maximize number of applications placed in the data center. Second, it introduces an agent based on Proximal Policy Optimization, for finding the optimal consolidation policy, and an asynchronous architecture with multiple workers-shared learner that enables for faster convergence, even with reduced amount of data. We show the potential in a simulated edge data center scenario with different VM sizes based on Microsoft Azure real traces, considering CPU, memory, disk and network requirements. Experiments show that ImpalaE effectively increases the number of VMs that can be placed per episode and that it quickly converges to an optimal policy.

Downloads

Download data is not yet available.

Article Details

How to Cite
Lorido-Botran, T. and Bhatti, M.K., 2022. ImpalaE: Towards an optimal policy for efficient resource management at the edge. Journal of Edge Computing [Online], 1(1), pp.43–54. Available from: https://doi.org/10.55056/jec.572 [Accessed 24 May 2024].
Section
Articles

How to Cite

Lorido-Botran, T. and Bhatti, M.K., 2022. ImpalaE: Towards an optimal policy for efficient resource management at the edge. Journal of Edge Computing [Online], 1(1), pp.43–54. Available from: https://doi.org/10.55056/jec.572 [Accessed 24 May 2024].
Received 2022-08-15
Accepted 2022-11-20
Published 2022-11-21

References

Choo, K.K.R., Lu, R., Chen, L. and Yi, X., 2018. A foggy research future: Advances and future opportunities in fog computing research. DOI: https://doi.org/10.1016/j.future.2017.09.014

Du, B., Wu, C. and Huang, Z., 2019. Learning resource allocation and pricing for cloud profit maximization. Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp.7570–7577. DOI: https://doi.org/10.1609/aaai.v33i01.33017570

Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I. and Others, 2018. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures. International Conference on Machine Learning. PMLR, pp.1407–1416.

Fan, Q. and Ansari, N., 2017. Cost aware cloudlet placement for big data processing at the edge. 2017 IEEE International Conference on Communications (ICC). IEEE, pp.1–6. DOI: https://doi.org/10.1109/ICC.2017.7996722

Hadary, O., Marshall, L., Menache, I., Pan, A., Greeff, E.E., Dion, D., Dorminey, S., Joshi, S., Chen, Y., Russinovich, M. and Others, 2020. Protean: VM Allocation Service at Scale. 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). pp.845–861.

Khan, W.Z., Ahmed, E., Hakak, S., Yaqoob, I. and Ahmed, A., 2019. Edge computing: A survey. Future Generation Computer Systems, 97, pp.219–235. DOI: https://doi.org/10.1016/j.future.2019.02.050

Li, Y. and Wang, S., 2018. An energy-aware edge server placement algorithm in mobile edge computing. 2018 IEEE International Conference on Edge Computing (EDGE). IEEE, pp.66–73. DOI: https://doi.org/10.1109/EDGE.2018.00016

Liu, J., Mao, Y., Zhang, J. and Letaief, K.B., 2016. Delay-optimal computation task scheduling for mobile-edge computing systems. 2016 IEEE International Symposium on Information Theory (ISIT). IEEE, pp.1451–1455. DOI: https://doi.org/10.1109/ISIT.2016.7541539

Lorido-Botran, T. and Bhatti, M.K., 2021. ImpalaE: Towards an optimal policy for efficient resource management at the edge. CEUR Workshop Proceedings, 2850, pp.71–82. Available from: http://ceur-ws.org/Vol-2850/paper5.pdf.

Market Insights Reports, 2021-02. Global Edge Computing Market Size, Status And Forecast 2020-2026.

Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D. and Kavukcuoglu, K., 2016. Asynchronous methods for deep reinforcement learning. International conference on machine learning. PMLR, pp.1928–1937.

Mondal, S., Das, G. and Wong, E., 2018. CCOMPASSION: A hybrid cloudlet placement framework over passive optical access networks. IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, pp.216–224. DOI: https://doi.org/10.1109/INFOCOM.2018.8485846

Puthal, D., Obaidat, M.S., Nanda, P., Prasad, M., Mohanty, S.P. and Zomaya, A.Y., 2018. Secure and sustainable load balancing of edge data centers in fog computing. IEEE Communications Magazine, 56(5), pp.60–65. DOI: https://doi.org/10.1109/MCOM.2018.1700795

Qiu, X., Liu, L., Chen, W., Hong, Z. and Zheng, Z., 2019. Online deep reinforcement learning for computation offloading in blockchain-empowered mobile edge computing. IEEE Transactions on Vehicular Technology, 68(8), pp.8050–8062. DOI: https://doi.org/10.1109/TVT.2019.2924015

Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O., 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.

Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M. and Others, 2016. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), pp.484–489. DOI: https://doi.org/10.1038/nature16961

Tao, Z., Xia, Q., Hao, Z., Li, C., Ma, L., Yi, S. and Li, Q., 2019. A survey of virtual machine management in edge computing. Proceedings of the IEEE, 107(8), pp.1482–1499. DOI: https://doi.org/10.1109/JPROC.2019.2927919

Trace, 2021-02. Azure Public Dataset. Available from: https://github.com/Azure/AzurePublicDataset.

Wang, Z., Gwon, C., Oates, T. and Iezzi, A., 2017. Automated cloud provisioning on aws using deep reinforcement learning. arXiv preprint arXiv:1709.04305.

Zhang, S., Wu, T., Pan, M., Zhang, C. and Yu, Y., 2020. A-SARSA: A Predictive Container Auto-Scaling Algorithm Based on Reinforcement Learning. 2020 IEEE International Conference on Web Services (ICWS). IEEE, pp.489–497. DOI: https://doi.org/10.1109/ICWS49710.2020.00072

Zhu, T., Shi, T., Li, J., Cai, Z. and Zhou, X., 2018. Task scheduling in deadline-aware mobile edge computing systems. Ieee internet of things journal, 6(3), pp.4854–4866. DOI: https://doi.org/10.1109/JIOT.2018.2874954

Similar Articles

You may also start an advanced similarity search for this article.