ImpalaE: Towards an optimal policy for efficient resource management at the edge

Main Article Content

Tania Lorido-Botran
Muhammad Khurram Bhatti


Edge computing is an extension of cloud computing where physical servers are deployed closer to the users in order to reduce latency. Edge data centers face the challenge of serving a continuously increasing number of applications with a reduced capacity compared to traditional data center. This paper introduces ImpalaE, an agent based on Deep Reinforcement Learning that aims at optimizing the resource usage in edge data centers. First, it proposes modeling the problem as a Markov Decision Process, with two optimization objectives: reducing the number of physical servers used and maximize number of applications placed in the data center. Second, it introduces an agent based on Proximal Policy Optimization, for finding the optimal consolidation policy, and an asynchronous architecture with multiple workers-shared learner that enables for faster convergence, even with reduced amount of data. We show the potential in a simulated edge data center scenario with different VM sizes based on Microsoft Azure real traces, considering CPU, memory, disk and network requirements. Experiments show that ImpalaE effectively increases the number of VMs that can be placed per episode and that it quickly converges to an optimal policy.


Download data is not yet available.

Article Details

How to Cite
Lorido-Botran, T. and Bhatti, M.K., 2022. ImpalaE: Towards an optimal policy for efficient resource management at the edge. Journal of Edge Computing [Online], 1(1), pp.43–54. Available from: [Accessed 24 May 2024].

How to Cite

Lorido-Botran, T. and Bhatti, M.K., 2022. ImpalaE: Towards an optimal policy for efficient resource management at the edge. Journal of Edge Computing [Online], 1(1), pp.43–54. Available from: [Accessed 24 May 2024].
Received 2022-08-15
Accepted 2022-11-20
Published 2022-11-21


Choo, K.K.R., Lu, R., Chen, L. and Yi, X., 2018. A foggy research future: Advances and future opportunities in fog computing research. DOI:

Du, B., Wu, C. and Huang, Z., 2019. Learning resource allocation and pricing for cloud profit maximization. Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp.7570–7577. DOI:

Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I. and Others, 2018. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures. International Conference on Machine Learning. PMLR, pp.1407–1416.

Fan, Q. and Ansari, N., 2017. Cost aware cloudlet placement for big data processing at the edge. 2017 IEEE International Conference on Communications (ICC). IEEE, pp.1–6. DOI:

Hadary, O., Marshall, L., Menache, I., Pan, A., Greeff, E.E., Dion, D., Dorminey, S., Joshi, S., Chen, Y., Russinovich, M. and Others, 2020. Protean: VM Allocation Service at Scale. 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). pp.845–861.

Khan, W.Z., Ahmed, E., Hakak, S., Yaqoob, I. and Ahmed, A., 2019. Edge computing: A survey. Future Generation Computer Systems, 97, pp.219–235. DOI:

Li, Y. and Wang, S., 2018. An energy-aware edge server placement algorithm in mobile edge computing. 2018 IEEE International Conference on Edge Computing (EDGE). IEEE, pp.66–73. DOI:

Liu, J., Mao, Y., Zhang, J. and Letaief, K.B., 2016. Delay-optimal computation task scheduling for mobile-edge computing systems. 2016 IEEE International Symposium on Information Theory (ISIT). IEEE, pp.1451–1455. DOI:

Lorido-Botran, T. and Bhatti, M.K., 2021. ImpalaE: Towards an optimal policy for efficient resource management at the edge. CEUR Workshop Proceedings, 2850, pp.71–82. Available from:

Market Insights Reports, 2021-02. Global Edge Computing Market Size, Status And Forecast 2020-2026.

Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D. and Kavukcuoglu, K., 2016. Asynchronous methods for deep reinforcement learning. International conference on machine learning. PMLR, pp.1928–1937.

Mondal, S., Das, G. and Wong, E., 2018. CCOMPASSION: A hybrid cloudlet placement framework over passive optical access networks. IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, pp.216–224. DOI:

Puthal, D., Obaidat, M.S., Nanda, P., Prasad, M., Mohanty, S.P. and Zomaya, A.Y., 2018. Secure and sustainable load balancing of edge data centers in fog computing. IEEE Communications Magazine, 56(5), pp.60–65. DOI:

Qiu, X., Liu, L., Chen, W., Hong, Z. and Zheng, Z., 2019. Online deep reinforcement learning for computation offloading in blockchain-empowered mobile edge computing. IEEE Transactions on Vehicular Technology, 68(8), pp.8050–8062. DOI:

Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O., 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.

Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M. and Others, 2016. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), pp.484–489. DOI:

Tao, Z., Xia, Q., Hao, Z., Li, C., Ma, L., Yi, S. and Li, Q., 2019. A survey of virtual machine management in edge computing. Proceedings of the IEEE, 107(8), pp.1482–1499. DOI:

Trace, 2021-02. Azure Public Dataset. Available from:

Wang, Z., Gwon, C., Oates, T. and Iezzi, A., 2017. Automated cloud provisioning on aws using deep reinforcement learning. arXiv preprint arXiv:1709.04305.

Zhang, S., Wu, T., Pan, M., Zhang, C. and Yu, Y., 2020. A-SARSA: A Predictive Container Auto-Scaling Algorithm Based on Reinforcement Learning. 2020 IEEE International Conference on Web Services (ICWS). IEEE, pp.489–497. DOI:

Zhu, T., Shi, T., Li, J., Cai, Z. and Zhou, X., 2018. Task scheduling in deadline-aware mobile edge computing systems. Ieee internet of things journal, 6(3), pp.4854–4866. DOI:

Similar Articles

You may also start an advanced similarity search for this article.