Using artificial intelligence tools for automating the assessment of future computer science teachers' work

Oleksandr M. Spazhev; Oksana V. Klochko

doi:10.55056/ed.801

Authors

Oleksandr M. Spazhev Vinnytsia Mykhailo Kotsiubynskyi State Pedagogical University https://orcid.org/0009-0004-5456-1783
Oksana V. Klochko Vinnytsia Mykhailo Kotsiubynskyi State Pedagogical University https://orcid.org/0000-0002-6505-9455

DOI:

https://doi.org/10.55056/ed.801

Keywords:

automated work checking, modified tasks, programming, GitHub Copilot, future computer science teachers, educational process, testing

Abstract

This paper examines the problem of automated testing of modified programming tasks for future computer science teachers. It is recommended that GitHub Copilot be used to generate tests based on the code. This approach makes it possible to solve the following tasks: reducing the time and effort required for manual checking of programming tasks completed by students; promoting better assimilation of the material of the relevant disciplines by students; promoting the development and improvement of students' skills in algorithmisation and programming; compliance by students with academic integrity; effective use of GitHub Copilot to generate baseline tests to test modified programming assignments completed by students; ensuring the flexibility and scalability of the approach to the development of various training courses in programming; development of students' software product testing skills. In the process of research, we found the following disadvantages of using the GitHub Copilot system for generating basic tests: GitHub Copilot does not always generate perfect code or tests; for complex tasks, GitHub Copilot may require additional correction of the generated code. Therefore, it is important to check and refine the generated tests carefully, if necessary. Therefore, at the moment, we recommend using GitHub Copilot as a template generator for writing tests. The proposed approach is a promising solution for facilitating the verification of modified programming tasks and increasing the effectiveness of the education process of future informatics teachers. The conducted research opens up new prospects for effective improvement of the verification of modified tasks performed by students and the generation of tests for verification. In particular, the integration of the proposed system based on GitHub Copilot with learning management systems (LMS) and automated task verification systems. Another area of research could be exploring the possibilities of using other tools for generating tests instead of GitHub Copilot or combining them in order to obtain better results.

Downloads

Download data is not yet available.

Abstract views: 309 / PDF views: 116

References

Albusac, J., Castro-Schez, J., González, C. and Vallejo, D., 2018. Model for detecting academic failure automatically and early on. INTED2018 Proceedings. IATED, 12th International Technology, Education and Development Conference, pp.8388–8397. Available from: https://doi.org/10.21125/inted.2018.2033. DOI: https://doi.org/10.21125/inted.2018.2033

algotester.com, 2025. Algotester. Available from: https://algotester.com/en.

Antonov, Y.S., 2022. Avtomatyzatsii perevirky prohramnoho kodu na C# studentamy ta vykladachamy pid chas vyvchennia dystsyplin prohramuvannia/tekhnolohii prohramuvannia. Available from: https://tinyurl.com/2btsytpt.

Burov, O.Y., Lytvynova, S.H., Semerikov, S.O. and Yechkalo, Y.V., 2023. ICT for disaster-resilient education and training. In: O.Y. Burov, S.H. Lytvynova, S.O. Semerikov and Y.V. Yechkalo, eds. Proceedings of the VII International Workshop on Professional Retraining and Life-Long Learning using ICT: Person-oriented Approach (3L-Person 2022), Virtual Event, Kryvyi Rih, Ukraine, October 25, 2022. CEUR-WS.org, CEUR Workshop Proceedings, vol. 3482, pp.1–25. Available from: https://ceur-ws.org/Vol-3482/paper000.pdf. DOI: https://doi.org/10.31812/123456789/7825

CodeChef, 2025. Learn and Practice Coding with Problems. Available from: https://www.codechef.com/.

Divasón, J., Martínez-de-Pisón, F.J., Romero, A. and Sáenz-de-Cabezón, E., 2023. Artificial Intelligence Models for Assessing the Evaluation Process of Complex Student Projects. IEEE Trans. Learn. Technol., 16(5), p.694–707. Available from: https://doi.org/10.1109/TLT.2023.3246589. DOI: https://doi.org/10.1109/TLT.2023.3246589

Dunder, N., Lundborg, S., Wong, J. and Viberg, O., 2024. Kattis vs ChatGPT: Assessment and Evaluation of Programming Tasks in the Age of Artificial Intelligence. Proceedings of the 14th Learning Analytics and Knowledge Conference. New York, NY, USA: Association for Computing Machinery, LAK ’24, p.821–827. Available from: https://doi.org/10.1145/3636555.3636882. DOI: https://doi.org/10.1145/3636555.3636882

Eolymp, 2025. Available from: https://eolymp.com/en.

Fedorets, V.M., Klochko, O.V., Tverdokhlib, I.A. and Sharyhin, O.A., 2024. Cognitive aspects of interaction in the “Human — Artificial Intelligence” system. Journal of Physics: Conference Series, 2871(1), p.012023. Available from: https://doi.org/10.1088/1742-6596/2871/1/012023. DOI: https://doi.org/10.1088/1742-6596/2871/1/012023

GitHub, Inc., 2025. GitHub Copilot · Your AI pair programmer. Available from: https://github.com/features/copilot.

HackerRank, 2025. Online Coding Tests and Technical Interviews. Available from: https://www.hackerrank.com/.

Kharchenko, V., Didkowsky, R. and Serdiuk, O., 2021. Development the server-side part of the system of checking software code for the presence of plagiarism. Cherkasy University Bulletin: Applied Mathematics. Informatics, (1), pp.68–69. Available from: https://doi.org/10.31651/2076-5886-2021-1-68-84. DOI: https://doi.org/10.31651/2076-5886-2021-1-68-84

Kiv, A.E., Semerikov, S.O., Striuk, A.M., Osadchyi, V.V., Vakaliuk, T.A., Nechypurenko, P.P., Bondarenko, O.V., Mintii, I.S. and Malchenko, S.L., 2023. XV International Conference on Mathematics, Science and Technology Education. Journal of Physics: Conference Series, 2611(1), p.011001. Available from: https://doi.org/10.1088/1742-6596/2611/1/011001. DOI: https://doi.org/10.1088/1742-6596/2611/1/011001

Klochko, O. and Fedorets, V., 2019. An empirical comparison of machine learning clustering methods in the study of Internet addiction among students majoring in Computer Sciences. In: A.E. Kiv, S.O. Semerikov, V.N. Soloviev and A.M. Striuk, eds. Proceedings of the 2nd Student Workshop on Computer Science & Software Engineering (CS&SE@SW 2019), Kryvyi Rih, Ukraine, November 29, 2019. CEUR-WS.org, CEUR Workshop Proceedings, vol. 2546, pp.58–75. Available from: https://ceur-ws.org/Vol-2546/paper03.pdf.

Klochko, O., Fedorets, V., Klochko, V. and Kormer, M., 2022. The Use of Ensemble Classification and Clustering Methods of Machine Learning in the Study of Internet Addiction of Students. Proceedings of the 1st Symposium on Advances in Educational Technology - Volume 1: AET. SciTePress, pp.241–260. Available from: https://doi.org/10.5220/0010923500003364. DOI: https://doi.org/10.5220/0010923500003364

Klochko, O., Fedorets, V., Tkachenko, S. and Maliar, O., 2020. The Use of Digital Technologies for Flipped Learning Implementation. In: O. Sokolov, G. Zholtkevych, V. Yakovyna, Y. Tarasich, V. Kharchenko, V. Kobets, O. Burov, S. Semerikov and H. Kravtsov, eds. Proceedings of the 16th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer. Volume II: Workshops, Kharkiv, Ukraine, October 06-10, 2020. CEUR-WS.org, CEUR Workshop Proceedings, vol. 2732, pp.1233–1248. Available from: https://ceur-ws.org/Vol-2732/20201233.pdf.

Klochko, O. and Sharyhin, O., 2023. Professional training experience of computer science students in European Universities. Ukrainian polonistics, 21(1), p.694–707. Available from: https://doi.org/10.35433/2220-4555.21.2023.ped-3. DOI: https://doi.org/10.35433/2220-4555.21.2023.ped-3

Klochko, O.V., 2024. Development of critical thinking of future teachers of computerscience and mathematics using artificial intelligence tools [Rozvytok krytychnoho myslennia maibutnikh vchyteliv informatyky ta matematyky z vykorystanniam zasobiv shtuchnoho intelektu]. Modern Information Technologies and Innovation Methodologies of Education in Professional Training Methodology Theory Experience Problems, 1(72), p.14–26. Available from: https://doi.org/10.31652/2412-1142-2024-72-14-26. DOI: https://doi.org/10.31652/2412-1142-2024-72-14-26

Klochko, O.V., Fedorets, V.M., Klochko, V.I. and Klochko, K.A., 2023. Anthropologically oriented strategies of interaction in the Human-Computer system. Journal of Physics: Conference Series, 2611(1), p.012018. Available from: https://doi.org/10.1088/1742-6596/2611/1/012018. DOI: https://doi.org/10.1088/1742-6596/2611/1/012018

Knuth, D.E., 1998. The Art of Computer Programming, vol. 3. Sorting and Searching. 2nd ed. Addison Wesley Longman. Available from: https://seriouscomputerist.atariverse.com/media/pdf/book/Art%20of%20Computer%20Programming%20-%20Volume%203%20(Sorting%20&%20Searching).pdf.

Moradi Dakhel, A., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M.C. and Jiang, Z.M.J., 2023. GitHub Copilot AI pair programmer: Asset or Liability? Journal of Systems and Software, 203, p.111734. Available from: https://doi.org/10.1016/j.jss.2023.111734. DOI: https://doi.org/10.1016/j.jss.2023.111734

Osadchyi, V.V., Pinchuk, O.P. and Vakaliuk, T.A., 2023. From the digital transformation strategy to the productive integration of technologies in education and training: Report 2023. In: T.A. Vakaliuk, V.V. Osadchyi and O.P. Pinchuk, eds. Proceedings of the 2nd Workshop on Digital Transformation of Education (DigiTransfEd 2023) co-located with 18th International Conference on ICT in Education, Research and Industrial Applications (ICTERI 2023), Ivano-Frankivsk, Ukraine, September 18-22, 2023. CEUR-WS.org, CEUR Workshop Proceedings, vol. 3553, pp.1–8. Available from: https://ceur-ws.org/Vol-3553/paper00.pdf.

Paiva, J.C., Leal, J.P. and Figueira, A., 2022. Automated Assessment in Computer Science Education: A State-of-the-Art Review. ACM Trans. Comput. Educ., 22(3), pp.1–40. Available from: https://doi.org/10.1145/3513140. DOI: https://doi.org/10.1145/3513140

Semerikov, S.O., Vakaliuk, T.A., Mintii, I.S., Hamaniuk, V.A., Soloviev, V.N., Bondarenko, O.V., Nechypurenko, P.P., Shokaliuk, S.V., Moiseienko, N.V. and Shepiliev, D.S., 2022. Immersive E-Learning Resources: Design Methods. Digital humanities workshop. New York, NY, USA: Association for Computing Machinery, DHW 2021, p.37–47. Available from: https://doi.org/10.1145/3526242.3526264. DOI: https://doi.org/10.1145/3526242.3526264

Topcoder, 2024. Home. Available from: https://www.topcoder.com/.

Turnitin LLC, 2022. Emerging trends in academic integrity: A free guide from Turnitin. Available from: https://uatpia.elearn.net.au/pluginfile.php/71044/mod_resource/content/2/TII_AI_EmergingTrends_eBook_UK_0222.pdf.

Tymokhina, V., Yurchenko, V. and Nalyvaiko, O., 2024. Akademichna dobrochesnist vs shtuchnyi intelekt: etychna dyskusiia v osvitnomu prostori. In: Y.D. Boichuk et al., eds. Tsyfrova transformatsiia osvity ta nauky : materialy II Vseukrainskykh naukovo-praktychnoi konferentsii, 14-15 berez. 2024. Kharkiv, pp.202–207. Available from: https://dspace.hnpu.edu.ua/server/api/core/bitstreams/ef7f9271-4458-4426-97bc-fe308e498325/content#page=203.

USACO, 2024. Available from: https://usaco.org/.

Vakaliuk, T.A., 2019. Theoretical and methodical principles of the cloud-based learning environment design and use in the training of bachelors in computer science. The dissertation for a Doctor of Pedagogical Sciences degree, specialty 13.00.10 – “Information and Communication Technologies in Education” (011 – Educational, pedagogical science). Zhytomyr Ivan Franko State University, Zhytomyr; Institute of Information Technologies and Learning Tools of NAPS of Ukraine, Kyiv. Available from: https://lib.iitta.gov.ua/id/eprint/715709/1/dis_15.pdf.

Wermelinger, M., 2023. Using GitHub Copilot to Solve Simple Programming Problems. Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. New York, NY, USA: Association for Computing Machinery, SIGCSE 2023, p.172–178. Available from: https://doi.org/10.1145/3545945.3569830. DOI: https://doi.org/10.1145/3545945.3569830

Yetistiren, B., Ozsoy, I. and Tuzun, E., 2022. Assessing the quality of GitHub copilot’s code generation. Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering. New York, NY, USA: Association for Computing Machinery, PROMISE 2022, p.62–71. Available from: https://doi.org/10.1145/3558489.3559072. DOI: https://doi.org/10.1145/3558489.3559072