Methodological evolution in educational technology research: a paradigm-sensitive narrative review of ICT integration studies

Olena Yu. Tarasova; Olena L. Pinska; Halyna I. Alieka; Natalia V. Moiseienko

doi:10.55056/cte.1131

Authors

Olena Yu. Tarasova Kryvyi Rih State Pedagogical University https://orcid.org/0000-0002-6001-5672
Olena L. Pinska Kryvyi Rih State Pedagogical University https://orcid.org/0000-0003-2558-6805
Halyna I. Alieka Kryvyi Rih State Pedagogical University https://orcid.org/0000-0001-6432-2154
Natalia V. Moiseienko Kryvyi Rih State Pedagogical University https://orcid.org/0000-0002-3559-6081

DOI:

https://doi.org/10.55056/cte.1131

Keywords:

educational technology, research methodology, paradigm coherence, design-based research, learning analytics, mixed methods, methodological quality, ICT integration, narrative review

Abstract

This narrative review critically examines the methodological landscape of educational technology and ICT integration research through the lens of evolving educational paradigms. Utilising Scopus AI's analytical capabilities, this paper examines how shifts from behaviourist to constructivist to connectivist paradigms have fundamentally reshaped research methodologies in the field. The review identifies methodological strengths, persistent weaknesses, and critical gaps in current EdTech research, proposing a paradigm-conscious methodological framework that aligns research approaches with underlying philosophical assumptions about learning, technology, and educational management. Drawing on sources spanning 2010-2025, the analysis documents the emergence of design-based research, learning analytics, mixed methods, and AI-enhanced approaches while identifying concerning patterns including theoretical thinness, paradigm-method incoherence, quality limitations, and the systematic overestimation of technology effects. The review contributes a comprehensive mapping of paradigm-methodology relationships, a comparative analysis of contemporary methodological approaches, quality criteria across research traditions, and practical guidelines for paradigm-coherent research practice. Implications for researchers, doctoral training, and the field's methodological development are discussed, with an emphasis on achieving "principled diversity" - the embrace of multiple approaches united by shared commitments to rigour, transparency, and the improvement of technology-enhanced learning.

Downloads

Download data is not yet available.

Abstract views: 112 / PDF views: 44

References

Ajani, O.A., Gamede, B.T. and Govender, S., 2025. Cultural and ethical dimensions of learning management system adoption in rural universities: Exploring data privacy, algorithmic bias, and contextual realities. Multidisciplinary Science Journal, 8(3), p.2026142. Available from: https://doi.org/10.31893/multiscience.2026142. DOI: https://doi.org/10.31893/multiscience.2026142

Alonzo, D. and Teng, S., 2023. Trustworthiness of Teacher Assessment and Decision-Making: Reframing the Consistency and Accuracy Measures. International Journal of Instruction, 16(3), pp.1075–1094. Available from: https://doi.org/10.29333/iji.2023.16357a. DOI: https://doi.org/10.29333/iji.2023.16357a

Anderson, T. and Shattuck, J., 2012. Design-Based Research: A Decade of Progress in Education Research? Educational Researcher, 41(1), pp.16–25. Available from: https://doi.org/10.3102/0013189X11428813. DOI: https://doi.org/10.3102/0013189X11428813

Andrews, D., Leitner, P., Schön, S. and Ebner, M., 2022. Developing a Professional Profile of a Digital Ethics Officer in an Educational Technology Unit in Higher Education. In: P. Zaphiris and A. Ioannou, eds. Learning and Collaboration Technologies. Designing the Learner and Teacher Experience. Cham: Springer International Publishing, Lecture Notes in Computer Science, vol. 13328, pp.157–175. Available from: https://doi.org/10.1007/978-3-031-05657-4_12. DOI: https://doi.org/10.1007/978-3-031-05657-4_12

Baker, R.S. and Inventado, P.S., 2014. Educational Data Mining and Learning Analytics. In: J.A. Larusson and B. White, eds. Learning Analytics: From Research to Practice. New York, NY: Springer New York, pp.61–75. Available from: https://doi.org/10.1007/978-1-4614-3305-7_4. DOI: https://doi.org/10.1007/978-1-4614-3305-7_4

Balaji, C.G., Rajeswari, G., Jain, H., Menaka, S. and Shukla, S.A., 2025. Understanding Data Privacy and Ethical Considerations in Learning Analytics. In: S. Ponnusamy, J. Antari, G. Jeon, M. Assaf and B. Sharma, eds. Revolutionizing Education With Remote Experimentation and Learning Analytics. Hershey, PA: IGI Global Scientific Publishing, chap. 34, pp.579–606. Available from: https://doi.org/10.4018/979-8-3693-8593-7.ch034. DOI: https://doi.org/10.4018/979-8-3693-8593-7.ch034

Baydas, O., Kucuk, S., Yilmaz, R.M., Aydemir, M. and Goktas, Y., 2015. Educational technology research trends from 2002 to 2014. Scientometrics, 105(1), pp.709–725. Available from: https://doi.org/10.1007/s11192-015-1693-4. DOI: https://doi.org/10.1007/s11192-015-1693-4

Bebell, D., O’Dwyer, L.M., Russell, M. and Hoffmann, T., 2010. Concerns, Considerations, and New Ideas for Data Collection and Research in Educational Technology Studies. Journal of Research on Technology in Education, 43(1), pp.29–52. Available from: https://doi.org/10.1080/15391523.2010.10782560. DOI: https://doi.org/10.1080/15391523.2010.10782560

Biddix, J.P. and Bourke, B., 2025. Using formative assessment approaches to enhance rigor and quality in research. Quality and Quantity. Available from: https://doi.org/10.1007/s11135-025-02461-8. DOI: https://doi.org/10.1007/s11135-025-02461-8

Bulfin, S., Henderson, M., Johnson, N.F. and Selwyn, N., 2014. Methodological capacity within the field of “educational technology” research: An initial investigation. British Journal of Educational Technology, 45(3), pp.403–414. Available from: https://doi.org/10.1111/bjet.12145. DOI: https://doi.org/10.1111/bjet.12145

Cela-Ranilla, J., Esteve-Mon, F.M. and Sánchez-Caballé, A., 2025. Design-based research in higher education: A systematic literature review between 2019 and 2023. Australasian Journal of Educational Technology, 41(4), p.67–83. Available from: https://doi.org/10.14742/ajet.10380. DOI: https://doi.org/10.14742/ajet.10380

Cham, H., Lee, H. and Migunov, I., 2024. Quasi-experimental designs for causal inference: an overview. Asia Pacific Education Review, 25(3), pp.611–627. Available from: https://doi.org/10.1007/s12564-024-09981-2. DOI: https://doi.org/10.1007/s12564-024-09981-2

Chang, R.L. and Gray, K., 2013. Ethics of research into learning and teaching with Web 2.0: Reflections on eight case studies. Journal of Computing in Higher Education, 25(3), pp.147–165. Available from: https://doi.org/10.1007/s12528-013-9071-9. DOI: https://doi.org/10.1007/s12528-013-9071-9

Chauke, M., 2024. Navigating Research Paradigms and Theories: A Roadmap for Advancing Research in Higher Education. In: A.A. Mdikana, ed. Enhancing Research for Academicians in Higher Education. Hershey, PA: IGI Global Scientific Publishing, chap. 4, pp.49–64. Available from: https://doi.org/10.4018/979-8-3693-4496-5.ch004. DOI: https://doi.org/10.4018/979-8-3693-4496-5.ch004

Closser, A.H., Sales, A. and Botelho, A.F., 2024. Should We account for classrooms? Analyzing online experimental data with student-level randomization. Educational Technology Research and Development, 72(5), pp.2865–2894. Available from: https://doi.org/10.1007/s11423-023-10325-x. DOI: https://doi.org/10.1007/s11423-023-10325-x

Cook, T.D., 2007. Randomized Experiments in Education: Assessing the Objections to Doing Them. Economics of Innovation and New Technology, 16(5), pp.331–355. Available from: https://doi.org/10.1080/10438590600982335. DOI: https://doi.org/10.1080/10438590600982335

Cypress, B.S., 2017. Rigor or Reliability and Validity in Qualitative Research: Perspectives, Strategies, Reconceptualization, and Recommendations. Dimensions of Critical Care Nursing, 36(4), pp.253–263. Available from: https://doi.org/10.1097/DCC.0000000000000253. DOI: https://doi.org/10.1097/DCC.0000000000000253

Dewey, J., 1976. Experience and Education, The Kappa Delta Pi Lecture Series. New York: Collier Books. Available from: https://dn720204.ca.archive.org/0/items/experienceeducat00dewe_0/experienceeducat00dewe_0.pdf.

Dillenbourg, P. and Jermann, P., 2010. Technology for Classroom Orchestration. In: M.S. Khine and I.M. Saleh, eds. New Science of Learning: Cognition, Computers and Collaboration in Education. New York, NY: Springer New York, pp.525–552. Available from: https://doi.org/10.1007/978-1-4419-5716-0_26. DOI: https://doi.org/10.1007/978-1-4419-5716-0_26

DiStefano, C., Odero, G. and Starrett, A., 2024. An Introduction to Machine Learning for Educational Researchers. In: M.S. Khine, ed. Machine Learning in Educational Sciences: Approaches, Applications and Advances. Singapore: Springer Nature Singapore, pp.11–30. Available from: https://doi.org/10.1007/978-981-99-9379-6_2. DOI: https://doi.org/10.1007/978-981-99-9379-6_2

Downes, S., 2012. Connectivism and Connective Knowledge: Essays on meaning and learning networks. National Research Council Canada. Available from: https://archive.org/details/ConnectiveKnowledge/page/n1/mode/2up.

Ferrari, R., 2015. Writing narrative style literature reviews. Medical Writing, 24(4), pp.230–235. Available from: https://doi.org/10.1179/2047480615Z.000000000329. DOI: https://doi.org/10.1179/2047480615Z.000000000329

Filiz, E. and Öz, E., 2019. Finding the best algorithms and effective factors in classification of Turkish science student success. Journal of Baltic Science Education, 18(2), pp.239–253. Available from: https://doi.org/10.33225/jbse/19.18.239. DOI: https://doi.org/10.33225/jbse/19.18.239

Gray, K., 2007. Educational Technology Practitioner-Research Ethics. In: M. Quigley, ed. Encyclopedia of Information Ethics and Security. Hershey, PA: IGI Global Scientific Publishing, pp.164–169. Available from: https://doi.org/10.4018/978-1-59140-987-8.ch025. DOI: https://doi.org/10.4018/978-1-59140-987-8.ch025

Greenhalgh, T., Thorne, S. and Malterud, K., 2018. Time to challenge the spurious hierarchy of systematic over narrative reviews? European Journal of Clinical Investigation, 48(6), p.e12931. Available from: https://doi.org/10.1111/eci.12931. DOI: https://doi.org/10.1111/eci.12931

Heinrich, E., 2024. Revolutionising educational technology: The imperative for authentic qualitative research. Social Sciences & Humanities Open, 10, p.101073. Available from: https://doi.org/10.1016/j.ssaho.2024.101073. DOI: https://doi.org/10.1016/j.ssaho.2024.101073

Ismail, I.A., 2024. Protecting Privacy in AI-Enhanced Education: A Comprehensive Examination of Data Privacy Concerns and Solutions in AI-Based Learning. In: S. Elmoudden and J. Wrench, eds. Impacts of Generative AI on the Future of Research and Education. Hershey, PA: IGI Global Scientific Publishing, chap. 10. Available from: https://doi.org/10.4018/979-8-3693-0831-8.ch010. DOI: https://doi.org/10.4018/979-8-3693-0884-4.ch006

Jacobsen, M. and McKenney, S., 2024. Educational design research: grappling with methodological fit. Educational Technology Research and Development, 72(5), pp.2743–2762. Available from: https://doi.org/10.1007/s11423-023-10282-5. DOI: https://doi.org/10.1007/s11423-023-10282-5

Jenkins, M.C., 2024. Mixed-Methods Research. In: B. Hott, F. Brigham and C. Peltier, eds. Research Methods in Special Education. New York: Routledge, pp.239–255. Available from: https://doi.org/10.4324/9781003526315-13. DOI: https://doi.org/10.4324/9781003526315-13

Joksimović, S., Kovanović, V., Jovanović, J., Zouaq, A., Gašević, D. and Hatala, M., 2015. What do cMOOC participants talk about in social media? a topic analysis of discourse in a cMOOC. Proceedings of the Fifth International Conference on Learning Analytics And Knowledge. New York, NY, USA: Association for Computing Machinery, LAK ’15, p.156–165. Available from: https://doi.org/10.1145/2723576.2723609. DOI: https://doi.org/10.1145/2723576.2723609

Joyce, K.E. and Cartwright, N., 2020. Bridging the Gap Between Research and Practice: Predicting What Will Work Locally. American Educational Research Journal, 57(3), pp.1045–1082. Available from: https://doi.org/10.3102/0002831219866687. DOI: https://doi.org/10.3102/0002831219866687

Kansagra, J., Singhadiya, C. and Suryanarayana, G., 2025. AI-enhanced evaluation: A survey of machine learning techniques for assessing long answers. In: S.K. Oruganti, D. Karras, S. Thakur, J.K. Chaithanya, S. Metta and A. Lathigara, eds. Digital Transformation and Sustainability of Business. London: CRC Press, pp.120–123. Available from: https://doi.org/10.1201/9781003606185-28. DOI: https://doi.org/10.1201/9781003606185-28

Karthikeyan, P., Nithish, K.K., Janani, H., Reshma, P., Vaishnavi, V. and Prakash, N., 2025. Education Revolution: LLM’s in Online Learning and Student Engagement. In: A. Bajaj, A. Abraham and R. Kamimura, eds. Bio-Inspired Computing. Cham: Springer Nature Switzerland, Lecture Notes in Networks and Systems, vol. 1229, pp.164–174. Available from: https://doi.org/10.1007/978-3-031-78940-3_16. DOI: https://doi.org/10.1007/978-3-031-78940-3_16

Ketsman, O., Droog, A. and Qazi, S., 2025. Mapping the prevalence of mixed methods research in educational technology journals. Computers & Education, 226, p.105207. Available from: https://doi.org/10.1016/j.compedu.2024.105207. DOI: https://doi.org/10.1016/j.compedu.2024.105207

Kishor, K., Vishwakarma, P., Sengar, L., Kumar, V. and Gupta, V., 2025. Development of Grader Provider System Using Deep Learning. Procedia Computer Science. vol. 259, pp.172–181. All Open Access. Available from: https://doi.org/10.1016/j.procs.2025.03.318. DOI: https://doi.org/10.1016/j.procs.2025.03.318

Kolmos, A., 2015. Design-based research – issues in connecting theory, research and practice. 6th Research in Engineering Education Symposium: Translating Research into Practice, REES 2015. Dublin Institute of Technology. Available from: https://vbn.aau.dk/ws/portalfiles/portal/221792862/Kolmos_Design_Based_Research_Connecting_Theory_Research_and_Practice.pdf.

Kumar, M., Singh, N., Wadhwa, J., Singh, P., Kumar, G. and Qtaishat, A., 2024. Utilizing Random Forest and XGBoost Data Mining Algorithms for Anticipating Students’ Academic Performance. International Journal of Modern Education and Computer Science, 16(2), pp.29–44. Available from: https://doi.org/10.5815/ijmecs.2024.02.03. DOI: https://doi.org/10.5815/ijmecs.2024.02.03

Lincoln, Y.S. and Guba, E.G., 1985. Naturalistic Inquiry. SAGE Publications, Inc. DOI: https://doi.org/10.1016/0147-1767(85)90062-8

Love, H.R., Fettig, A. and Steed, E.A., 2023. Putting the “Mix” in Mixed Methods: How to Integrate Quantitative and Qualitative Research in Early Childhood Special Education Research. Topics in Early Childhood Special Education, 43(3), pp.174–186. Available from: https://doi.org/10.1177/02711214231199268. DOI: https://doi.org/10.1177/02711214231199268

Makel, M.C., Hodges, J., Cook, B.G. and Plucker, J.A., 2021. Both Questionable and Open Research Practices are Prevalent in Education Research. Educational Researcher, 50(8), pp.493–504. Available from: https://doi.org/10.3102/0013189X211001356. DOI: https://doi.org/10.3102/0013189X211001356

Medina, J.R. and Avi, B.R., 2021. Mixed methods in education research. In: C.J. Gómez Carrasco, P. Miralles Martínez and R. López Facal, eds. Handbook of Research on Teacher Education in History and Geography. Peter Lang, pp.53–70.

Narayana, B.J.L., Harsha, V.N.S.S., Prudhvi, K., Vardhan, B.G.H. and Sravika, A., 2024. Automated Question Paper Generation using Natural Language Processing. 2024 International Conference on Computational Intelligence for Green and Sustainable Technologies, ICCIGST 2024. Available from: https://doi.org/10.1109/ICCIGST60741.2024.10717510. DOI: https://doi.org/10.1109/ICCIGST60741.2024.10717510

Osmanović Zajić, J. and Maksimović, J., 2022. Quasi-Experimental Research as An Epistemological-Methodological Approach in Education Research. International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 10(3), p.177–183. Available from: https://doi.org/10.23947/2334-8496-2022-10-3-177-183. DOI: https://doi.org/10.23947/2334-8496-2022-10-3-177-183

Perry, T., Morris, R. and Lea, R., 2022. A decade of replication study in education? a mapping review (2011–2020). Educational Research and Evaluation, 27(1-2), pp.12–34. Available from: https://doi.org/10.1080/13803611.2021.2022315. DOI: https://doi.org/10.1080/13803611.2021.2022315

Peters, M. and Fàbregues, S., 2024. Missed opportunities in mixed methods EdTech research? Visual joint display development as an analytical strategy for achieving integration in mixed methods studies. Educational technology research and development, 72(5), pp.2477–2497. Available from: https://doi.org/10.1007/s11423-023-10234-z. DOI: https://doi.org/10.1007/s11423-023-10234-z

Pretorius, L., 2024. Demystifying Research Paradigms: Navigating Ontology, Epistemology, and Axiology in Research. Qualitative Report, 29(10), pp.2698–2715. Available from: https://doi.org/10.46743/2160-3715/2024.7632. DOI: https://doi.org/10.46743/2160-3715/2024.7632

Prinsloo, P., Khalil, M. and Slade, S., 2024. Learning analytics as data ecology: a tentative proposal. Journal of Computing in Higher Education, 36(1), pp.154–182. Available from: https://doi.org/10.1007/s12528-023-09355-4. DOI: https://doi.org/10.1007/s12528-023-09355-4

Romeo, G. and Russell, G., 2010. Why ‘what works’ is not enough for information technology in education research. In: A. McDougall, J. Murnane, A. Jones and N. Reynolds, eds. Researching IT in Education: Theory, Practice and Future Directions. Routledge, chap. 6, pp.54–61.

Saqr, M. and López-Pernas, S., eds, 2024. Learning Analytics Methods and Tutorials: A Practical Guide Using R. Cham: Springer. Available from: https://doi.org/10.1007/978-3-031-54464-4. DOI: https://doi.org/10.1007/978-3-031-54464-4

Sharma, M. and Gupta, S., 2024. The Socioeconomic Impact of AI in Research and Education. In: A. Mutawa, ed. Impacts of Generative AI on the Future of Research and Education. Hershey, PA: IGI Global Scientific Publishing, chap. 17, pp.445–473. Available from: https://doi.org/10.4018/979-8-3693-0884-4.ch017. DOI: https://doi.org/10.4018/979-8-3693-0884-4.ch017

Siemens, G., 2005. Connectivism: A Learning Theory for the Digital Age. International Journal of Instructional Technology and Distance Learning, 2(1). Available from: https://itdl.org/Journal/Jan_05/article01.htm.

Stoumpos, A.I. and Stoumpou, R.I., 2025. Modern Digital and Technological Educational Methods. Trends in Higher Education, 4(2), p.25. Available from: https://doi.org/10.3390/higheredu4020025. DOI: https://doi.org/10.3390/higheredu4020025

Sullivan, E., 2024. SIDEs: Separating Idealization from Deceptive ‘Explanations’ in xAI. Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. New York, NY, USA: Association for Computing Machinery, FAccT ’24, p.1714–1724. Available from: https://doi.org/10.1145/3630106.3658999. DOI: https://doi.org/10.1145/3630106.3658999

Svihla, V., 2014. Advances in Design-Based Research. Frontline Learning Research, 2(4), pp.35–45. Available from: https://doi.org/10.14786/flr.v2i4.114. DOI: https://doi.org/10.14786/flr.v2i4.114

Tamim, R.M., Borokhovski, E., Bernard, R.M., Schmid, R.F., Abrami, P.C. and Pickup, D.I., 2021. A study of meta-analyses reporting quality in the large and expanding literature of educational technology. Australasian Journal of Educational Technology, 37(4), pp.100–115. Available from: https://doi.org/10.14742/ajet.6322. DOI: https://doi.org/10.14742/ajet.6322

Tepgeç, M. and Ifenthaler, D., 2022. Learning Analytics Based Interventions: A Systematic Review of Experimental Studies. Proceedings of the 19th International Conference on Cognition and Exploratory Learning in the Digital Age, CELDA 2022. pp.327–330. Available from: https://www.celda-conf.org/wp-content/uploads/2022/11/3_CELDA2022_S_125.pdf.

Thomson, G. and Crowther, S., 2022. Attuning to trustworthiness and final reflections. In: S. Crowther and G. Thomson, eds. Hermeneutic Phenomenology in Health and Social Care Research. London: Routledge, pp.213–227. Available from: https://doi.org/10.4324/9781003081661-13. DOI: https://doi.org/10.4324/9781003081661-13

Tinoca, L., Piedade, J., Santos, S., Pedro, A. and Gomes, S., 2022. Design-Based Research in the Educational Field: A Systematic Literature Review. Education Sciences, 12(6), p.410. Available from: https://doi.org/10.3390/educsci12060410. DOI: https://doi.org/10.3390/educsci12060410

Tobin, G.A. and Begley, C.M., 2004. Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48(4), pp.388–396. Available from: https://doi.org/10.1111/j.1365-2648.2004.03207.x. DOI: https://doi.org/10.1111/j.1365-2648.2004.03207.x

Waller, A.A., 2006. Special Session - Fish is Fish: Learning to See the Sea We Swim In: Theoretical Frameworks for Education Research. Proceedings. Frontiers in Education. 36th Annual Conference. pp.1–2. Available from: https://doi.org/10.1109/FIE.2006.322605. DOI: https://doi.org/10.1109/FIE.2006.322605

Wang, Q., 2025. The Optimization of an English Writing Automated Assessment Model Based on Big Data Analysis. International Journal of Web-Based Learning and Teaching Technologies, 20(1), pp.1–15. All Open Access. Available from: https://doi.org/10.4018/IJWLTT.389876. DOI: https://doi.org/10.4018/IJWLTT.389876

Wei, Z. and Yuan, M., 2023. Research on the Current Situation and Future Development Trend of Immersive Virtual Reality in the Field of Education. Sustainability, 15(9), p.7531. Available from: https://doi.org/10.3390/su15097531. DOI: https://doi.org/10.3390/su15097531

Williamson, K., 2006. Research in Constructivist Frameworks Using Ethnographic Techniques. Library Trends, 55(1), pp.83–101. Available from: https://doi.org/10.1353/lib.2006.0054. DOI: https://doi.org/10.1353/lib.2006.0054

Yılmaz, K. and Deniz, K.Z., 2024. Natural Language Processing and Machine Learning Applications For Assessment and Evaluation in Education: Opportunities and New Approaches. Journal of Measurement and Evaluation in Education and Psychology, 15(4), pp.421–445. Available from: https://doi.org/10.21031/epod.1551568. DOI: https://doi.org/10.21031/epod.1551568

Yuxin, M. and Harmon, S.W., 2009. A Case Study of Design-Based Research for Creating a Vision Prototype of a Technology-Based Innovative Learning Environment. Journal of Interactive Learning Research, 20(1), pp.75–93. Available from: https://www.learntechlib.org/primary/p/25226/.

Zhou, Y., Zhou, Y. and Machtmes, K., 2024. Mixed methods integration strategies used in education: A systematic review. Methodological Innovations, 17(1), pp.41–49. Available from: https://doi.org/10.1177/20597991231217937. DOI: https://doi.org/10.1177/20597991231217937