Teaching WebAR development with integrated machine learning: a methodology for immersive and intelligent educational experiences

Authors

DOI:

https://doi.org/10.55056/ed.660

Keywords:

web-based augmented reality, WebAR, machine learning, TensorFlow.js, Teachable Machine, educational technology

Abstract

Augmented reality (AR) and machine learning (ML) are rapidly growing technologies with immense potential for transforming education. Web-based augmented reality (WebAR) provides a promising approach to delivering immersive learning experiences on mobile devices. Integrating machine learning models into WebAR applications can enable advanced interactive effects by responding to user actions, thus enhancing the educational content. However, there is a lack of effective methodologies to teach students WebAR development with integrated machine learning. This paper proposes a methodology with three main steps: (1) Integrating standard TensorFlow.js models like handpose into WebAR scenes for gestures and interactions; (2) Developing custom image classification models with Teachable Machine and exporting to TensorFlow.js; (3) Modifying WebAR applications to load and use exported custom models, displaying model outputs as augmented reality content. The proposed methodology is designed to incrementally introduce machine learning integration, build an understanding of model training and usage, and spark ideas for using machine learning to augment educational content. The methodology provides a starting point for further research into pedagogical frameworks, assessments, and empirical studies on teaching WebAR development with embedded intelligence.

Downloads

Download data is not yet available.
Abstract views: 98 / PDF (Ukrainian) views: 19

References

LATICE ’14: Proceedings of the 2014 International Conference on Teaching and Learning in Computing and Engineering. IEEE Computer Society, USA (2014), ISBN 9781479935925, URL https://www.computer.org/csdl/proceedings/latice/2014/12OmNrAdsty

Proceedings of 2015 International Conference on Interactive Mobile Communication Technologies and Learning, IMCL 2015. Institute of Electrical and Electronics Engineers Inc. (2015), https://doi.org/10.1109/IMCL37494.2015 DOI: https://doi.org/10.1109/IMCL37494.2015

Innovative Technologies and Learning: 4th International Conference, ICI-TL 2021, Virtual Event, November 29 – December 1, 2021, Proceedings, Lecture Notes in Computer Science, vol. 13117. Springer International Publishing (2021), ISBN 9783030915407, https://doi.org/10.1007/978-3-030-91540-726

Preface. Journal of Physics: Conference Series 1860(1), 011001 (mar 2021), https://doi.org/10.1088/1742-6596/1860/1/011001 DOI: https://doi.org/10.1088/1742-6596/1860/1/011001

Face Landmarks Detection (2023), URL https://github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection

Face reference assets for Meta Spark Studio (2023), URL https://spark.meta.com/learn/articles/people-tracking/face-reference-assets

Hand Pose Detection (2023), URL https://github.com/tensorflow/tfjs-models/tree/master/hand-pose-detection

MobileNet (2023), URL https://github.com/tensorflow/tfjs-models/tree/master/mobilenet

The face mask template in Adobe® Photoshop® (2023), URL https://spark.meta.com/learn/articles/creating-and-prepping-assets/the-face-mask-template-in-Adobe

AR.js Documentation (2024), URL https://ar-js-org.github.io/AR.js-Docs/

Find Pre-trained Models | Kaggle (2024), URL https://www.kaggle.com/models

Pose Detection in the Browser: PoseNet Model (2024), URL https://github.com/tensorflow/tfjs-models/tree/master/posenet

Speech Command Recognizer (2024), URL https://github.com/tensorflow/tfjs-models/tree/master/speech-commands

TensorFlow (2024), URL https://www.tensorflow.org/

TensorFlow.js | Machine Learning for JavaScript Developers (2024), URL https://www.tensorflow.org/js

TensorFlow.js models (2024), URL https://www.tensorflow.org/js/models

Three.js – JavaScript 3D Library (2024), URL https://threejs.org/

Al-Gharaibeh, J., Jeffery, C.: Portable non-player character tutors with quest activities. In: 2010 IEEE Virtual Reality Conference (VR), pp. 253–254 (2010), https://doi.org/10.1109/VR.2010.5444779 DOI: https://doi.org/10.1109/VR.2010.5444779

An, J., Poly, L.P., Holme, T.A.: Usability Testing and the Development of an Augmented Reality Application for Laboratory Learning. Journal of Chemical Education 97, 97–105 (2020), https://doi.org/10.1021/acs.jchemed.9b00453 DOI: https://doi.org/10.1021/acs.jchemed.9b00453

Antoniou, P.E., Dafli, E., Arfaras, G., Bamidis, P.D.: Versatile Mixed Reality Educational Spaces; A Medical Education Implementation Case. In: Georgalas, N., Jin, Q., Garcia-Blas, J., Carretero, J., Ray, I. (eds.) Proceedings - 2016 15th International Conference on Ubiquitous Computing and Communications and 2016 8th International Symposium on Cyberspace and Security, IUCC-CSS 2016, pp. 132–137, Institute of Electrical and Electronics Engineers Inc. (2017), https://doi.org/10.1109/IUCC-CSS.2016.026 DOI: https://doi.org/10.1109/IUCC-CSS.2016.026

Antoniou, P.E., Dafli, E., Arfaras, G., Bamidis, P.D.: Versatile mixed reality medical educational spaces; requirement analysis from expert users. Personal and Ubiquitous Computing 21, 1015–1024 (2017), https://doi.org/10.1007/s00779-017-1074-5 DOI: https://doi.org/10.1007/s00779-017-1074-5

Anwar, S., LeClair, J., Peskin, A.: Development Of Nanotechnology And Power Systems Options For An On Line Bseet Degree. In: 2010 Annual Conference & Exposition, pp. 15.420.1 – 15.420.10, ASEE Conferences, Louisville, Kentucky (June 2010), https://doi.org/10.18260/1-2--15776 DOI: https://doi.org/10.18260/1-2--15776

Arteaga, J.V., Gravini-Donado, M.L., Riva, L.D.Z.: Digital Technologies for Heritage Teaching: Trend Analysis in New Realities. International Journal of Emerging Technologies in Learning 16, 132–148 (2021), https://doi.org/10.3991/ijet.v16i21.25149 DOI: https://doi.org/10.3991/ijet.v16i21.25149

Arvanitis, T.N., Petrou, A., Knight, J.F., Savas, S., Sotiriou, S., Gargalakos, M., Gialouri, E.: Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Personal and Ubiquitous Computing 13, 243–250 (2009), https://doi.org/10.1007/s00779-007-0187-7 DOI: https://doi.org/10.1007/s00779-007-0187-7

Atmaca, H.T., Terzi, O.S.: Building a Web-Augmented Reality application for demonstration of kidney pathology for veterinary education. Polish Journal of Veterinary Sciences 24, 345–350 (2021), https://doi.org/10.24425/pjvs.2021.137671 DOI: https://doi.org/10.24425/pjvs.2021.137671

Baashar, Y., Alkawsi, G., Ahmad, W.N.W., Alhussian, H., Alwadain, A., Capretz, L.F., Babiker, A., Alghail, A.: Effectiveness of Using Augmented Reality for Training in the Medical Professions: Meta-analysis. JMIR Serious Games 10, e32715 (2022), https://doi.org/10.2196/32715 DOI: https://doi.org/10.2196/32715

Bhavika, K., Martin, J., Ardit, B.: Technology will never replace hands on surgical training in plastic surgery. Journal of Plastic, Reconstructive and Aesthetic Surgery 75, 439–488 (2022), https://doi.org/10.1016/j.bjps.2021.11.034 DOI: https://doi.org/10.1016/j.bjps.2021.11.034

Bradford, H.M., Farley, C.L., Escobar, M., Heitzler, E.T., Tringali, T., Walker, K.C.: Rapid Curricular Innovations During COVID-19 Clinical Suspension: Maintaining Student Engagement with Simulation Experiences. Journal of Midwifery and Women’s Health 66, 366–371 (2021), https://doi.org/10.1111/jmwh.13246 DOI: https://doi.org/10.1111/jmwh.13246

Brewster, S., Murray-Smith, R. (eds.): Haptic Human-Computer Interaction: First International Workshop, Glasgow, UK, August 31 - September 1, 2000, Proceedings, Lecture Notes in Computer Science, vol. 2058. Springer-Verlag, Berlin Heidelberg (2001), ISBN 9783540445890, https://doi.org/10.1007/3-540-44589-7 DOI: https://doi.org/10.1007/3-540-44589-7

Brunzini, A., Papetti, A., Serrani, E.B., Scafà, M., Germani, M.: How to Improve Medical Simulation Training: A New Methodology Based on Ergonomic Evaluation. In: Karwowski, W., Ahram, T., Nazir, S. (eds.) Advances in Human Factors in Training, Education, and Learning Sciences, Advances in Intelligent Systems and Computing, vol. 963, pp. 145–155, Springer International Publishing, Cham (2020), ISBN 978-3-030-20135-7, https://doi.org/10.1007/978-3-030-20135-7_14 DOI: https://doi.org/10.1007/978-3-030-20135-7_14

Budiharto, W., Gunawan, A.A.S., Wulandhari, L.A., Williem, Faisal, Sutoyo, R., Meiliana, Suryani, D., Arifin, Y. (eds.): The 3rd International Conference on Computer Science and Computational Intelligence (ICCSCI 2018) : Empowering Smart Technology in Digital Era for a Better Life, Procedia Computer Science, vol. 135. Elsevier B.V. (2018), URL https://www.sciencedirect.com/journal/procedia-computer-science/vol/135/suppl/C DOI: https://doi.org/10.1016/j.procs.2018.08.140

Burian, B.K., Ebnali, M., Robertson, J.M., Musson, D., Pozner, C.N., Doyle, T., Smink, D.S., Miccile, C., Paladugu, P., Atamna, B., Lipsitz, S., Yule, S., Dias, R.D.: Using extended reality (XR) for medical training and real-time clinical support during deep space missions. Applied Ergonomics 106, 103902 (2023), https://doi.org/10.1016/j.apergo.2022.103902 DOI: https://doi.org/10.1016/j.apergo.2022.103902

Cardenas-Sainz, B., Zatarain-Cabada, R., Barron-Estrada, M., Chavez-Echeagaray, M., Cabada, R.: FisicARtivo: Design of a learning tool for physics education using web-based XR technology. In: 2022 IEEE Mexican International Conference on Computer Science, ENC 2022 - Proceedings, Institute of Electrical and Electronics Engineers Inc. (2022), https://doi.org/10.1109/ENC56672.2022.9882930 DOI: https://doi.org/10.1109/ENC56672.2022.9882930

Centre for Science and Technology Studies, Leiden University, The Netherlands: VOSviewer - Visualizing scientific landscapes (2024), URL https://www.vosviewer.com/

Coma-Tatay, I., Casas-Yrurzum, S., Casanova-Salas, P., Fernández-Marı́n, M.: FI-AR learning: a web-based platform for augmented reality educational content. Multimedia Tools and Applications 78, 6093–6118 (2019), https://doi.org/10.1007/s11042-018-6395-5 DOI: https://doi.org/10.1007/s11042-018-6395-5

Cortés Rodrı́guez, F., Dal Peraro, M., Abriata, L.A.: Online tools to easily build virtual molecular models for display in augmented and virtual reality on the web. Journal of Molecular Graphics and Modelling 114, 108164 (2022), https://doi.org/10.1016/j.jmgm.2022.108164 DOI: https://doi.org/10.1016/j.jmgm.2022.108164

Coughlin, T.: Impact of COVID-19 on the Consumer Electronics Market. IEEE Consumer Electronics Magazine 10, 58–59 (2021), https://doi.org/10.1109/MCE.2020.3016753 DOI: https://doi.org/10.1109/MCE.2020.3016753

Crandall, P.G., Engler III, R.K., Beck, D.E., Killian, S.A., O’Bryan, C.A., Jarvis, N., Clausen, E.: Development of an Augmented Reality Game to Teach Abstract Concepts in Food Chemistry. Journal of Food Science Education 14(1), 18–23 (2015), https://doi.org/https://doi.org/10.1111/1541-4329.12048 DOI: https://doi.org/10.1111/1541-4329.12048

Dar, S.A.: Mobile library initiatives: a new way to revitalize the academic library settings. Library Hi Tech News 36, 15–21 (2019), https://doi.org/10.1108/LHTN-05-2019-0032 DOI: https://doi.org/10.1108/LHTN-05-2019-0032

Demir, I.: Interactive web-based hydrological simulation system as an education platform. In: Rizzoli, A.E., Quinn, N.W.T., Ames, D.P. (eds.) Proceedings - 7th International Congress on Environmental Modelling and Software: Bold Visions for Environmental Modeling, iEMSs 2014, vol. 2, pp. 910–912, International Environmental Modelling and Software Society (2014), https://doi.org/10.17077/aseenmw2014.1008 DOI: https://doi.org/10.17077/aseenmw2014.1008

Dunkel, L., Fernandez-Luque, L., Loche, S., Savage, M.O.: Digital technologies to improve the precision of paediatric growth disorder diagnosis and management. Growth Hormone and IGF Research 59, 101408 (2021), https://doi.org/10.1016/j.ghir.2021.101408 DOI: https://doi.org/10.1016/j.ghir.2021.101408

Erçağ, E., Yasakcı, A.: The Perception Scale for the 7E Model-Based Augmented Reality Enriched Computer Course (7EMAGBAÖ): Validity and Reliability Study. Sustainability 14, 12037 (2022), https://doi.org/10.3390/su141912037 DOI: https://doi.org/10.3390/su141912037

Farella, M., Taibi, D., Arrigo, M., Todaro, G., Fulantelli, G., Chiazzese, G.: An Augmented Reality Mobile Learning Experience Based on Treasure Hunt Serious Game. In: Busch, C., Steinicke, M., Friess, R., Wendler, T. (eds.) Proceedings of the European Conference on e-Learning, ECEL, pp. 148–154, Academic Conferences and Publishing International Limited (2021), https://doi.org/10.34190/EEL.21.109

Faridi, E., Ghaderian, A., Honarasa, F., Shafie, A.: Next generation of chemistry and biochemistry conference posters: Animation, augmented reality, visitor statistics, and visitors’ attention. Biochemistry and Molecular Biology Education 49, 619–624 (2021), https://doi.org/10.1002/bmb.21520 DOI: https://doi.org/10.1002/bmb.21520

Farra, S., Hodgson, E., Miller, E., Timm, N., Brady, W., Gneuhs, M., Ying, J., Hausfeld, J., Cosgrove, E., Simon, A., Bottomley, M.: Effects of Virtual Reality Simulation on Worker Emergency Evacuation of Neonates. Disaster Medicine and Public Health Preparedness 13, 301–308 (2019), https://doi.org/10.1017/dmp.2018.58 DOI: https://doi.org/10.1017/dmp.2018.58

Ferguson, J., Mentzelopoulos, M., Protopsaltis, A., Economou, D.: Small and flexible web based framework for teaching QR and AR mobile learning application development. In: Proceedings of 2015 International Conference on Interactive Mobile Communication Technologies and Learning, IMCL 2015, pp. 383–385, Institute of Electrical and Electronics Engineers Inc. (2015), https://doi.org/10.1109/IMCTL.2015.7359624 DOI: https://doi.org/10.1109/IMCTL.2015.7359624

Google: Teachable Machine (2017), URL https://teachablemachine.withgoogle.com/

Google LLC: Face landmark detection guide | MediaPipe | Google for Developers (2023), URL https://developers.google.com/mediapipe/solutions/vision/face_landmarker/

Gordon, N., Brayshaw, M., Aljaber, T.: Heuristic Evaluation for Serious Immersive Games and M-instruction. In: Zaphiris, P., Ioannou, A. (eds.) Learning and Collaboration Technologies, Lecture Notes in Computer Science, vol. 9753, pp. 310–319, Springer International Publishing, Cham (2016), ISBN 978-3-319-39483-1, https://doi.org/10.1007/978-3-319-39483-1_29 DOI: https://doi.org/10.1007/978-3-319-39483-1_29

Hai-Jew, S.: Adult Coloring Books as Emotional Salve/Stress Relief, Tactual-Visual Learning: An Analysis from Mass-Scale Social Imagery. In: Common Visual Art in a Social Digital Age, pp. 171–186, Nova Science Publishers, Inc. (2022) DOI: https://doi.org/10.52305/BKOT1869

Harun, Tuli, N., Mantri, A.: Experience Fleming’s rule in electromagnetism using augmented reality: Analyzing impact on students learning. Procedia Computer Science 172, 660–668 (2020), https://doi.org/10.1016/j.procs.2020.05.08630 DOI: https://doi.org/10.1016/j.procs.2020.05.086

Hensen, B., Koren, I., Klamma, R., Herrler, A.: An Augmented Reality Framework for Gamified Learning. In: Hancke, G., Spaniol, M., Osathanunkul, K., Unankard, S., Klamma, R. (eds.) Advances in Web-Based Learning – ICWL 2018, Lecture Notes in Computer Science, vol. 11007, pp. 67–76, Springer International Publishing, Cham (2018), ISBN 978-3-319-96565-9, https://doi.org/10.1007/978-3-319-96565-9_7 DOI: https://doi.org/10.1007/978-3-319-96565-9_7

Hoog, T.G., Aufdembrink, L.M., Gaut, N.J., Sung, R.J., Adamala, K.P., Engelhart, A.E.: Rapid deployment of smartphone-based augmented reality tools for field and online education in structural biology. Biochemistry and Molecular Biology Education 48, 448–451 (2020), https://doi.org/10.1002/bmb.21396 DOI: https://doi.org/10.1002/bmb.21396

Huang, L.: Chemistry Apps on Smartphones and Tablets. In: Garcı́a-Martı́nez, J., Serrano-Torregrosa, E. (eds.) Chemistry Education, chap. 25, pp. 621–650, John Wiley & Sons, Ltd (2015), ISBN 9783527679300, https://doi.org/10.1002/9783527679300.ch25 DOI: https://doi.org/10.1002/9783527679300.ch25

Huang, T.C.: Seeing creativity in an augmented experiential learning environment. Universal Access in the Information Society 18, 301–313 (2019), https://doi.org/10.1007/s10209-017-0592-2 DOI: https://doi.org/10.1007/s10209-017-0592-2

Ibanez, M.B., Di-Serio, A., Villaran-Molina, D., Delgado-Kloos, C.: Augmented Reality-Based Simulators as Discovery Learning Tools: An Empirical Study. IEEE Transactions on Education 58, 208–213 (2015), https://doi.org/10.1109/TE.2014.2379712 DOI: https://doi.org/10.1109/TE.2014.2379712

Ibáñez, M.B., Peláez, J., Kloos, C.D.: Using an Augmented Reality Geolocalized Quiz Game as an Incentive to Overcome Academic Procrastination. In: Auer, M.E., Tsiatsos, T. (eds.) Mobile Technologies and Applications for the Internet of Things, Advances in Intelligent Systems and Computing, vol. 909, pp. 175–184, Springer International Publishing, Cham (2019), ISBN 978-3-030-11434-3, https://doi.org/10.1007/978-3-030-11434-3_21 DOI: https://doi.org/10.1007/978-3-030-11434-3_21

Ibáñez, M., Uriarte Portillo, A., Zatarain Cabada, R., Barrón, M.: Impact of augmented reality technology on academic achievement and motivation of students from public and private Mexican schools. A case study in a middle-school geometry course. Computers and Education 145, 103734 (2020), https://doi.org/10.1016/j.compedu.2019.103734 DOI: https://doi.org/10.1016/j.compedu.2019.103734

Ibáñez, M.B., Di Serio, Á., Villarán, D., Delgado Kloos, C.: Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness. Computers and Education 71, 1–13 (2014), https://doi.org/10.1016/j.compedu.2013.09.004 DOI: https://doi.org/10.1016/j.compedu.2013.09.004

Jara, C.A., Candelas, F.A., Torres, F.: Internet virtual and remote control interface for robotics education. In: Developments in Higher Education, pp. 136–154, Nova Science Publishers, Inc. (2009)

Jung, K., Nguyen, V.T., Yoo, S.C., Kim, S., Park, S., Currie, M.: Palmitoar: The last battle of the U.S. Civil war reenacted using augmented reality. ISPRS International Journal of Geo-Information 9, 75 (2020), https://doi.org/10.3390/ijgi9020075 DOI: https://doi.org/10.3390/ijgi9020075

Kang, B., Heo, J., Choi, H.H.S., Lee, K.H.: 2030 Toy Web of the Future. In: Kim, S., Jung, J.W., Kubota, N. (eds.) Soft Computing in Intelligent Control, Advances in Intelligent Systems and Computing, vol. 272, pp. 69–75, Springer International Publishing, Cham (2014), ISBN 978-3-319-05570-1, https://doi.org/10.1007/978-3-319-05570-1_8 DOI: https://doi.org/10.1007/978-3-319-05570-1_8

Karas, S.I., Grakova, E.V., Balakhonova, M.V., Arzhanik, M.B., Kara-Sal, E.E.: Distance learning in cardiology: The use of multimedia clinical diagnostic tasks. Russian Journal of Cardiology 25, 187–194 (2020), https://doi.org/10.15829/1560-4071-2020-4116 DOI: https://doi.org/10.15829/1560-4071-2020-4116

Karayilan, M., McDonald, S.M., Bahnick, A.J., Godwin, K.M., Chan, Y.M., Becker, M.L.: Reassessing Undergraduate Polymer Chemistry Laboratory Experiments for Virtual Learning Environments. Journal of Chemical Education 99, 1877–1889 (2022), https://doi.org/10.1021/acs.jchemed.1c01259 DOI: https://doi.org/10.1021/acs.jchemed.1c01259

Katika, T., Bolierakis, S.N., Vasilopoulos, E., Antonopoulos, M., Tsimiklis, G., Karaseitanidis, I., Amditis, A.: Coupling AR with Object Detection Neural Networks for End-User Engagement. In: Zachmann, G., Alcañiz Raya, M., Bourdot, P., Marchal, M., Stefanucci, J., Yang, X. (eds.) Virtual Reality and Mixed Reality, Lecture Notes in Computer Science, vol. 13484, pp. 135–145, Springer International Publishing, Cham (2022), ISBN 978-3-031-16234-3, https://doi.org/10.1007/978-3-031-16234-3_8 DOI: https://doi.org/10.1007/978-3-031-16234-3_8

Kazanidis, I., Pellas, N., Christopoulos, A.: A learning analytics conceptual framework for augmented reality-supported educational case studies. Multimodal Technologies and Interaction 5, 9 (2021), https://doi.org/10.3390/mti5030009 DOI: https://doi.org/10.3390/mti5030009

Klavins, A.: 9 ideas for creating tech-infused augmented reality T-shirts (2021), URL https://overlyapp.com/blog/9-ideas-for-creating-tech-infused-augmented-reality-t-shirts/

Kobayashi, T., Sasaki, H., Toguchi, A., Mizuno, K.: A discussion on web-based learning contents with the AR technology and its authoring tools to improve students’ skills in exercise courses. In: Mohd Ayub, A.F., Kashihara, A., Matsui, T., Liu, C.C., Ogata, H., Kong, S.C. (eds.) Work-In-Progress Poster - Proceedings of the 22nd International Conference on Computers in Education, ICCE 2014, pp. 34–36, Asia-Pacific Society for Computers in Education (2014)

Le, H., Nguyen, M.: An Online Platform for Enhancing Learning Experiences with Web-Based Augmented Reality and Pictorial Bar Code. In: Geroimenko, V. (ed.) Augmented Reality in Education: A New Technology for Teaching and Learning, pp. 45–57, Springer Series on Cultural Computing, Springer International Publishing, Cham (2020), ISBN 978-3-030-42156-4, https://doi.org/10.1007/978-3-030-42156-4_3 DOI: https://doi.org/10.1007/978-3-030-42156-4_3

Liu, E., Cai, S., Liu, Z., Liu, C.: WebART : Web-based Augmented Reality Learning Resources Authoring Tool and Its User Experience Study among Teachers. IEEE Transactions on Learning Technologies 16, 53–65 (2023), https://doi.org/10.1109/TLT.2022.321485432 DOI: https://doi.org/10.1109/TLT.2022.3214854

Lou, D.: Two fast prototypes of web-based augmented reality enhancement for books. Library Hi Tech News 36, 19–24 (2019), https://doi.org/10.1108/LHTN-08-2019-0057 DOI: https://doi.org/10.1108/LHTN-08-2019-0057

Lytridis, C., Tsinakos, A., Kazanidis, I.: ARTutor—An augmented reality platform for interactive distance learning. Education Sciences 8, 6 (2018), https://doi.org/10.3390/educsci8010006 DOI: https://doi.org/10.3390/educsci8010006

Maggi, L.O., Teixeira, J.M.X.N., Junior, J.R.F.E.S., Cajueiro, J.P.C., De Lima, P.V.S.G., De Alencar Bezerra, M.H.R., Melo, G.N.: 3DJPi: An Open-Source Web-Based 3D Simulator for Pololu’s 3Pi Platform. In: Proceedings - 2019 21st Symposium on Virtual and Augmented Reality, SVR 2019, pp. 52–58, Institute of Electrical and Electronics Engineers Inc. (2019), https://doi.org/10.1109/SVR.2019.00025 DOI: https://doi.org/10.1109/SVR.2019.00025

Marı́n, R., Sanz, P.J.: The Human-Machine Interaction through the UJI Telerobotic Training System. In: Hamza, M.H. (ed.) IASTED International Conference Robotics and Applications, RA 2003, June 25-27, 2003, Salzburg, Austria, pp. 47–52, IASTED/ACTA Press (2003)

Marı́n, R., Sanz, P.J., Del Pobil, A.P.: The UJI Online Robot: An Education and Training Experience. Autonomous Robots 15, 283–297 (2003), https://doi.org/10.1023/A:1026220621431 DOI: https://doi.org/10.1023/A:1026220621431

MDN contributors: WebGL: 2D and 3D graphics for the web (2023), URL https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API

Narman, H.S., Berry, C., Canfield, A., Carpenter, L., Giese, J., Loftus, N., Schrader, I.: Augmented Reality for Teaching Data Structures in Computer Science. In: 2020 IEEE Global Humanitarian Technology Conference, GHTC 2020, p. 9342932, Institute of Electrical and Electronics Engineers Inc. (2020), https://doi.org/10.1109/GHTC46280.2020.9342932 DOI: https://doi.org/10.1109/GHTC46280.2020.9342932

Nemirovsky, D.R., Garcia, A.J., Gupta, P., Shoen, E., Walia, N.: Evaluation of Surgical Improvement of Clinical Knowledge Ops (SICKO), an Interactive Training Platform. Journal of Digital Imaging 34, 1067–1071 (2021), https://doi.org/10.1007/s10278-021-00482-x DOI: https://doi.org/10.1007/s10278-021-00482-x

ngrok: Unified Application Delivery Platform for Developers (2024), URL https://ngrok.com/

ngrok: Your Authtoken (2024), URL https://dashboard.ngrok.com/get-started/your-authtoken

Nguyen, M., Le, H., Lai, P.M., Yan, W.Q.: A web-based augmented reality platform using pictorial QR code for educational purposes and beyond. In: Spencer, S.N. (ed.) Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST, p. 3364793, Association for Computing Machinery (2019), https://doi.org/10.1145/3359996.3364793 DOI: https://doi.org/10.1145/3359996.3364793

Nguyen, V.T., Jung, K., Dang, T.: BlocklyAR: A visual programming interface for creating augmented reality experiences. Electronics 9, 1–20 (2020), https://doi.org/10.3390/electronics9081205 DOI: https://doi.org/10.3390/electronics9081205

Nguyen, V.T., Jung, K., Yoo, S., Kim, S., Park, S., Currie, M.: Civil war battlefield experience: Historical event simulation using augmented reality technology. In: Proceedings - 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality, AIVR 2019, pp. 294–297, Institute of Electrical and Electronics Engineers Inc. (2019), https://doi.org/10.1109/AIVR46125.2019.00068 DOI: https://doi.org/10.1109/AIVR46125.2019.00068

Nordin, N., Markom, M.A., Suhaimi, F.A., Ishak, S.: A Web-Based Campus Navigation System with Mobile Augmented Reality Intervention. Journal of Physics: Conference Series 1997, 012038 (2021), https://doi.org/10.1088/1742-6596/1997/1/012038 DOI: https://doi.org/10.1088/1742-6596/1997/1/012038

Nordin, N., Nordin, N.R.M., Omar, W.: REV-OPOLY: A Study on Educational Board Game with Webbased Augmented Reality. Asian Journal of University Education 18, 81–90 (2022), https://doi.org/10.24191/ajue.v18i1.17172 DOI: https://doi.org/10.24191/ajue.v18i1.17172

Proskura, S., Lytvynova, S., Kronda, O., Demeshkant, N.: Mobile Learning Approach as a Supplementary Approach in the Organization of the Studying Process in Educational Institutions. In: Sokolov, O., Zholtkevych, G., Yakovyna, V., Tarasich, Y., Kharchenko, V., Kobets, V., Burov, O., Semerikov, S., Kravtsov, H. (eds.) Proceedings of the 16th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer. Volume II: Workshops, Kharkiv, Ukraine, October 06-10, 2020, CEUR Workshop Proceedings, vol. 2732, pp. 650–664, CEUR-WS.org (2020), URL https://ceur-ws.org/Vol-2732/20200650.pdf

Proskura, S.L., Lytvynova, S.H.: The approaches to Web-based education of computer science bachelors in higher education institutions. CTE Workshop Proceedings 7, 609–625 (Mar 2020), https://doi.org/10.55056/cte.416 DOI: https://doi.org/10.55056/cte.416

Redondo, E., Navarro, I., Sánchez, A., Fonseca, D.: Implementation of Augmented Reality in “3.0 Learning” Methodology: Case Studies with Students of Architecture Degree. In: Pătruţ, B., Pătruţ, M., Cmeciu, C. (eds.) Social Media and the New Academic Environment: Pedagogical Challenges, pp. 391–413, IGI Global, Hershey, PA (2013), https://doi.org/10.4018/978-1-4666-2851-9.ch019 DOI: https://doi.org/10.4018/978-1-4666-2851-9.ch019

Rollo, M.E., Aguiar, E.J., Williams, R.L., Wynne, K., Kriss, M., Callister, R., Collins, C.E.: eHealth technologies to support nutrition and physical activity behaviors in diabetes self-management. Diabetes, Metabolic Syndrome and Obesity: Targets and Therapy 9, 381–390 (2016), https://doi.org/10.2147/DMSO.S95247 DOI: https://doi.org/10.2147/DMSO.S95247

Ryan, G., Murphy, J., Higgins, M., McAuliffe, F., Mangina, E.: Work-in-Progress-Development of a Virtual Reality Learning Environment: VR Baby. In: Economou, D., Klippel, A., Dodds, H., Pena-Rios, A., Lee, M.J.W., Beck, D., Pirker, J., Dengel, A., Peres, T.M., Richter, J. (eds.) Proceedings of 6th International Conference of the Immersive Learning Research Network, iLRN 2020, pp. 312–315, Institute of Electrical and Electronics Engineers Inc. (2020), https://doi.org/10.23919/iLRN47897.2020.9155203 DOI: https://doi.org/10.23919/iLRN47897.2020.9155203

Rønningsbakk, L., Wu, T.T., Sandnes, F.E., Huang, Y.M. (eds.): Innovative Technologies and Learning: Second International Conference, ICITL34 2019, Tromsø, Norway, December 2–5, 2019, Proceedings, Lecture Notes in Computer Science, vol. 11937. Springer International Publishing (2019), ISBN 9783030353438, https://doi.org/10.1007/978-3-030-35343-8 DOI: https://doi.org/10.1007/978-3-030-35343-8

Samat, C., Chaijaroen, S.: Design and Development of Constructivist Augmented Reality (AR) Book Enhancing Analytical Thinking in Computer Classroom. In: Rønningsbakk, L., Wu, T.T., Sandnes, F.E., Huang, Y.M. (eds.) Innovative Technologies and Learning, Lecture Notes in Computer Science, vol. 11937, pp. 175–183, Springer International Publishing, Cham (2019), ISBN 978-3-030-35343-8, https://doi.org/10.1007/978-3-030-35343-8_19 DOI: https://doi.org/10.1007/978-3-030-35343-8_19

Semerikov, S.O., Mintii, M.M., Mintii, I.S.: Review of the course “Development of Virtual and Augmented Reality Software” for STEM teachers: implementation results and improvement potentials. In: Lytvynova, S.H., Semerikov, S.O. (eds.) Proceedings of the 4th International Workshop on Augmented Reality in Education (AREdu 2021), Kryvyi Rih, Ukraine, May 11, 2021, CEUR Workshop Proceedings, vol. 2898, pp. 159–177, CEUR-WS.org (2021), URL http://ceur-ws.org/Vol-2898/paper09.pdf

Sendari, S., Wibawanto, S., Jasmine, J., Jiono, M., Puspitasari, P., Diantoro, M., Nur, H.: Integrating Robo-PEM with AR Application for Introducing Fuel Cell Implementation. In: 7th International Conference on Electrical, Electronics and Information Engineering: Technological Breakthrough for Greater New Life, ICEEIE 2021, Institute of Electrical and Electronics Engineers Inc. (2021), https://doi.org/10.1109/ICEEIE52663.2021.9616683 DOI: https://doi.org/10.1109/ICEEIE52663.2021.9616683

Sepasgozar, S.M.E.: Digital twin and web-based virtual gaming technologies for online education: A case of construction management and engineering. Applied Sciences 10, 4678 (2020), https://doi.org/10.3390/app10134678 DOI: https://doi.org/10.3390/app10134678

Sharkey, T., Twomey, R., Eguchi, A., Sweet, M., Wu, Y.C.: Need Finding for an Embodied Coding Platform: Educators’ Practices and Perspectives. In: Cukurova, M., Rummel, N., Gillet, D., McLaren, B., Uhomoibhi, J. (eds.) International Conference on Computer Supported Education, CSEDU - Proceedings, vol. 1, pp. 216–227, Science and Technology Publications, Lda (2022), https://doi.org/10.5220/0011000200003182 DOI: https://doi.org/10.5220/0011000200003182

Sharp, K., McCorvie, M., Wagner, M.: Sharing Hidden Histories: The XRchaeology at Miller Grove, a Free African American Community in Southern Illinois. Journal of African Diaspora Archaeology and Heritage 12(1), 5–31 (2023), https://doi.org/10.1080/21619441.2021.1902706 DOI: https://doi.org/10.1080/21619441.2021.1902706

Shepiliev, D.S., Modlo, Y.O., Yechkalo, Y.V., Tkachuk, V.V., Mintii, M.M., Mintii, I.S., Markova, O.M., Selivanova, T.V., Drashko, O.M., Kalinichenko, O.O., Vakaliuk, T.A., Osadchyi, V.V., Semerikov, S.O.: WebAR development tools: An overview. CEUR Workshop Proceedings 2832, 84–93 (2020), URL http://ceur-ws.org/Vol-2832/paper12.pdf DOI: https://doi.org/10.31812/123456789/4356

Shepiliev, D.S., Semerikov, S.O., Yechkalo, Y.V., Tkachuk, V.V., Markova, O.M., Modlo, Y.O., Mintii, I.S., Mintii, M.M., Selivanova, T.V., Maksyshko, N.K., Vakaliuk, T.A., Osadchyi, V.V., Tarasenko, R.O., Amelina, S.M., Kiv, A.E.: Development of career guidance quests using WebAR. Journal of Physics: Conference Series 1840(1), 012028 (mar 2021), https://doi.org/10.1088/1742-6596/1840/1/012028 DOI: https://doi.org/10.1088/1742-6596/1840/1/012028

Smith, E., McRae, K., Semple, G., Welsh, H., Evans, D., Blackwell, P.: Enhancing Vocational Training in the Post-COVID Era through Mobile Mixed Reality. Sustainability 13, 6144 (2021), https://doi.org/10.3390/su13116144 DOI: https://doi.org/10.3390/su13116144

Spasova, N., Ivanova, M.: Towards augmented reality technology in CAD/CAM systems and engineering education. In: Roceanu, I. (ed.) eLearning and Software for Education Conference, pp. 496–503, National Defence University - Carol I Printing House (2020), https://doi.org/10.12753/2066-026X-20-151 DOI: https://doi.org/10.12753/2066-026X-20-151

Syrovatskyi, O.V., Semerikov, S.O., Modlo, Y.O., Yechkalo, Y.V., Zelinska, S.O.: Augmented reality software design for educational purposes. CEUR Workshop Proceedings 2292, 193–225 (2018), ISSN 16130073, URL http://ceur-ws.org/Vol-2292/paper20.pdf DOI: https://doi.org/10.31812/123456789/2895

Tennakoon, D., Usmani, A.U., Usman, M., Vasileiou, A., Latchaev, S., Baljko, M., Khan, U.T., Perras, M.A., Jadidi, M.: Teaching Earth Systems Beyond the Classroom: Developing a Mixed Reality (XR) Sandbox. In: ASEE Annual Conference and Exposition, Conference Proceedings, American Society for Engineering Education (2022)

Thabvithorn, C., Samat, C.: Development of Web-Based Learning with Augmented Reality (AR) to Promote Analytical Thinking on Computational Thinking for High School. In: Huang, Y.M., Cheng, S.C., Barroso, J., Sandnes, F.E. (eds.) Innovative Technologies and Learning, Lecture Notes in Computer Science, vol. 13449, pp. 125–133, Springer International Publishing, Cham (2022), ISBN 978-3-031-15273-3, https://doi.org/10.1007/978-3-031-15273-3_14 DOI: https://doi.org/10.1007/978-3-031-15273-3_14

Tkachuk, V.V., Semerikov, S.O., Yechkalo, Y.V., Markova, O.M., Mintii, M.M.: WebAR development tools: comparative analysis. Physical and Mathematical Education (2(24)) (2020), https://doi.org/10.31110/2413-1571-2020-024-2-021 DOI: https://doi.org/10.31110/2413-1571-2020-024-2-021

Toguchi, A., Sasaki, H., Mizuno, K., Shikoda, A.: Build a prototype of new e-Learning contents by using the AR technology. In: IMSCI 2011 - 5th International Multi-Conference on Society, Cybernetics and Informatics, Proceedings, vol. 1, pp. 261–264, International Institute of Informatics and Systemics, IIIS (2011)

Toguchi, A., Sasaki, H., Mizuno, K., Shikoda, A.: Development of new e-Learning contents for improvement of laboratory courses by using the AR technology. In: IMSCI 2012 - 6th International Multi-Conference on Society, Cybernetics and Informatics, Proceedings, pp. 189–193, International Institute of Informatics and Systemics, IIIS (2012)

TrackJS LLC: Remote JavaScript Debugger - RemoteJS (2022), URL https://remotejs.com/36

Tsiatsos, T., Auer, M.E. (eds.): 11th International Conference on Interactive Mobile Communication Technologies and Learning, IMCL2017, Advances in Intelligent Systems and Computing, vol. 725. Springer Verlag (2018), https://doi.org/10.1007/978-3-319-75175-7 DOI: https://doi.org/10.1007/978-3-319-75175-7

Tuli, N., Mantri, A., Sharma, S.: Impact of augmented reality tabletop learning environment on learning and motivation of kindergarten kids. AIP Conference Proceedings 2357, 040017 (2022), https://doi.org/10.1063/5.0080600 DOI: https://doi.org/10.1063/5.0080600

Turner, F., Welch, I.: The mixed reality toolkit as the next step in the mass customization co-design experience. International Journal of Industrial Engineering and Management 10, 191–199 (2019), https://doi.org/10.24867/IJIEM-2019-2-239 DOI: https://doi.org/10.24867/IJIEM-2019-2-239

Vahabzadeh, A., Keshav, N.U., Salisbury, J.P., Sahin, N.T.: Improvement of Attention-Deficit/Hyperactivity Disorder Symptoms in School-Aged Children, Adolescents, and Young Adults With Autism via a Digital Smartglasses-Based Socioemotional Coaching Aid: Short-Term, Uncontrolled Pilot Study. JMIR Mental Health 5, e25 (2018), https://doi.org/10.2196/mental.9631 DOI: https://doi.org/10.2196/mental.9631

Villarán, D., Ibáñez, M.B., Kloos, C.D.: Augmented Reality-Based Simulations Embedded in Problem Based Learning Courses. In: Conole, G., Klobučar, T., Rensing, C., Konert, J., Lavoué, E. (eds.) Design for Teaching and Learning in a Networked World, Lecture Notes in Computer Science, vol. 9307, pp. 540–543, Springer International Publishing, Cham (2015), ISBN 978-3-319-24258-3, https://doi.org/10.1007/978-3-319-24258-3_55 DOI: https://doi.org/10.1007/978-3-319-24258-3_55

Wang, I., Nguyen, M., Le, H., Yan, W., Hooper, S.: Enhancing Visualisation of Anatomical Presentation and Education Using Marker-based Augmented Reality Technology on Web-based Platform. In: Proceedings of AVSS 2018 - 2018 15th IEEE International Conference on Advanced Video and Signal-Based Surveillance, p. 8639147, Institute of Electrical and Electronics Engineers Inc. (2019), https://doi.org/10.1109/AVSS.2018.8639147 DOI: https://doi.org/10.1109/AVSS.2018.8639147

Westwood, J.D., Westwood, S.W., Felländer-Tsai, L., Fidopiastis, C.M., Liu, A., Senger, S., Vosburgh, K.G. (eds.): Medicine Meets Virtual Reality 22 - NextMed, MMVR 2016, Los Angeles, California, USA, April 7-9, 2016, Studies in Health Technology and Informatics, vol. 220. IOS Press (2016), ISBN 978-1-61499-624-8, URL http://ebooks.iospress.nl/volume/medicine-meets-virtual-reality-22-nextmed-mmvr22

Wongchiranuwat, S., Samat, C.: Synthesis of theoretical framework for augmented reality learning environment to promote creative thinking on topic implementation of graphic design for grade 9 students. In: Wong, S.L., Barrera, A.G., Mitsuhara, H., Biswas, G., Jia, J., Yang, J.C., Banawan, M.P., Demirbilek, M., Gaydos, M., Lin, C.P., Shon, J.G., Iyer, S., Gulz, A., Holden, C., Kessler, G., Rodrigo, M.M.T., Sengupta, P., Taalas, P., Chen, W., Murthy, S., Kim, B., Ochoa, X., Sun, D., Baloian, N., Hoel, T., Hoppe, U., Hsu, T.C., Kukulska-Hulme, A., Chu, H.C., Gu, X., Chen, W., Huang, J.S., Jan, M.F., Wong, L.H., Yin, C. (eds.) ICCE 2016 - 24th International Conference on Computers in Education: Think Global Act Local - Main Conference Proceedings, pp. 639–641, Asia-Pacific Society for Computers in Education (2016), URL https://files.eric.ed.gov/fulltext/EJ1211500.pdf

Yang, S., Mei, B., Yue, X.: Mobile Augmented Reality Assisted Chemical Education: Insights from Elements 4D. Journal of Chemical Education 95, 1060–1062 (2018), https://doi.org/10.1021/acs.jchemed.8b00017 DOI: https://doi.org/10.1021/acs.jchemed.8b00017

Yuen, H.: HiuKim Yuen (2023), URL https://www.youtube.com/channel/UC-JyA1Z1-p0wgxj5WEX56wg/featured

Yuen, H.: MindAR (2023), URL https://hiukim.github.io/mind-ar-js-doc/

Zatarain-Cabada, R., Barrón-Estrada, M., Cárdenas-Sainz, B.A., Chavez-Echeagaray, M.E.: Experiences of web-based extended reality technologies for physics education. Computer Applications in Engineering Education 31(1), 63–82 (2023), https://doi.org/10.1002/cae.22571 DOI: https://doi.org/10.1002/cae.22571

Zitzmann, N.U., Matthisson, L., Ohla, H., Joda, T.: Digital undergraduate education in dentistry: A systematic review. International Journal of Environmental Research and Public Health 17, 3269 (2020), https://doi.org/10.3390/ijerph17093269 DOI: https://doi.org/10.3390/ijerph17093269

Published

15-06-2024

Issue

Section

Articles

How to Cite

Semerikov, S.O., Foki, M.V., Shepiliev, D.S., Mintii, M.M., Mintii, I.S. and Kuzminska, O.H., 2024. Teaching WebAR development with integrated machine learning: a methodology for immersive and intelligent educational experiences. Educational Dimension [Online], 10, pp.198–234. Available from: https://doi.org/10.55056/ed.660 [Accessed 22 July 2024].
Received 2023-03-21
Accepted 2024-02-19
Published 2024-06-15

Similar Articles

1-10 of 133

You may also start an advanced similarity search for this article.

Most read articles by the same author(s)

1 2 3 > >>