The Rusch model-based knowledge assessment algorithm

Alexander A. Kostikov; Kateryna V. Vlasenko; Iryna V. Lovianova; Sergii V. Volkov; Evgeny O. Avramov

doi:10.31812/educdim.4482

Authors

Alexander A. Kostikov Donbass State Engineering Academy https://orcid.org/0000-0003-3503-4836
Kateryna V. Vlasenko National University of Kyiv Mohyla Academy https://orcid.org/0000-0002-8920-5680
Iryna V. Lovianova Kryvyi Rih State Pedagogical University https://orcid.org/0000-0003-3186-2837
Sergii V. Volkov Volodymyr Dahl East Ukrainian National University https://orcid.org/0000-0001-7938-3080
Evgeny O. Avramov Donbass State Engineering Academy https://orcid.org/0000-0002-8405-7164

DOI:

https://doi.org/10.31812/educdim.4482

Keywords:

adaptive algorithm, Rasch model, Item Response Theory (IRT), information function of test item, latent variables

Abstract

In this paper, an algorithm for adaptive testing of students' knowledge in distance learning is proposed, along with an assessment of its effectiveness in the educational process. The paper provides an overview of the results of modern test theory application, a description and block diagram of the proposed algorithm, and the results of its application in the real educational process. The effectiveness of using this algorithm for objectively assessing students' knowledge has been demonstrated experimentally.

Downloads

Download data is not yet available.

Abstract views: 274 / PDF views: 60

References

Al-A’ali, M.: IRT-Item Response Theory Assessment for an Adaptive Teaching Assessment System. In: Proceedings of the 10th WSEAS International Conference on Applied Mathematics. p. 518–522. MATH’06, World Scientific and Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA (2006)

Andersen, E.B.: The asymptotic distribution of conditional likelihood ratio tests. Journal of the American Statistical Association 66(335), 630–633 (1971). https://doi.org/10.1080/01621459.1971.10482321 DOI: https://doi.org/10.1080/01621459.1971.10482321

Andersen, E.B.: A goodness of fit test for the Rasch model. Psychometrika 38(1), 123–140 (1973). https://doi.org/10.1007/BF02291180 DOI: https://doi.org/10.1007/BF02291180

Andrich, D.: The Rasch model explained. In: Applied Rasch measurement: A book of exemplars, pp. 27–59. Springer (2005) DOI: https://doi.org/10.1007/1-4020-3076-2_3

Andrich, D.: Rasch Models for Measurement. Thousand Oaks (2021). https://doi.org/10.4135/9781412985598, https://methods.sagepub.com/book/rasch-models-for-measurement

Andrich, D., Sheridan, B., Luo, G.: Rumm2010: Rasch unidimensional measurement models (2001), http://www.rummlab.com.au/

Avanesov, V.S.: The problem of psychological tests. Soviet Education 22(6), 6–23 (1980). https://doi.org/10.2753/RES1060-939322066 DOI: https://doi.org/10.2753/RES1060-939322066

Bezruczko, N. (ed.): Rasch measurement in health sciences. Jam Press Maple Grove, MN (2005)

Birnbaum, A.: Combining independent tests of significance*. Journal of the American Statistical Association 49(267), 559–574 (1954). https://doi.org/10.1080/01621459.1954.10483521 DOI: https://doi.org/10.1080/01621459.1954.10483521

Bond, T., Fox, C.: Applying the Rasch model: Fundamental measurement in the human sciences. Second edn. (2007). https://doi.org/10.4324/9781410614575 DOI: https://doi.org/10.4324/9781410614575

Bond, T., Yan, Z., Heene, M.: Applying the Rasch model: Fundamental measurement in the human sciences. Routledge, fourth edn. (2020) DOI: https://doi.org/10.4324/9780429030499

Brown, W.: Some experimental results in the correlation of mental abilities. British Journal of Psychology, 1904-1920 3(3), 296–322 (1910). https://doi.org/10.1111/j.2044-8295.1910.tb00207.x DOI: https://doi.org/10.1111/j.2044-8295.1910.tb00207.x

Cronbach, L.J.: Coefficient alpha and the internal structure of tests. Psychometrika 16(3), 297–334 (1951). https://doi.org/10.1007/BF02310555 DOI: https://doi.org/10.1007/BF02310555

Eckes, T.: Introduction to Many-Facet Rasch Measurement. Peter Lang, Bern, Switzerland (2011), https://www.peterlang.com/view/title/13347 DOI: https://doi.org/10.3726/978-3-653-04844-5

Fischer, G.H., Molenaar, I.W. (eds.): Rasch models: Foundations, recent developments, and applications. Springer Science & Business Media (1995). https://doi.org/10.1007/978-1-4612-4230-7 DOI: https://doi.org/10.1007/978-1-4612-4230-7

Guilford, J.P.: Fundamental statistics in psychology and education. McGraw-Hill, New York (1942)

Gulliksen, H.: Perspective on Educational Measurement. Applied Psychological Measurement 10(2), 109–132 (1986). https://doi.org/10.1177/014662168601000201 DOI: https://doi.org/10.1177/014662168601000201

Guttman, L.: A basis for scaling qualitative data. American Sociological Review 9, 139–150 (1944). https://doi.org/10.2307/2086306 DOI: https://doi.org/10.2307/2086306

Ingebo, G.S.: Probability in the Measure of Achievement. Mesa Press (1997)

Kim, S.H., Baker, F.B.: birtr: A Package for “The Basics of Item Response Theory Using R”. Applied Psychological Measurement 42(5), 403–404 (2018). https://doi.org/10.1177/0146621617748327 DOI: https://doi.org/10.1177/0146621617748327

Kuder, G.F., Richardson, M.W.: The theory of the estimation of test reliability. Psychometrika 2(3), 151–160 (1937). https://doi.org/10.1007/BF02288391 DOI: https://doi.org/10.1007/BF02288391

Lazarsfeld, P.F.: Regression analysis with dichotomous attributes. Social Science Research 1(1), 25–34 (1972). https://doi.org/10.1016/0049-089X(72)90056-7 DOI: https://doi.org/10.1016/0049-089X(72)90056-7

Linacre, J.M.: Predicting responses from Rasch measures. Journal of Applied Measurement 11(1), 1–10 (2010)

van der Linden, W.J., Hambleton, R.K. (eds.): Handbook of Modern Item Response Theory. Springer Science & Business Media (1997). https://doi.org/10.1007/978-1-4757-2691-6 DOI: https://doi.org/10.1007/978-1-4757-2691-6

Lord, F.M., Novick, M.R., Birnbaum, A.: Statistical theories of mental test scores. Addison-Wesley, Oxford (1968)

Lord, F.M.: Applications of Item Response Theory To Practical Testing Problems. Routledge (1980). https://doi.org/10.4324/9780203056615 DOI: https://doi.org/10.4324/9780203056615

Luce, R.D., Tukey, J.W.: Simultaneous conjoint measurement: A new type of fundamental measurement. Journal of Mathematical Psychology 1(1), 1–27 (1964). https://doi.org/10.1016/0022-2496(64)90015-X DOI: https://doi.org/10.1016/0022-2496(64)90015-X

Maslak, A.A., Karabatsos, G., Anisimova, T.S., Osipov, S.A.: Measuring and comparing higher education quality between countries worldwide. Journal of Applied Measurement 6(4), 432–442 (2005)

Masters, G.N.: Educational measurement: Prospects for research and innovation. The Australian Educational Researcher 15(4), 23–34 (1988). https://doi.org/10.1007/BF03219425 DOI: https://doi.org/10.1007/BF03219425

Perline, R., Wright, B.D., Wainer, H.: The Rasch model as additive conjoint measurement. Applied Psychological Measurement 3(2), 237–255 (1979). https://doi.org/10.1177/014662167900300213 DOI: https://doi.org/10.1177/014662167900300213

Rasch, G.: Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests. Nielsen & Lydiche (1960)

Sax, G.: Principles of educational and psychological measurement and evaluation. Wadsworth Pub. Co., Belmont, 3rd edn. (1989)

Smith, E.V., Smith, R.M. (eds.): Introduction to Rasch measurement: Theory, models and applications. JAM Press (2004)

Spearman, C.: Correlation calculated from faulty data. British Journal of Psychology, 1904-1920 3(3), 271–295 (1910). https://doi.org/10.1111/j.2044-8295.1910.tb00206.x DOI: https://doi.org/10.1111/j.2044-8295.1910.tb00206.x

Weiss, D.J.: Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement 6(4), 473–492 (1982). https://doi.org/10.1177/014662168200600408 DOI: https://doi.org/10.1177/014662168200600408

Weiss, D.J.: Computerized adaptive testing for effective and efficient measurement in counseling and education. Measurement and Evaluation in Counseling and Development 37(2), 70–84 (2004). https://doi.org/10.1080/07481756.2004.11909751 DOI: https://doi.org/10.1080/07481756.2004.11909751

Wilson, M.: Constructing Measures: An Item Response Modeling Approach. Routledge (2005)

Wright, B.D., Linacre, J.M.: Dichotomous rasch model derived from specific objectivity. Rasch measurement transactions 1(1), 5–6 (1987), https://www.rasch.org/rmt/rmt11a.htm

Wright, B.D.: Solving Measurement Problems with the Rasch Model. Journal of Educational Measurement 14(2), 97–116 (1977), http://www.jstor.org/stable/1434010 DOI: https://doi.org/10.1111/j.1745-3984.1977.tb00031.x

Wright, B.D., Masters, G.N.: Rating scale analysis. Mesa Press, Chicago (1982)

Wright, B.D., Stone, M.H.: Best test design. Mesa Press, Chicago (1979), https://www.rasch.org/BTD_RSA/pdf%20[reduced%20size]/Best%20Test%20Design.pdf