Simulation of intellectual system for evaluation of multilevel test tasks on the basis of fuzzy logic

Authors

DOI:

https://doi.org/10.55056/cte.304

Keywords:

intelligent system, multilevel test tasks, fuzzy test characteristics, fuzzy assessment, Sugeno inference system

Abstract

The article describes the stages of modeling an intelligent system for evaluating multilevel test tasks based on fuzzy logic in the MATLAB application package, namely the Fuzzy Logic Toolbox. The analysis of existing approaches to fuzzy assessment of test methods, their advantages and disadvantages is given. The considered methods for assessing students are presented in the general case by two methods: using fuzzy sets and corresponding membership functions; fuzzy estimation method and generalized fuzzy estimation method. In the present work, the Sugeno production model is used as the closest to the natural language. This closeness allows for closer interaction with a subject area expert and build well-understood, easily interpreted inference systems. The structure of a fuzzy system, functions and mechanisms of model building are described. The system is presented in the form of a block diagram of fuzzy logical nodes and consists of four input variables, corresponding to the levels of knowledge assimilation and one initial one. The surface of the response of a fuzzy system reflects the dependence of the final grade on the level of difficulty of the task and the degree of correctness of the task. The structure and functions of the fuzzy system are indicated. The modeled in this way intelligent system for assessing multilevel test tasks based on fuzzy logic makes it possible to take into account the fuzzy characteristics of the test: the level of difficulty of the task, which can be assessed as “easy”, “average", “above average”, “difficult”; the degree of correctness of the task, which can be assessed as “correct”, “partially correct”, “rather correct”, “incorrect”; time allotted for the execution of a test task or test, which can be assessed as “short”, “medium”, “long”, “very long”; the percentage of correctly completed tasks, which can be assessed as “small”, “medium”, “large”, “very large”; the final mark for the test, which can be assessed as “poor”, “satisfactory”, “good”, “excellent”, which are included in the assessment. This approach ensures the maximum consideration of answers to questions of all levels of complexity by formulating a base of inference rules and selection of weighting coefficients when deriving the final estimate. The robustness of the system is achieved by using Gaussian membership functions. The testing of the controller on the test sample brings the functional suitability of the developed model.

Downloads

Download data is not yet available.
Abstract views: 132 / PDF views: 65

References

Anohina-Naumeca, A. and Grundspenkis, J., 2010. Evaluating students’ concept maps in the concept map based intelligent knowledge assessment system. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 5968 LNCS, pp.8–15. Available from: https://doi.org/10.1007/978-3-642-12082-4_2. DOI: https://doi.org/10.1007/978-3-642-12082-4_2

Anohina-Naumeca, A., Strautmane, M. and Grundspenkis, J., 2010. Development of the scoring mechanism for the concept map based intelligent knowledge assessment system. Sofia, vol. 471, pp.376–381. Available from: https://doi.org/10.1145/1839379.1839446. DOI: https://doi.org/10.1145/1839379.1839446

Barker, T., 2010. An automated feedback system based on adaptive testing: Extending the model. International journal of emerging technologies in learning, 5(2), pp.11–14. Available from: https://doi.org/10.3991/ijet.v5i2.1235. DOI: https://doi.org/10.3991/ijet.v5i2.1235

Belov, E.A., 2006. Development of a method and algorithms for testing knowledge based on intelligent processing of answers in natural language. Ph.D. thesis. Bryansk.

Bespalko, V., 1960. Requirements of educational films for professional and technical education. Soviet education, 2(3), pp.17–19. Available from: https://doi.org/10.2753/RES1060-9393020317. DOI: https://doi.org/10.2753/RES1060-9393020317

Buyak, B., Tsidylo, I., Repskyi, V. and Lyalyuk, V., 2018. Stages of conceptualization and formalization in the design of the model of the neuro-fuzzy expert system of professional selection of pupils. Ceur workshop proceedings, 2257, pp.112–121. DOI: https://doi.org/10.31812/123456789/2669

Cherednichenko, O. and Yangolenko, O., 2013. Towards quality monitoring and evaluation methodology: Higher education case-study. Lecture notes in business information processing, 137, pp.120–127. Available from: https://doi.org/10.1007/978-3-642-38370-0_11. DOI: https://doi.org/10.1007/978-3-642-38370-0_11

Clotfelter, C., Ladd, H. and Vigdor, J., 2006. Teacher-student matching and the assessment of teacher effectiveness. Journal of human resources, 41(4), pp.778–820. Available from: https://doi.org/10.3368/jhr.xli.4.778. DOI: https://doi.org/10.3368/jhr.XLI.4.778

Danilova, S.D., 2008. Evaluation of test results in an adaptive automated testing system. Bulletin of vsstu, (1), pp.12–20.

Duplik, S.V., 2004. Model of adaptive testing on fuzzy mathematics. Computer science and education, (11), pp.57–65.

Falchikov, N. and Boud, D., 1989. Student self-assessment in higher education: A metaanalysis. Review of educational research, 59(4), pp.395–430. Available from: https://doi.org/10.3102/00346543059004395. DOI: https://doi.org/10.3102/00346543059004395

Falchikov, N. and Goldfinch, J., 2000. Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of educational research, 70(3), pp.287–322. Available from: https://doi.org/10.3102/00346543070003287. DOI: https://doi.org/10.3102/00346543070003287

Gierłowski, K. and Nowicki, K., 2008. A novel architecture for e-learning knowledge assessment systems. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 4823 LNCS, pp.276–287. Available from: https://doi.org/10.1007/978-3-540-78139-4_25. DOI: https://doi.org/10.1007/978-3-540-78139-4_25

Grundspenkis, J., 2019. Intelligent knowledge assessment systems: Myth or reality. Frontiers in artificial intelligence and applications, 315, pp.31–46. Available from: https://doi.org/10.3233/978-1-61499-941-6-31.

Hameed, I.A. and Sorensen, C.G., 2010. Fuzzy systems in education: A more reliable system for student evaluation. In: A.T. Azar, ed. Fuzzy systems. IntechOpen, pp.1–17. Available from: https://doi.org/10.5772/7216. DOI: https://doi.org/10.5772/7216

He, L. and He, W., 2009. Study on the construction of internal monitoring system of chinese independent colleges’ education quality. Genova, pp.191–194.

Host, M., Regnell, B. and Wohlin, C., 2000. Using students as subjects - a comparative study of students and professionals in lead-time impact assessment. Empirical software engineering, 5(3), pp.201–214. Available from: https://doi.org/10.1023/A:1026586415054. DOI: https://doi.org/10.1023/A:1026586415054

Hwang, G.J. and Chang, H.F., 2011. A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students. Computers and education, 56(4), pp.1023–1031. Available from: https://doi.org/10.1016/j.compedu.2010.12.002. DOI: https://doi.org/10.1016/j.compedu.2010.12.002

Igbape, E. and Idogho, P., 2019. N-dimension data visualization spaces for academic programmes quality monitoring in nigeria higher education. Newswood Limited, vol. 2019-October, pp.238–242.

Khvostina, I., Oliinyk, V., Semerikov, S., Solovieva, V., Yatsenko, V. and Kohut-Ferens, O., 2021. Hazards and risks in assessing the impact of oil and gas companies on the environment. Iop conference series: Earth and environmental science, 628(1), p.012027. Available from: https://doi.org/10.1088/1755-1315/628/1/012027. DOI: https://doi.org/10.1088/1755-1315/628/1/012027

Leontev, M., Bondarenko, N., Shebzuhova, T., Butko, S. and Egorova, L., 2018. Improving the efficiency of university management: Teacher’s performance monitoring as a tool to promote the quality of education. European research studies journal, 21(2), pp.527–540. Available from: https://doi.org/10.35808/ersj/1020. DOI: https://doi.org/10.35808/ersj/1020

Li, Y., Li, P., Zhu, F. and Wang, R., 2017. Design of higher education quality monitoring and evaluation platform based on big data. Institute of Electrical and Electronics Engineers Inc., pp.337–342. Available from: https://doi.org/10.1109/ICCSE.2017.8085513. DOI: https://doi.org/10.1109/ICCSE.2017.8085513

Linn, R., 1998. Validating inferences from national assessment of educational progress achievement-level reporting. Applied measurement in education, 11(1), pp.23–47. Available from: https://doi.org/10.1207/s15324818ame1101_2. DOI: https://doi.org/10.1207/s15324818ame1101_2

Lutsyk, I., Franko, Y., Rak, V., Lutsyk, I., Leshchii, R. and Potapchuk, O., 2019. Mathematical modeling of energy-efficient active ventilation modes of granary. 2019 9th international conference on advanced computer information technologies (acit). pp.105–108. Available from: https://doi.org/10.1109/ACITT.2019.8780109. DOI: https://doi.org/10.1109/ACITT.2019.8780109

Muhd Nor, N., Azlan, M., Kiong, S., Mohamad, F., Ismail, A., Kasmin, A., Ahmad, M. and Yokoyama, S., 2014. Development of course management and monitoring system as a quality tools in engineering education. Applied mechanics and materials, 465-466, pp.395–400. Available from: https://doi.org/10.4028/www.scientific.net/AMM.465-466.395. DOI: https://doi.org/10.4028/www.scientific.net/AMM.465-466.395

Nechypurenko, P. and Semerikov, S., 2017. VlabEmbed - the new plugin Moodle for the chemistry education. Ceur workshop proceedings, 1844, pp.319–326. DOI: https://doi.org/10.31812/0564/731

Newble, D. and Jaeger, K., 1983. The effect of assessments and examinations on the learning of medical students. Medical education, 17(3), pp.165–171. Available from: https://doi.org/10.1111/j.1365-2923.1983.tb00657.x. DOI: https://doi.org/10.1111/j.1365-2923.1983.tb00657.x

Osadchyi, V.V., Osadcha, K.P., Varina, H.B., Shevchenko, S.V. and Bulakh, I.S., 2021. Specific features of the use of augmented reality technologies in the process of the development of cognitive component of future professionals’ mental capacity. Journal of physics: Conference series. DOI: https://doi.org/10.1088/1742-6596/1946/1/012022

Phankokkruad, M. and Woraratpanya, K., 2008. An automated decision system for computer adaptive testing using genetic algorithms. Phuket, pp.655–660. Available from: https://doi.org/10.1109/SNPD.2008.118. DOI: https://doi.org/10.1109/SNPD.2008.118

Qin, F., Zeng, W., Li, L. and Zhao, R., 2020. Construction of big data monitoring platform for teaching quality under intelligent education. Institute of Electrical and Electronics Engineers Inc., pp.1594–1597. Available from: https://doi.org/10.1109/IWCMC48107.2020.9148224. DOI: https://doi.org/10.1109/IWCMC48107.2020.9148224

Rotshtein, A. and Shtovba, S., 1998. Predicting the reliability of algorithmic processes with fuzzy input data. Cybernetics and systems analysis, 34(4), pp.545–552. Available from: https://doi.org/10.1007/BF02666999. DOI: https://doi.org/10.1007/BF02666999

Rotshtein, A. and Shtovba, S., 2001. Fuzzy multicriteria analysis of variants with the use of paired comparisons. Journal of computer and systems sciences international, 40(3), pp.499–503.

Rotshtein, A. and Shtovba, S., 2009. Modeling of the human operator reliability with the aid of the Sugeno fuzzy knowledge base. Automation and remote control, 70(1), pp.163–169. Available from: https://doi.org/10.1134/S0005117909010123. DOI: https://doi.org/10.1134/S0005117909010123

Rotshtein, A., Shtovba, S. and Mostav, I., 2001. Fuzzy rule based innovation projects estimation. Vancouver, BC, vol. 1, pp.122–126.

Rotshteina, A. and Shtovbab, S., 2006. Identification of a nonlinear dependence by a fuzzy knowledgebase in the case of a fuzzy training set. Cybernetics and systems analysis, 42(2), pp.176–182. Available from: https://doi.org/10.1007/s10559-006-0051-1. DOI: https://doi.org/10.1007/s10559-006-0051-1

Rotshtejn, A. and Shtovba, S., 2001. Fuzzy rule based control of a dynamic system. Avtomatika i vychislitel’naya tekhnika, (2), pp.23–31.

Rotshtejn, A. and Shtovba, S., 2002. Influence of methods of defuzzification on speed of tuning the fuzzy model. Kibernetika i sistemnyj analiz, (5), pp.169–176.

Rudinskiy, I.D., 2007. Fuzzy knowledge evaluation model as a methodological basis for automation of pedagogical testing. Ieee transactions on education, 50(1), pp.68–73. Available from: https://doi.org/10.1109/TE.2006.888904. DOI: https://doi.org/10.1109/TE.2006.888904

Rust, C., Price, M. and Berry, O., 2003. Improving students’ learning by developing their understanding of assessment criteria and processes. Assessment and evaluation in higher education, 28(2), pp.147–164. Available from: https://doi.org/10.1080/02602930301671. DOI: https://doi.org/10.1080/02602930301671

Rótshtein, A. and Stovba, S., 2001. Managing a dynamic system by means of a fuzzy knowledge base. Automatic control and computer sciences, 35(2), pp.16–22.

Schmuck, B., Sima, D. and Szöllosi, S., 2006. The design space of the implementation of knowledge assessment systems. Sydney, NSW, pp.587–593. Available from: https://doi.org/10.1109/ITHET.2006.339672. DOI: https://doi.org/10.1109/ITHET.2006.339672

Scouller, K., 1998. The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher education, 35(4), pp.453–472. Available from: https://doi.org/10.1023/A:1003196224280. DOI: https://doi.org/10.1023/A:1003196224280

Shtovba, S., 2006. Fuzzy identification on the basis of regression models of parametric membership function. Journal of automation and information sciences, 38(11), pp.36–44. Available from: https://doi.org/10.1615/JAutomatInfScien.v38.i11.40.

Shtovba, S., 2007. Ensuring accuracy and transparency of Mamdani fuzzy model in learning by experimental data. Journal of automation and information sciences, 39(8), pp.39–52. Available from: https://doi.org/10.1615/JAutomatInfScien.v39.i8.50. DOI: https://doi.org/10.1615/JAutomatInfScien.v39.i8.50

Shtovba, S., 2007. Fuzzy model tuning based on a training set with fuzzy model output values. Cybernetics and systems analysis, 43(3), pp.334–340. Available from: https://doi.org/10.1007/s10559-007-0054-6. DOI: https://doi.org/10.1007/s10559-007-0054-6

Shtovba, S., Pankevich, O. and Nagorna, A., 2015. Analyzing the criteria for fuzzy classifier learning. Automatic control and computer sciences, 49(3), pp.123–132. Available from: https://doi.org/10.3103/S0146411615030098. DOI: https://doi.org/10.3103/S0146411615030098

Shtovba, S. and Pankevych, O., 2018. Fuzzy technology-based cause detection of structural cracks of stone buildings. CEUR-WS, vol. 2105, pp.209–218.

Shtovba, S. and Shtovba, E., 2006. Prediction of competitive position of brand product by fuzzy knowledge base. Journal of automation and information sciences, 38(8), pp.69–80. Available from: https://doi.org/10.1615/JAutomatInfScien.v38.i8.70.

Sorour, A., Atkins, A., Stanier, C. and Alharbi, F., 2020. Comparative frameworks for monitoring quality assurance in higher education institutions using business intelligence. Institute of Electrical and Electronics Engineers Inc. Available from: https://doi.org/10.1109/ICCIT-144147971.2020.9213808. DOI: https://doi.org/10.1109/ICCIT-144147971.2020.9213808

Szöllosi, S., Sima, D. and Schmuck, B., 2006. The design space of the services of knowledge assessment systems. Sydney, NSW, pp.571–578. Available from: https://doi.org/10.1109/ITHET.2006.339816. DOI: https://doi.org/10.1109/ITHET.2006.339816

Taylor, A., 2017. Fuzzy logic with matlab: Analyzing, designing, and simulating systems. CreateSpace Independent Publishing Platform.

Tereshchuk, H.V. and Tsidylo, I.M., 2018. Automated system of fuzzy identification of expert’s competence for assessing the quality of pedagogical phenomena and processes. Information technologies and learning tools, 64(2), pp.234–244. Available from: https://doi.org/10.33407/itlt.v64i2.2079. DOI: https://doi.org/10.33407/itlt.v64i2.2079

Topping, K., 1998. Peer assessment between students in colleges and universities. Review of educational research, 68(3), pp.249–276. Available from: https://doi.org/10.3102/00346543068003249. DOI: https://doi.org/10.3102/00346543068003249

Wei, C., 2013. Higher vocational education quality monitoring system. Lecture notes in electrical engineering, 217 LNEE(VOL. 2), pp.55–60. Available from: https://doi.org/10.1007/978-1-4471-4850-0_8. DOI: https://doi.org/10.1007/978-1-4471-4850-0_8

Wiliam, D., Lee, C., Harrison, C. and Black, P., 2004. Teachers developing assessment for learning: Impact on student achievement. Assessment in education: Principles, policy and practice, 11(1), pp.49–65. Available from: https://doi.org/10.1080/0969594042000208994. DOI: https://doi.org/10.1080/0969594042000208994

Zhi, Z. and Nan, Z., 2011. Study on the construction of teaching quality monitoring system for the undergraduate physical education majors. Jilin, pp.758–761. Available from: https://doi.org/10.1109/HHBE.2011.6028937. DOI: https://doi.org/10.1109/HHBE.2011.6028937

Downloads

Published

2021-03-19

Issue

Section

Educational Data Mining and Social Analytics in Education

How to Cite

Tsidylo, I.M., Semerikov, S.O., Gargula, T.I., Solonetska, H.V., Zamora, Y.P. and Pikilnyak, A.V., 2021. Simulation of intellectual system for evaluation of multilevel test tasks on the basis of fuzzy logic. CTE Workshop Proceedings [Online], 8, pp.507–520. Available from: https://doi.org/10.55056/cte.304 [Accessed 14 June 2024].
Received 2020-10-30
Accepted 2020-12-18
Published 2021-03-19

Similar Articles

1-10 of 225

You may also start an advanced similarity search for this article.

Most read articles by the same author(s)

1 2 3 > >>