Methodological quality and reporting practices in gamification research for history education: a critical methodological analysis

Анотація

Gamification has attracted growing research attention as an instructional strategy for history education, yet the methodological rigour of this evidence base has not been systematically evaluated. This study presents a secondary methodological analysis of 74 empirical studies included in a recent systematic review of gamification in history education [15], published between 2004 and 2024 across 27 countries. We introduce a ten-item Methodological Reporting Quality Score (MRQS) to assess the completeness and rigour of reporting practices, and we re-analyse risk-of-bias assessments across three adapted ROBIS domains. The mean MRQS was 4.31 out of 10 (SD = 1.93, median = 4.0), indicating substantial gaps in methodological reporting. The most frequently satisfied criteria were the presence of a theoretical framework (94.6%) and explicit reporting of sample size (83.8%), while the least satisfied were reporting of effect sizes (9.5%), provision of an explicit definition of gamification or game-based learning (14.9%), and use of validated measurement instruments (21.6%). Risk-of-bias analysis confirmed that 83.8% of studies received a high overall rating, with Domain 1 (sampling and selection, 85.1% high) representing the most persistent source of weakness - no study received a favourable rating on sample representativeness - followed by Domain 2 (data collection and analysis, 63.5% high) and Domain 3 (interpretation and reporting, 55.4% high). Studies providing explicit gamification definitions scored higher on the MRQS (5.36 vs. 4.13, p = .065) but showed equivalent risk-of-bias ratings, suggesting that definitional clarity functions as a marker of reporting conscientiousness rather than of design rigour. Comparison of GPT-4o's automated risk-of-bias assessments against human-verified judgments revealed only 37.6% overall agreement (Cohen's κ = -0.04 to 0.21), with the model consistently applying less stringent criteria than human reviewers, while factual metadata extraction across multiple LLMs achieved 82-97% pairwise agreement. These findings indicate that the current evidence base for gamification in history education suffers from pervasive methodological shortcomings that limit the strength of causal inferences and preclude meta-analytic synthesis. We propose the MRQS as a replicable, domain-adapted instrument for assessing reporting quality in educational technology research.