Gamification implementation for educational purposes: a scoping review (2013-2018)

. This review paper examines a worldwide phenomenon, gamification, in the educational context. After the high popularity of video games, the integration of gamification in the learning context has become one of the practices that have a considerable impact on learning. It is important to contribute to the literature on the educational value of gamification by providing a literature review of some aspects of the studies that have emerged in the field of educational gamification (EG) over a five-year period. It is crucial to have a more complete understanding of educational gamification without restricting emerged literature to defined criteria so that it can be discussed in relation to how much of an impact it has on learning outcomes. The current study systematically reviewed four well-known social science databases for studies on gamification in education published between January 2013 and April 2018. The study yielded 141 relevant papers that focused on only educational gamification concept and excluded other game related approaches. The findings indicate that gathering data regarding learner opinions on the application of gamification is the most frequent ( N =34) study goal of these studies. Although the majority of papers highlight that gamification increases learner motivation and engagement, there is not yet enough data in this large body of research to conclusively state that using gamification would enhance learning and academic performance at the same level in every context. The findings imply that careful design based on instructional goals and context, as well as the needs of the students, is necessary for integration of gamification to serve its purpose effectively. To sum up, ensuring the description of the most accurate implementation process for specific education contexts and subject fields may still be challenging, given the use of gamification in many distinct disciplines. Furthermore, educational gamification is studied in many contexts with various expectations, and the outcomes would inevitably be different. This review categorized the numerous educational advantages and challenges of gamification.

Today, because so much of done tasks and jobs are intervened by digital tools and social media, that is, we play games everywhere now essentially to generate gratifying experiences [89]. Companies and countries have transformed traditional processes into game-like experiences to engage customers and employees more, and gamification is the term used frequently to describe this process [89]. Most worldwide known brands have launched loyalty programs embedded in their mobile apps (or web pages) that include game elements (points, levels.). This gamification included marketing strategies is dynamic and always interchanging another new one. Gartner [41] imply that because of business processes have been inappropriately gamified, nearly 80% of currently available gamified applications will fall short of meeting business goals. In this sense, might improper or excessive use of gamification in education have the same effect?
As for educators, the remarkable and compelling properties of video games have promoted the idea of incorporating game-like elements into instructional materials and learning processes [31,33,77]. López-Belmonte et al. [73] examined the scientific mapping of gamification in the Web of Science using 1230 documents. They found that the topics studied have progressed from the effect of classic games on learning to video games and their impact on performance. Similarly, Martinez, Gimenes and Lambert [77] stated that entertainment video games inclusion in academic curricula has positive effects on cognition and learning.
Gamification has proven to be an effective educational method to increase motivation and encourage participation [88]. In their evaluation of empirical studies, Dichev and Dicheva [33] found a lack of sufficient evidence for the long-term benefits of gamification in education. Moreover, Sailer and Homner [92] stated that, even in 2019, the factors that contribute to the implementation of gamification to improve learning have not been fully identified. However, gamification implementation in the classroom is needed more research to know better its scope and extent. This review's objective is to determine without placing research-type-related limitations on the papers whether these EG uncertainties exist in EG applications for the chosen time frame. Also, adding to gamification implementation body of information will also help clarify these uncertainties. Moreover, to recognize routes and time-frame developments in this expanding EG field between 2013 and 2018 years. The purpose of this review evaluation is to determine how studies conducted between 2013 and 2018 influenced the evolving field of research on gamification in education.

Definition of gamification
Gamification is initially described as the integration of game-like elements in non-game contexts to engage or motivate computer users with applications [31,37,48]. By rewarding preferred behaviors of consumers and employees, gamification can bring about the intended behavior change in an organization, leading to more gratifying outcomes for both parties than in a nongamified environment [89]. This coincides well with the approach that gamification improves people's experience with a service or system by incorporating game-like experiences into the service or application [65]. It has been studied in various fields including: Education, Business, Marketing and Medicine, etc. [79]. Some business and marketing professionals have been inspired by gamification [30]. The positive results of its use for marketing and the increasing popularity of video games have led it to become more well-known in various sectors. Foursquare, for example, a popular search and discovery app, uses gamification to reward visitors when they check in to a location on the app. This increases user engagement with the app. Another example that demonstrates the engaging aspect of gamification is FoldIt. By using game design elements such as points and leaderboards, high user activity was achieved. The gamified online puzzle interface allowed players to "solve" protein folding puzzles. As a result, a long-standing scientific problem, the discovery of a complex HIV protein structure, was made by players [25].
Werbach and Hunter [111] divided these elements into three categories: Dynamics, Mechanics, and Components. The categories are ordered from more abstract to less abstract. Each component in the framework causes a mechanics; each mechanic in the framework causes a dynamic. Dynamics are abstract aspects of the gamified system. It shapes the big-picture of gamification but cannot directly enter into the system [111]. Secondly, mechanics are the elements that drive user engagement to the system and operate the process. Finally, components are less abstract than the first two categories and refer to the real-world tools that integrate gamification into the actual contexts.  Therefore, this organizational scheme of gamification is used for this systematic review study. Although other game-centric approaches share similarities with gamification in terms of their intended use in education, they have different technological and pedagogical formats. These are detailed in figure 2.
In addition, gamification is not the same as these similar concepts, gamification also is not the game itself, but is reminiscent of a game in that it contains game-like elements. While gamification includes the style of game design, game elements and serious purpose, it does not include the complete game environment and fun purpose [28]. More explicitly, this current paper focuses only on the gamification concept and excludes the other game-centric approaches.
Gamification can be referred to as the inclusion of individuals' intrinsic game experience in the learning process [96]. Gamification can promote education through some game design elements. Gamification has been seen as a valuable, useful tool especially to entice learner engagement in educational environments [12,93]. Gamification is being used increasingly in educational settings to boost student motivation and, as a result, learning results [67].

Earlier review studies
Previous literature review studies on gamification [19,28,33,34,71,93,94] have included specific studies due to their research purpose. For example, while Dichev and Dicheva [33] included only empirical studies, Looyestyn et al. [71] examined the effectiveness of gamification to increase engagement in online programs. Also, Saleem, Noori and Ozdamli [93] restricted their review's included papers by only gamification in online education (e-learning). They stated that gamification has being accepted day by day and forms more engaging educational environments. Alomari, Al-Samarraie and Yousef [4] limited their review to presenting empirical findings in higher education. Only papers in the Web of Science database were chosen by López-Belmonte et al. [73] for their mapping of gamification study.
Hung [50] gave a brief overview of gamification in higher education by looking at the results of empirical studies. Some other reviews only discussed the pitfalls or failures of gamification [19,50]. Similarly, some reviews focused on the differences or similarities between gamification and other game-centric concepts [56,74]. Sardi, Idri and Fernández-Alemán [94], on the other hand, included the studies that examine gamification only in the eHealth domain. Darejeh and Salim [28] included only the software environment in their review. Moreover, Xu [115] investigated the gamification of websites and web applications. Hamari, Koivisto and Sarsa [46] conducted a systematic review by examining only empirical studies on gamification to evaluate the effectiveness of it on the gamified systems. The study examined only psychological/behavioral outcomes of gamification, the contexts of gamification, and the types of studies conducted on the gamified systems. Similarly, another study reported the use of gamification, which included interactive systems and human-computer studies [95]. Furthermore, the conceptualization of the term gamification is becoming extensively defined over time, and in parallel, the use of it and the expectations from its implementation are increasing.
It is crucial to have a complete understanding of EG without restricting papers to defined criteria so that it can be discussed in relation to how much of an impact it has on learning outcomes. All gamification implementations to any instructional processes (cognitive, affective, and psychomotor) in a broad time frame from 2013 to April 2018 are within the scope of this synthesis. Because of the popular use of online gamification tools and implementations in courses already has entered most classrooms and informal learning processes around the world. This review attempts to highlight research trends on EG by being broad and not just focusing on one particular field of education, because its use in certain fields can be useful or insufficient. Kitchenham [59,p. iv] defines the technique of systematic literature review as "identifying, evaluating, and interpreting all available research relevant to a particular research question, topic, or phenomenon of interest". This review paper, which aims to include all subject fields, conducted to identify the trends of gamification used for educational purposes and investigated in educational research, aims to include all educational fields. It classified the included studies according to the research questions (RQ) that guided the current study and were listed in table 1. To identify EG researches were done with how many samples • What were the educational levels of the samples?

Research questions and search strategy
To identify the educational level of the samples in EG researches • What subject fields were researched on EG?
Identify the subject fields in which EG have been researched.

How is gamification used in educational contexts?
• Which game elements (dynamics, mechanics & components) have been implemented and tested?
To identify how many different elements are used and the frequency distribution of these elements. • What digital platforms were used in the research of EG?
To identify ICT-based applications or methods or tools have been used in studies of EG.

What are the main findings of EG research?
• What are the advantages of gamification in education according to the studies reviewed?
Identify the reported benefits of gamification for learning and educational outcomes.
• What are the challenges of gamification in education according to the studies reviewed?
To explore possible limitations, unexpected outcomes, negative educational outcomes, disappointments of gamification use.
The following part explains the article selection procedure.

Inclusion criteria
For this study, the keywords were searched in the best-known digital research paper databases: Web of Science, ERIC, EBSCO and Taylor & Francis. The keywords searched were "gami*", "gamification", "education AND gamification", "gamification in education" and "gami* AND education" to include as many eligible studies as possible. All listed papers in the databases were downloaded in case the full text was available. Attempts were made to access some articles through Google Scholar when they were not directly accessible through the databases. Duplicates were removed. Studies were included if they matched one of the following criteria: (1) they included gamification solutions for educational purposes in any context or setting; (2) they had gamification tools, applications, and websites in an educational context; or (3) they contained gamification techniques or game design elements for educational reasons.
Studies implicated gamification were considered in this study even if it was only one dimension of the article, especially for review papers. Next, the papers were examined using the definition of gamification ("the use of game design elements in non-gaming contexts" [31]) and those that did not meet this definition were excluded.

Exclusion criteria
The exclusion procedure was performed by reading the entire paper. The studies were excluded has been shown in figure 3.
(1) if the studies were games, serious games, video games, game-based learning, gamified simulation-based learning and simulations (n=70).
(2) if the studies are not written in English or Turkish (n=32).
(3) if the studies do not investigate gamification or are irrelevant to gamification (n=71) (4) if the studies do not deal with education (n=19) (5) if the studies were extensive reviews that did not meet the review criteria (n=13) In addition, some extensive reviews were excluded because they were so comprehensive and highly detailed that they could dominate the relevant dataset for the RQ.

Extraction strategy of papers and synthesis method
Data extraction was conducted using a scanning tool (appendix A). This standardized form was structured specifically for this review. The tool was filled in for the each of 141 selected papers and consists of ten sections. All sections could not always be filled in for all papers as the study may not have the relevant data for the section due to its methodology. Therefore, the missing data are collected under the "not reported" row for each RQs. It is important to explain the distribution frequencies, how many different variables are considered for each RQs.

Trustworthiness of the study
Systematic review seeks to review and synthesize existing research to answer specific research/review questions [5]. Because the synthesis and review of articles in this study is a qualitative research process, a qualitative approach was opted. One of the criticisms of qualitative research in terms of trustworthy elements is often based on the low quality of documentation and reporting of results [6]. There are some suggestions to assure the trustworthiness of qualitative studies. Guba [45], Lincoln and Guba [68] construct four criteria which are credibility, transferability, dependability and confirmability. In addition, Eisenhart and Howe [35] define some critical stages. First, there must be consistency between research questions and data collection and analysis techniques. Second, appropriate data collection and analysis methods must be used. Third, credible findings must be obtained by comparison with the relevant literature. In order to obtain valuable results that can be used in practice, this study presents the results in detailed data segments with the aim of showing the scope of the data. Finally, consistency in the criteria for qualitative research is required.
In the current study, to meet the need for credibility, frequent debriefing was conducted between the researcher and the peer reviewer, who is a doctoral candidate in the Department of Computer Education and Instructional Design department in the USA. The quartile of the selected studies was randomly determined and analyzed by the peer reviewer to verify if the data extracted from the articles were close to the researcher's initial extraction. Importantly, the researcher's 'reflective commentary' was addressed at every stage of the study, particularly during data collection. In addition, frequencies for 141 completed forms for each of the included studies were recorded individually by the researcher to determine if there were any missing data or an error. While reporting the findings of the study, previous research findings were detailed.
The contextual factors and sampling method must be clearly defined to meet the requirement of transferability [97]. On the other hand, the criterion of dependability is about the fact that the process by which the results are obtained should be clear and preferably repeatable [6]. To meet the criterion of dependability, all research processes and data sets were recorded regularly and clearly. An extraction document was completed separately for each of the 141 articles included. They are saved as word processor documents. The categorical data were summarized in an Excel document using the data from the extraction documents and presented in the finding section.

Findings
This section describes the results of the systematic literature review. Assorted data were compiled to answer 3 main research questions. Similar statements in the data for each sub-RQ were grouped and combined into one category. RQ1 categorized 6 publication years, 13 research methods, 10 research purposes, 13 groups of research sample size, 10 educational levels of the samples, 25 subject areas. RQ2 categorized 37 game elements, 19 ICT platforms; RQ3 identified 13 advantages and 13 challenges.
The systematic selection of papers is shown in figure 4. After eliminating duplicates in 423 papers from 4 databases, 346 papers remained. 205 papers were excluded according to the exclusion criteria listed in the methodology section. Each of the selected papers is numbered in ascending order. The numbered papers between {1-14} were retrieved from EBSCO, between {15-64} from ERIC, between {65-132} from Web of Science, and finally between {133-141} from Taylor & Francis. The articles and formed data have been stored in a 442 MB file.

RQ1: What are the research trends in gamification in education studies? What is the distribution of EG studies by year?
When the included articles were listed by years of publication, the frequency of publication by year is shown in figure 5. Although the number of studies is gradually increasing, only 8 studies from the beginning of 2018 were included, as the selection process was conducted in March 2018. It can be inferred that the interest in EG has been increasing year by year and it seems that it will continue to increase [34].
a. What research methods were conducted for EG? Table 2 shows 13 research methods that were identified. A few studies (Chujitarom and Piriyasurawong [22], Jensen [54]) did not aim to investigate gamification itself, but gamification was incorporated in the studies and made an educational contribution to the results. Hence, the related parts of these studies are considered in this review.
Forty-one of the studies were case studies. Quasi-experimental studies (24) and experimental studies (22) were almost half as many. Banfield and Wilkerson [9] chose loose interviewing, a direct observation, and the researcher was embedded into observed class. Action research papers examined how gamification contributes to learning or teaching [26,98,113]. For example, Cruaud [27] conducted an action research by emphasizing learner-centeredness, student collaboration and the teacher as facilitator. Two exploratory research studies were conducted as pilots to investigate preliminary findings about contexts within which gamified learning activities work effectively [14] and research on the use of gamification in school settings for foreign language learning [42]. One of two Delphi method studies was analyzed key e-learning trends [72] and the other tried to reach a consensus on what tools and information should be brought together in a patient-centric app [110].

b. What were the purposes of the included EG articles?
The research purposes of the included studies were examined and categorized in table 3. The objectives of the 141 articles were divided into 10 different categories with a total of 176 research objectives (table 3). The purposes of some articles are represented in more than one Table 2 Number of articles by research methods.

No
Research methods Frequency   1  Case study  41  2  Quasi-experimental  24  3  Experimental  22  4 Review 21 5 Survey methodology 12 6 Mixed method (experimental & qualitative) 6 7 Qualitative research 4 8 Action research 4 9 Exploratory research 2 10 Delphi-based approach 2 11 Design-based research 1 12 Design science research 1 13 Secondary data analysis research 1 Total 141 Table 3 Number of included articles by research purpose. 1  perceptions of samples  34  2  gamification implementation  32  3 impact of gamification on learning related variables 29 4 effectiveness of game elements 27 5 impact of gamification on learner motivation 14 6 contribution to instruction 14 7 reviews and conceptualization of gamification 12 8 redesigning content with gamification 7 9

No Research purpose of article Frequency
possible dilemmas of gamification 6 10 support for exam preparation 1 category because they have more than one research purpose. For example, Moritz's [80, p. 5] dissertation examined how gamification increases student motivation and engagement with course content. It was therefore represented in both the fourth and fifth categories, as were several other studies. The articles that did not examine the effect of gamification on a particular variable and that examined the effectiveness of gamification or game elements were included under the third category, "effectiveness of game elements", which was addressed 27 times. The selected articles aimed 32 times to investigate how to gamify a classroom, an educational system or a course that represented under the name of "gamification implementation". The other aim named "sample perceptions", which is repeated 34 times, is to explore students' current knowledge, attitudes toward, reception of, or experiences with gamification. Another purpose recurring 27 times, "impact of gamification on learning related variables" is how gamification is used to increase student engagement or achievement or participation or satisfaction. For example, Walsh [108] examined the gamification potential on student engagement and achievement. In the category "impact of gamification on learner motivation," 14 articles examined how gamification increases learner motivation. For example, Banfield and Wilkerson [9] sought to test usefulness of gamification to increase intrinsic motivation and self-efficacy in students. In the "contribution to instruction" category, another 14 articles highlight how gamification contributes to teaching and/or learning or increases awareness". One of these papers investigated the way in which Minecraft Edu as a gamification tool can contribute to the teaching and learning of secondary students [17].
In the category "reviews and conceptualization of gamification", 12 articles aimed to define or conceptualize gamification or a game element or gaming resources and toolkits, or provided an overview of gamification. The category "redesigning with gamification" includes seven articles that explore the process of implementing gamification into an existing educational content. Six different articles discuss common obstacles, challenges, and/or pitfalls in implementing gamification. One article looked at ways to better prepare high school students for the demands of college decision research.

c. What were the samples sizes of the included EG articles?
The studies involve a total of nearly 16 231 participants, not including the 18000 participants in a study by Simões, Redondo and Vilas [98].
As shown in figure 6, 32 articles did not report a sample size and were presented as "not reported". This category includes 20 review articles and several other nonempirical studies. The sample sizes of the articles ranged from five to 18000 samples. 18 of the articles studied with a sample size between 21 and 40. The sample size of 12 articles is up to 20 samples. d. What was the educational level of the samples? Table 4 shows that more than half of the papers were researched with undergraduate students. 10 other included studies reported outcomes with high school students. Eight studies reported outcomes for in-service training. This segment included in-service teacher education [69,106].
Dichev and Dicheva [33] noted that studies conducted on more than one demographic group are beneficial because the result of the study conducted on one demographic group cannot be generalized to other situations. This is also acceptable for the present study. To illustrate this advantage, these types of studies were presented separately. For example, adults and middle school students were studied consecutively in two different applications [7].

e. Which subject fields were researched on gamification in education?
The included articles cover a wide range of academic subjects, from engineering to library use. Table 5 lists 25 different subject groups. Some closely related subject areas have been grouped together.
The largest number of studies dealt with ICT and programming fields. This category included a range of courses such as computer science and programming basics, instructional design, and introduction to information literacy, etc. The second highest percentage (14 percent) addressed implementation of gamification in educational sciences departments. This category includes the use of gamification and the design of gamified learning activities in education departments to train teacher candidates about EG or to teach them by implementing gamification in the course.
In the use of gamification in language teaching, ICT platforms were mainly used to achieve the teaching objectives. 11 studies covered the ways of technical implementation of game design techniques; other 10 studies dealt with the implementation of gamification in pedagogical technology courses in teacher education. Medical education (10 studies) has made remarkable progress in the use of gamification in medical-related courses. Gamification provides a safe condition for failure for medical students [113]. 6 of these studies found gamification to be a useful method for increasing student learning [20,63,81,107,113]. Nearly seven studies each explored the fields of mathematics education, nursing education, business and management, psychology education, e-health or health and science teaching. Other 12 subject fields also are listed in the table 5. 4 studies are represented in more than one academic subject [33,34,71,99]. In addition, some review studies consisted of more than one study subject, so they were distributed to related categories. Experimental and quasi-experimental studies mostly examined three purposes. The first is the effectiveness of gamification elements or techniques. The other is to examine students' perceptions, attitudes, or experiences with gamification. The last is how gamification increases student engagement, achievement, participation, or satisfaction. Ten of the reviews examine the definition of gamification or its elements. Survey methodology mostly aims to examine students' perceptions, attitudes, current knowledge of, acceptance of gamification or experiences with gamification [11,18,108].

a. Which game elements (dynamics, mechanics & components) were used for gamification implementation?
It is not intended to list name by name all the gamification elements mentioned in the 141 articles because there are so many variations. On the plus side, there are differences in the classification of gamification elements in the literature. Elements are categorized according to Werbach and Hunter [111] classification. The frequency of detected gamification elements from the included articles in these three categories was presented separately for this review in table 6 (dynamics), table 7 (mechanics), and table 8 (components). Some studies examined the combined impact of multiple gamification features. Thus, some article codes are present in more than one category. Table 6 Frequency of studied dynamics in the included articles.

No
Name of dynamics Frequency Codes of articles   The most common dynamics features evaluated were limited time / resources / constraints. It is not necessary to use dynamics in all gamification. Dynamics are big-picture aspect of the gamification process, so there may not always be dynamics in a process where mechanics and components are contained for gamification. In the current review, the dynamics used were limitations (9), narratives (7), storylines (5), progression (5) and emotions (1) and were listed in table 7. 11 game mechanics categories were listed in table 7. The first three most frequently used mechanics were rewards and their equivalents, then feedback and challenges. 30 out of 141 papers include reward/gift/share/reward/reward mechanics to implement gamification. Feedback and challenges are other two popular mechanics that took place to implement gamification. In their qualitative study, Aldemir, Celik and Kaplan [3] found that challenge was recognised by all participants as an important concept for gamification. Children who participated in interactive social games research by Inal and Cagiltay [52] also deemed challenge to be the most crucial component.
According to table 8, besides dynamics and mechanics, the researchers mainly used components feature for gamification implementation. 21 different components were assessed. Points and badges are awarded for the gamification mechanic, reward. Points, badges and leaderboards were most commonly used as components in EG (table 8). Points are an essential component of Table 9 Distribution of the number of articles according to ICT platforms.

No
Name of ICT platform Frequency Codes of articles gamification [34,94]. Badges / symbol / star / trophies / medals and leaderboards are among the most used components, they come after points.

b. What digital platforms were used in EG research?
The platforms presented in table 9 are ICT software or other digital tools, more than one of which was observed in the selected papers. 36 of the 141 selected studies did not include ICT platforms and represented as "not reported" as shown in table 9. Another 30 articles employed an app, a "research-specific app" that was not accessible via the Internet search and was a mobile or desktop application developed or revised specifically for the study that could not be recommended for other situations. For example, Mallon [76] only listed some platforms that could be used by librarians. LMS platforms, with a frequency of 25, were generally used in engineering / IT and educational technology courses. Kahoot, one of the contemporary gamification platforms, was used in engineering / IT and language education in five research papers.
Importantly, most articles did not use a digital platform (category "not reported" in table 9) to implement gamification. This could reinforce the idea that the use of an ICT platform is not essential for the implementation of gamification. It can also be seen that gamification is usually achieved by incorporating one or more game elements into the lesson.

RQ3: What are the major findings of EG research in education?
a. What are the advantages/benefits of EG? Table 10 summarizes the various benefits that emerged from the studies evaluated. As shown in table 10, the most frequently cited benefits of EG are increased student achievement, goal attainment, and increased engagement and motivation. Thus, first, 34 percent of the included articles, named as "useful strategy for learning", implies that gamification is a strategy that helps increase learning and/or to achieve intended goals. Secondly, gamification was found to increase student motivation for content a little or too much in 41 different articles. Papadakis, Zourmpakis and Kalogiannakis [84] added similar promising results to this statement through their research in science education. Third, in 36 studies, the implementation of gamification made the course fun and engaged students in the course; in 20 studies, it is found that the implementation of gamification promoted enthusiasm for the content and made students feel excited about the content. In 14 articles, it is clearly reported that gamification aroused positive emotions in participants and created equal opportunities. In 14 articles, it is clearly reported that gamification aroused positive emotions in participants and created equal opportunities. 10 articles argue that integrating elements of game design allows for consideration of learner types and overarching learning objectives. In addition to these stated benefits, 7 other benefits were listed in table 10. Table 11 shows the recurring challenges that may hinder the learning outcome effectiveness of EG.

b. What are the challenges/dilemmas of EG?
As most of the articles indicated a positive effect of gamification on different dimensions (table 10), half of the studies did not indicate any challenge. However, the most frequently observed challenge of gamification (n=19) is the differential impact of gamification implementation on different types of learners. Gamification may not impact all students equally if there is a difference in students' academic abilities [24]. The learner types and the demographic, social, and cultural characteristics of the learners responded differently. 16 studies indicated that gamification may have a limited effect on increasing scores or, in some empirical studies, no significant differences were found between experimental and control groups [38,50,62,65,103]. This may be due to the short time frame for implementation. Logical and technical concerns in implementing gamification (n=9) are another dimension of challenges [19,64]. Another considerable challenge is the insufficient theoretical background of gamification. 8 studies stated that there is insufficient theoretical evidence to support the relationship between digital badges and learner motivation [112]. Scientific rationale behind gamification implementing in education is still seem imprecise. Yildirim [116] stated that adding a limited number of game components to the educational process cannot be accepted as gamification implementation. 36 articles do not include a digital tool to implement gamification (table 9), i.e., they might have only incorporated some game components into the lessons. 6 "Sparse and inconclusive empirical evidence on the relationship between gamification and motivation or need to confirm initial findings" Therefore, challenges related to students' social and cultural context or students' recognition of gamification, etc. were mentioned in eighth of the articles. Tu et al. [105] discussed that understanding learners' player personalities, preferences, or characteristics and the context of gamified instruction is essential to implementing effective gamification. Likewise, in their review study, Alomari, Al-Samarraie and Yousef [4] noted that some students might not engage with the gamified learning activity because they are inexperienced with gamification and its techniques.
Furthermore, five of the studies argued that gamification only provides engagement for a short period of time. Tan and Hew [104], for example, compared the experimental and control groups for only three days, and the reported positive results were observed during this short period.
Five of the reviewed studies expressed that gamification, when well designed and properly used, contributes to learning [34]. Teachers failed to take into account the importance of student capabilities in learning the task [83]. Another four articles pointed out that the limitation in settings can prevent positive results. Specifically, the result was only presented for modifications in a very specific setting [43]; the results of the studies depend on a small sample size or activity was designed for limited number of students [19,42,80,90].
Another aspect is the hesitation of teachers to use gamification. The lack of experience with this type of activity or the low expectation of teacher reduces the use of EG [85]. Buck [13] criticizes gamification, stating that EG threatens the institutional structure of schools and the teaching profession; the ongoing virtualization of schools could lead to social isolation, and thus gamification instead of teaching. Four of the studies indicated that the use of gamification is time consuming. This means that designing, developing and setting up a gamification used procedure is time consuming [18]. Providing individualised feedback for each lesson also takes up teachers' time [10].
The lack of a theoretical framework for significant fields was indicated in 3 studies. The use of gamification in the classroom needs methodological support to enhance learning [36]. Four other articles pointed out four different challenges. One of the studies noted that strict implementation of EG hinders creativity [86]. Evaluating educational outcomes is a necessary part of education, but the nature of the object of evaluation could lead to poorer outcomes [29]. Another pointed out that the positive results were achieved because of highly engaged instructors [32].

Discussion
Gamification, which has emerged as a new field of research with the popularization of digital games and the spread of their components into other fields such as education, has steadily increased the number of publications year by year over the past decade. In the period from 2013 to March 2018, gamification in education research has mostly been conducted, in descending order, using case studies, quasi-experimental, and experimental methods. Sardi, Idri and Fernández-Alemán [94] also identified the three main research methods in their review study of eHealth: case studies, experiments, and surveys. In this current study, case studies mostly tested the impact or effectiveness of a gamification application or explained the process of gamification implementation as part of instruction. Also, 18 of these case papers provided insight into the gamification of a classroom, instruction or system. Few of them explained the experience of samples with gamification or game elements [1,91,100]. Similarly, 21 included studies, reviews (a high percentage), explain the conceptualization, definition, composition, and implementation of gamification, which may indicate that research at EG is still in its early stages.
For included 141 studies the most recurring research aim (n=34) is to gather information about the opinions and prior knowledge of the samples about gamification. It can be said that seeking data about users' reaction to gamification is popular at this stage of the EG research.
In conclusion, gamification studies seem to examine how students respond to gamification rather than what or how it contributes to their learning. Olsson, Mozelius and Collin [82] also stated that EG is less focused on learning outcomes and therefore has limited contribution to positive learning outcomes. Looking at table 10 for the benefits of gamification implementation, we see that the motivating, engaging and etc. effect of gamification on learning is implied, but it is not explicitly stated that it increases learning. From these two pieces of information, it can be deduced that gamification is a more helpful strategy for the learning context than for the learning itself. On the other hand, 5 of the 141 studies strongly criticized gamification and pointed out that gamification is a less effective method [13,19,61,62,78]. Thus, thirteen different challenges of gamification were identified. Hopefully, they can be helpful in avoiding pitfalls in the implementation of gamification in future research and application by educators. More than half of the studies reviewed (n=81) reported outcomes for undergraduate (college) students, including nursing and teacher education faculty. One of the reasons for this high percentage is that university researchers and instructors are able to conduct more research with their students because of the ease of participation and data collection [33]. Another reason to consider could be that the increasing contemporary prevalence of gamification among educators has led to attempts to train pre-service teachers to be able to incorporate gamification into their teaching in the future. It can also be as a preparation for the interface of future virtual learning environments based on artificial intelligence. If we look at the names of the courses in which gamification has been implemented, we find that gamification is usually used in introductory courses of subject fields, especially ICT (table 5). So, as a conclusion, the results they report have to do with basic skills. This result leads to ambiguity about whether gamification has an impact on the acquisition of metacognitive skills in these courses or what that impact is.
Since gamification is the incorporation of game elements into an existing system, all included studies were found to contain data on a/some game element(s). There is a plethora of game elements (dynamics, mechanics, components listed in tables 6, 7, and 8). Many different game elements have been named that have been used to construct gamification. The most common gamification components evaluated in examined studies were points, badges and leaderboards in that order (table 8) and is also stated by Alomari, Al-Samarraie and Yousef [4], Looyestyn et al. [71], Sardi, Idri and Fernández-Alemán [94], Werbach and Hunter [111], and Wood and Reiners [114] in their studies. Points are an essential component of gamification [34,94]. Forty-seven percent of the selected papers used just points for gamification or as an aspect of gamification in this study. This could be a prediction for the usability of points, and they could continue to be widely used for gamification in education. Two of selected studies stated that points for gamification is less effective for educational purposes [7,62]. If so, the widespread use of points can be argued.
The concept of gamification allows the use of game design elements to engage and motivate people and to encourage desired behaviors [55,66]. The use of badges, points, and leaderboards has positive effects on student engagement [104]. The relevance of the gamification element used and its relevance to the student is an important factor in increasing engagement [16]. Jaskari and Syrjälä [53] recommend that it would be better for gamification to engage and motivate different types of students by considering students' different game-playing motives.
They can trigger positive feelings such as a fun trick and they can also help prevent loneliness and boredom among e-learning students [82]. Gamification promotes concentration and attentiveness [2]. It also can contribute to learning, problem solving, participation, engagement, and motivation [15,30,32,55]. Gamification can increase the effectiveness of educational activities through engagement mechanisms that improve learner performance and make learning experiences more enjoyable [8,23,65]. Consistent with these finding, the engagement effect of gamification ranks third in this study after being a useful strategy for learning and increasing motivation. What is thought-provoking is that it can engage with but motivates even more.
After obtaining the results of the RQs, we focus on the point of the educational value of gamification, since there are a total of 92 challenge statements in the included studies (table 11). The two most rated challenges are the limited effect on grades/learning and secondly that gamification does not affect all participants in the same way. This situation is challenging because the goal of gamification use is to motivate and engage students and ultimately improve learning. The reason could be that the existence of an effect has not been clearly demonstrated or that researchers have not yet found a valid way to measure it. The findings of the current study suggest the latter. EG research has mainly focused on making instructions more engaging, fun, and attractive, and asking about perceptions of samples. This means that the purpose of learning improvement through gamification comes later. Parallel to these findings, the current evidence on the degree of impact of gamification on student learning is still controversial [104]. Martinez, Gimenes and Lambert [77] also concluded that the amount of study on the effects of entertainment video gaming on academic learning is currently insufficient in their systematic review. If student don't enjoy the gamification, did not complete the tasks, they indicate negative views about the gamification in the process [16].
Moreover, some studies address that gamification has difficulty in increasing students' performance because of the limited settings [43,80,90]. Articles generally warn that gamification can affect academic outcomes under certain circumstances, as Christy and Fox [21] did. These conditions refer regarding learner type, long-term or short-term design, complexity of planning and implementation processes of gamification [58,75,82]. Potential of gamification to increase learning is linked with its proper design and correct usage [34]. Similarly, Wang [109] suggests that game designers should consider the cognitive learning load and information processing ability of learners when designing educational games.
The results achieved by different applications of gamification under different conditions may vary (see table 11 for most challenging findings). It can be concluded that there is no result that says that a particular application of gamification is effective with every learning context. In total, however, this review identified nearly 60 different ICT platforms that could be seen in more than one paper. Highlighting the different platforms used on EG is suggestive for those who wish to implement gamification in their teaching contexts. Students' reaction to gamification, such as more clicks or a higher number of posts, does not always imply a correspondence to actual learning [50]. Gamification grabs attention, engages with content, and motivates reaction but that does not always promise promotion in learning.
Gamification may not work the same way for all learners. It works better for some students and learning styles than others [14], but at the same time can lead to social isolation of students [13]. Another criticism is that gamification increases motivation and participation for a short period of time [7]. Furthermore, one of the popular misconceptions of gamification is that it is just adding points, badges, or leaderboards (the PBLs) to an existing context [19,114]. Even though it was noted that adding such game elements is not enough to build gamification, these three are the most commonly used game elements (table 8). Besides, gamification can be implemented in different ways with a combination of them makes it difficult to determine exactly which factor caused the observed outcome [50]. To address these concerns in relation to gamification implementation, the learning activity could come closer to full gamification if the gamification components, mechanics, and dynamics are structured including each other, respectively.
As well as notable challenges, student engagement and motivation through gamification are generally positive, but their contribution to long-term learning or good grades is questionable at this stage of EG research (according to papers published in the years between 2013 and 2018). This may require more research. Hitchens and Tulloch [49] come to the conclusion that students can respond favorably to well-designed gamification systems. Six studies are stated strongly positive results on EG [39,44,70,91,101,102]. Lopez-Catalan and Bañuls [72] stated that EG will increase its popularity to triple and will be a mainstream in postgraduate e-learning. In 2022, if we search the Web of Science for the keywords gamification and education, there will be more than a thousand new articles listed after March 2018.
The studies reviewed provide some recommendations for the beneficiaries of gamification in education to implement effective gamification: • It requires interdisciplinary knowledge such as information systems, human psychology, education specialists, technology experts [32,58]. If practitioners ensure these aspects, it will be a better implementation of EG. • The results of the review show that gamification should be planned for learner learning outcomes. If this is not considered, it remains an aspect that does not serve learning in the learning environment. Game design elements should be designed to express recognition of student achievement and teacher expectations [32]. • It would be useful to examine the claims that gamification has short-term effects for their long-term consequences. • It requires more experimental research to determine which gamification elements work best in which contexts for which subject fields that is valuable to researchers, teachers, and students.

Conclusion
It is highly recommended to examine the result section tables to rationale the discussion and conclusion. A total of 141 studies on gamification in education were considered for this review.
In between 2013 and March 2018-time frame, EG is in its initial stage of research and case studies are the most common research methods that deal with an educational process in which gamification is applied (including a computer application or other forms of gamification) and report the educational outcomes of that process in terms of students, teachers, and instructional materials. This indicates that the implementation of gamification is designed in many variations for different learning contexts.
In order to summarize EG research trends, the use of gamification in a considerable number of different fields (25 subject fields, see table 5) could make it difficult to ensure a focused implementation procedure for specific situations and subject fields. Table 3 shows 10 different groups of research purposes at EG. It can be concluded that EG is researched in different situations with different expectations and the results obtained would naturally expand. Similarly, the large number of case studies (table 2) may indicate that researchers were looking for effective ways to implement gamification for different subjects and participants. The most frequently asked research question is the opinion of participants from almost 16 different groups of samples (table 5) about gamification in the articles. 37 different group game elements (mechanics, dynamics, and components) were used to implement gamification in 25 different subject fields for sample groups at 10 educational levels. From the overall results obtained, it can be concluded that there is no typical way to implement gamification.
Gamification is most frequently researched -in that order -in the fields of ICT, educational technology and teacher training, medical training, and language teaching. These categories include the use of gamification in the education departments and designing gamified learning activities to train teacher candidates in the use of EG or teach them some ways of implementation gamification in their curriculum. These two results could mean that gamification have been used in the process of training some certain professionals, such as teachers and nurses. Because gamification is used primarily in introductory courses, its impact on increasing advanced skills may be limited or may not be discovered until the next research.
The other two most common methods that follow are quasi-experimental and experimental research are the studies that test the effectiveness of the educational outcomes of gamification, the experiences and opinions of the participants. One of the two most frequently examined research purposes is how participants' perceptions, opinions, and knowledge about gamification. This can suggest that in this phase of EG research, participants' opinions about gamification come first, followed by its impact on learning.
The studies targeted 15 different participant groups ( figure 6). The total number of participants was approximately 16231. The most frequently repeated number of participants is 13, not very large groups. More than half of the studies reported outcomes for undergraduate (college) students, including nursing students and student teachers. Only 71 of the selected studies discussed the challenges encountered in the implementation of gamification in education. 19 of them mentioned a particular weakness that different levels of impact of gamification with participants from different types of socio-demographic groups and personalities.
Gamification is not just good or bad like other educational innovations [50]. It works depending on some factors, such as the information required by the conditions and contexts, the value learners place on the game elements, and the suitability to learners' needs, the nature of the objectives. There is no sufficiently good method or practice to call all existing courses gamified at best [32]. It is not enough to just add some game elements in the classroom to implement gamification.
This study also allows researchers and teachers to see the important titles of the 141 studies reviewed and quickly access the article they are interested in via a shared cloud link.

Limitations
Papers reviewed were limited to a publication date between 2013 and March 2018. The restricted number of languages in which the papers are written in is another drawback. Many Spanish language studies and some other languages accessed were excluded because the researcher did not know these languages. According to López-Belmonte et al. [73], who conducted a scientific mapping of gamification on the Web of Science database based on 1230 documents, the most research on gamification was conducted in Spain between 2011 and 2019. Similarly, numerous studies from Spain were included in the selected studies of this review. Some results could be affected in case the studies written in Spanish were also included.