Effectiveness analysis of e-learning implementation models and resource support in higher education institutions: case studies and insights amidst the COVID-19 pandemic

. The COVID-19 pandemic has accentuated the need for comprehensive analysis of the effectiveness of e-learning implementation models and the supporting resources in higher education institutions. This article provides an overview of solutions and case studies, focusing on the selection and evaluation of individual services and learning management platforms. To gauge the effectiveness of electronic resources in meeting students’ educational needs, the study recommends employing quantitative indicators in conjunction with students’ performance descriptions. This includes leveraging educational analytics to gather data on students’ frequency and duration of engagement with specific e-resources. The functionality modules “Course Comparison” of the Moodle Learning Management System (LMS) and “Statistics”, along with the optional Analytics module, are reviewed. The article presents the results of applying these modules to analyze e-learning courses at the National University of Life and Environmental Sciences of Ukraine and Boris Grinchenko Kyiv University. Furthermore, the study investigates the factors contributing to students’ limited usage of individual e-courses. By examining these case studies and shedding light on the reasons behind students’ reduced engagement, this article contributes valuable insights to enhance the efficacy of e-learning implementation and resource support in higher education institutions.


Introduction
The issue of the quality of e-learning resources in a distance learning environment is extremely relevant during the quarantine period associated with COVID-19 [8,15]. Despite the various quality assurance procedures for e-learning resources for students, especially in forced distance learning settings, it is often very difficult to assess the quality of the resources used relying on these procedures, which mainly include student surveys and peer review. However, due to the transition of higher education institutions to distance learning and in the context of the pandemic generated by COVID-19, the problem of analyzing the quality and adapting the design models of educational environments [7,12], according to the types of institutions, educational program, available resources, has become more relevant and other [5].
Looking at e-learning quality indicators from 2000 to 2017, Silva et al. [16], based on an analysis of scientific publications, identifies that indicators that have the greatest weight can be grouped into three categories: e-resources, data, processes. In the context of this study, we can look at the same resource provision with a focus on practical cases and a combination of quantitative and qualitative assessment.
For example, in distance learning for future health professionals, digital imaging should be used extremely closely to real-life practice. Educational institutions use specialized software, e.g. Clinical Study Export (TCE) as a platform for extending the PACS infrastructure by connecting educational functions [11].
Understanding the importance of communication and cooperation in the implementation of distance learning, the effectiveness of the use of different services is the subject of analysis. For example, Biasutti [2] presents the results of a comparative analysis of forums and wikis as tools for online collaborative learning. However, as each service has to be seen in the context of student training requirements and educational goals, more and more researchers are looking at integrated solutions to support self-regulated educational strategies [10].
In this context, among the various learning Management Systems (LMS) on the market [1], LMS Moodle is the most popular in the implementation of distance and blended learning in higher education [14]. According to research by scientists from different countries, technological satisfaction about Moodle in higher education is quite high -the effect is equal to 0.78 with a 95% confidence interval from 0.72 to 0.84 [6]. The transition to distance learning and increasing the frequency of interaction between teachers and students through LMS has necessitated the search for resources to bring online communication closer to offline. One such solution is Moodle LMS integration with Amazon Alexa for creating voice content [13].
On the other hand, the success of a distance education program can be assessed, in addition to academic performance, by the level of student satisfaction. However, there is a correlation between student satisfaction with e-courses (e-resources and content) and readiness for online learning [4]. The latter is usually determined by surveying students, and is checked -by data of educational analytics which allow to analyze behavior of students in LMS [9]. Since the use of educational analytics or analytics for student success, according to Brown et al. [3] is identified as one of the areas of educational technology, we consider it appropriate to use educational analytics to determine the causes of student satisfaction / dissatisfaction, followed by recommendations for improving resources and methodological support its application in higher education institutions. To do this, it is necessary to explore the tools that can be used to quickly analyze the effectiveness of resource use, in particular, e-learning courses in disciplines, and relevant indicators. Such analysis will allow a rapid response to the problems that users of e-courses are experiencing, allow for quick resolution of these problems and make student learning more productive.
Research goal: justify the choice of tools for educational analytics on the use of e-learning courses, in particular to determine user activity, frequency and duration of use of course resources in order to recommend to teachers to improve the quality of both course materials and methods of using relevant resources during distance learning.

Methods and study materials
The study was conducted using data from the training portal of National University of Life and Environmental Sciences of Ukraine (NULES) and Borys Grinchenko Kyiv University (BGKU). Methods and technologies of statistical analysis were used for the research.
The university's learning portal usually operates on the basis of CLMS (Content Learning Management System) platforms and is designed to support the learning process with e-resources in the format of e-books, web pages, lessons and video lessons, test tasks, laboratory and practical, independent work. For each discipline in the e-learning course can be placed the above and other resources. It should be noted that the use of e-learning courses in the educational process should correspond to the working curriculum of the discipline, and the resources should contain relevant, popular information. The procedure for attestation of electronic courses in universities involves the implementation of a number of criteria, which are usually spelled out in the relevant regulations. In particular, in NULES, these criteria are divided into structuralfunctional, scientific-substantive and methodological. But the use of certified courses is not uniform throughout the semester. At the same time, students actively use part of the resources in the courses, and some resources are not used at all. In order to identify the reasons for this and to select tools for quick analysis of course performance data and its resources, it is necessary to analyze the relevant tools, which are built-in or complementary.
Let us focus on the statistical and analytical tools of CLMS Moodle. Course resource efficiency indicators can be obtained through the use of the embedded modules: "Course Comparison" and "Statistics" modules, as well as the optional "Analytics" module.

Features of the module "Comparison of courses"
The analysis modules in the base Moodle distribution are not very powerful, but they are present too. One of the first modules that is appropriate to use when analyzing courses is Course Comparison. This can be found under Manage -Site Management -Reports. This module allows you to view four reports: 1. The most active courses (ranks courses by the number of actions taken by their participants as a whole). 2. Most active courses (weighted) (calculates the average performance per user in the course).
3. Participation rate (shows courses in descending order of user participation rate). 4. Activity ratio (determined by the ratio of involvement and participation of users in the course).
Some of these concepts need clarification. The first analytical report gives us the opportunity to see on which course there is active activity. Activity on a course is the total number of all views and publications on the course during the period under study. However, this is of little use as a large number of enrolled users will visually create more activity compared to courses with a relatively small number of participants.
Consequently, a weighted average of each user's activity on the course is already more informative and will show everyone's participation. Here you can see how truly active courses with a small number of users come to the fore. However, if there are few active users on the course, the mass of enrolled but inactive users will drag the course down in the rankings.
The following report can clarify this nuance. It shows how many real active students are on the course. The participation rate of active users is calculated as the share of active users to the total enrolment in the course. Active users are defined as those who had had activity during the period under study. But here again the question arises, what are these users doing on the course? Are they just reading (receiving information) or are they active?
The fourth report gives us the answer to this question. The publication and views activity ratio is calculated as the share of publications in relation to views. Where views refer to any user going to another page and "reading" it or downloading a file resource from the course to their computer. Publications are defined as any activity the user performs on a course, for example, completing a quiz, completing a task (downloading a file or writing a text response), replying to a forum post, and so on. That is publishing is not just a forum post.

Features of the "Statistics" module
Another auxiliary module for analyzing course performance is the Statistics module, which operates at the site level, providing some statistics on the activity on courses as a whole, as well as in each individual course. At the site level, the results of this module are available to administrators and site managers. At the level of each course, teachers can use it to generate statistics within that course.
About the Course Comparison module analyzing activities without dividing into teachers and students, it is important to say exactly who generates such activities -the actual training of students or the active creation of a teaching course by teachers. The Statistics module brings clarification. Using this module, we can see the activities of each individual role. It is also possible to look separately at views, only publications, or only introductions. At a site-wide level, these metrics plot all roles, while at a course level we can get a separate graph for each role.

Possibilities of the "Analytics" module
Additional analytical reports can be obtained using third-party modules. One such powerful addition is the Analytics module. With its help it is possible to analyze activity of each student both as a whole on a course, and in each concrete resource of a course. We are provided with such reports: • Valuation chart (shows the distribution of valuations using a stock chart) • Work with content (shows the activities of students with each resource separately -how much they worked and how much they ignored) • Student activity (shows a consolidated distribution of student activities on the course in terms of hours per day) • Execution of tasks (shows a diagram for all Tasks, which demonstrates compliance with the deadlines) • Passing tests (shows a diagram for all tests, which demonstrates compliance with the deadlines) • Distribution of views (shows the schedule of personal activities on the course of each student) Figure 1: Moodle e-course comparison module data model.

Main findings
Only certified electronic learning courses (ELC) were chosen for the study, i.e. courses in which the structure and set of resources are correctly selected for the implementation of the educational process. Thus in NULES the certified courses operate within 5 years, and in BGKU -within 1 year. The number of certified courses at the end of 2020 in NULES was 1644 courses, in BGKU -768 courses. The first hundred most active courses (with high average activity per user) can be obtained by using the Course Comparison module and its "active courses (weighted)" report (figure 2). Based on the results (figure 2) it is possible to identify courses with low user activity. The following analysis of the content of such courses, didactic features of the use of course resources will make it possible to identify relevant problems with their use.
Usage statistics courses and the way they use within the categories are also using the module "Statistics". For example, in BGKU such statistics can be obtained by categories of departments (figure 3), and in NULES -by categories of specialties.
A direct query to the database of courses makes it possible to obtain such data for all certified courses.
As a result, all courses can be divided into 3 categories: courses with low efficiency, sufficient and high in terms of "activity per user". For example, NULES with a high degree of use has 12% of certified ELC, with sufficient -57%, with low -31%.
Analyzing ELCs with high efficiency, a number of studies have been conducted on the use of resources of these ELCs using an analytical module built into Moodle.
For example, figure 4 reflects the students' activity in the course "Computer Technologies and Programming" during the last semester. The activity of students in revising resources and publishing completed tasks or passing tests is uneven. The peak falls towards the mid-term examinations and the end of the semester, which may explain the need to complete the quizzes. But we can conclude that the use of the course at the beginning of the module is not active enough and indicates that the laboratory and self-study assignments are not completed on time and, consequently, students are not working with the resources.
To find out with which resources in e-courses, students work actively, the function "Working with content" of the module "Analytics" is used. For example, figure 5 shows the activity of using theoretical resources of the course "Information Technology". From this diagram we can conclude about the extremely low activity of students in the use of theoretical resources, in contrast to laboratory work.
Often students actively use electronic learning resources in a discipline only because they have to take a test every day and hand in work to be tested, but they do not use theoretical resources because they are not very informative. Another option is for students to actively use methodological materials in the discipline, watching video tutorials, and to a lesser extent use resources designed to monitor learning achievements. An important task for universities is to obtain tools to quickly assess the quality of electronic resources by further using their content and methodology in the teaching process.
Next, you need to analyze the content of educational material set out in theoretical resources, in terms of structure, accessibility, relevance, practical orientation. These are all tasks of scientific and substantive examination. Such an electronic course can be reconsidered by the educational and methodical commissions of the faculties regarding the possibility of its use in the educational process.
As a result of using such tools, we have the opportunity to determine which e-courses contain: • an excessive number of tasks (exceeding the number of laboratory, modular and independent tasks) that required the work of students with the course; • little informative, unstructured training materials • qualitatively presented theoretical materials and methodical recommendations, which were actively used by students; • educational resources that were systematically used during the semester.
To increase the efficiency of the use of ELC in higher education, a number of steps can be taken to use statistical and analytical tools Moodle ( figure 6).
The first step should be to rank the courses by activity per 1 user (weighted indicator). For all courses that are actively used, the second step is performed -statistics of general activity in e-courses and analytics of resource use in e-courses. The third step is to form conclusions and recommendations.  Identifying e-courses that are insufficiently used in the educational process, provides an opportunity to intensify work with teachers to improve their skills with information support, to create e-courses, the use of e-resources in the educational process. The e-resources found not to be used by students in the learning process should be reviewed according to scientific peer review criteria. Built-in and additional CLMS Moodle tools allow you to analyze the effectiveness of e-courses in general and in terms of different types of resources and, based on this analysis, to form general recommendations for course teachers to improve the use of e-courses in education.

Conclusions and the research perspective
The use of statistical and analytical tools in CLMS Moodle to determine the effectiveness of the use of e-courses contributes to the quality of the educational process, in particular blended and distance learning. By measuring weighted course user activity, overall activity within the course and analyzing the use of course resources, it is possible to identify the reasons for the inefficient use of e-courses in the educational process. Since the study was carried out on the basis of two higher education institutions, we can assert general trends on the problems of using e-courses in blended and distance learning. In the future, we see the need to develop a model that provides automated determination of levels of effectiveness of e-courses, e-course resources, identification of factors that affect the effectiveness of the use of courses, and specific resources.