Return to Article Details Educational technology for the formation of project competence for engineering students

Educational technology for the formation of project competence for engineering students

Liudmyla V. Hapon-Baida[0000-0003-0352-9447]
and Tetiana M. Derkach[0000-0003-1087-8274]
Kyiv National University of Technologies and Design,
2 Mala Shyianovska Str., Kyiv, 01011, Ukraine
Abstract. Project competence is essential to the professional education of engineering students. It synchronously affects all three components of future professional activity, forming the ability to be creative, carry out scientific research and apply modern technological solutions. However, there often is a lack of attention to master projecting in educational programs, especially for junior undergraduate students. The article describes the developed educational technology for organising project-based learning (PjBL) in the actual educational process conditions to mould engineering students’ project competence. It was based on incorporating a short module dedicated to mastering the basic principles of PjBL into the existing curriculum. During the module, students developed, executed and publicly defended their projects. The choice of topic was up to the students, provided the project was dedicated to waste management. The mastery of the seven PjBL essential elements was considered an indicator of the level of project competence formation. Conducting research in the current educational process put forward strict limitations regarding the project execution time, their permanent adjustment through feedback, and timely evaluation. Short daily surveys via Google Forms devoted to each lesson topic let one organise permanent feedback between the teacher and students. The effectiveness of the applied technology was evaluated by two more detailed surveys at the beginning and after the end of the training. The understanding of PjBL elements was noticeably improved. The improvement was statistically significant for four elements, while for the remaining three, it was borderline.

Keywords: project-based learning · project design elements · learning styles · survey using Google Forms · development of reflexivity

1 Introduction

The needs of the modern world require producers and consumers to be economical with the consumption of natural resources and switch to other types of materials, resources, and technologies to reduce the burden on the environment and human health. It is related to the idea of conscious consumption or the idea of sustainable development [2635].

Almost ten years ago, the UN formulated 17 sustainable development goals to be achieved by 2030 [1517]. Subsequently, the goals were structured into four main types of sustainability: human, social, economic, and environmental. In turn, socioeconomic changes place specific requirements on engineering education. Namely, they require from future specialists new knowledge, abilities, skills and professionally important internal qualities that contribute to the ability to quickly adapt to changes caused by technological advances, such as the appearance of new materials, devices and technical processes, as well as satisfy the demands of consumers and the labour market.

In other words, multidisciplinary training comes to the fore when the learner simultaneously and synchronously acquires skills and abilities that ensure the fulfilment of the UN’s human, social, economic and environmental tasks. The fulfilment of these tasks requires the simultaneous development of three main components of professional activity: creative, scientific and technological. The creative component is oriented (first of all but not only) to achieving human and social goals of sustainable development. The development of the scientific component is closely related to the economic aspect of sustainable development, which consists of innovation, the financial income of companies without harming the environment, and the development of conscious production in balance with conscious consumption. The technological component is based on the processes of development, manufacture, and processing of products and industrial waste following established technological and environmental requirements. Accordingly, the ecological tasks of sustainable development are primarily related to technologies, their development level, and specialists’ training.

The synchronous development of these main components of professional activity is a complex problem and depends strongly on educational programs and current education standards. The imbalance of these documents often prevents the achievement of desired learning outcomes. Even though the concept of the competence approach dominates in Ukraine, it is not always supported by relevant defined skills and abilities in the context of individual specialities. In particular, insufficient attention is paid to developing the ability to design for many engineering and technological study fields [25].

For example, graduates of creative specialities must demonstrate a pronounced creative component (due to mastered creativity) along with the practical component of their activity. It is a requirement of the modern labour market. However, there is a discrepancy between this requirement and the approved educational standards. For one speciality, competence “creativity” is defined as being without competence in project activity, and in another related program, on the contrary, there is projecting without creativity. In training for a bachelor in the speciality “Professional Education (by specialisation)”, such competencies as creativity and the ability to design are not defined.

Purposeful correction of the training content cannot be done without preliminary studies to identify the effectiveness and feasibility of proposed changes. It is especially relevant for the modern conditions of the development of information and communication technologies when new, more and more sophisticated technologies and approaches to the learning process appear every year [2130313741].

Traditional long-term pedagogical research takes years, which is not always acceptable when changes in the pedagogical process, primarily due to developing new technologies, occur much faster. There is a demand for designing and conducting short-term studies to help solve the problems of adjusting training courses.

The research, aimed at bringing professional competencies to the modern requirements of the labour market by optimising the content of educational disciplines, helps to overcome this challenge and, therefore, is a topical scientific problem. It is known from the literature that project-based learning (PjBL) is quite popular and has certain advantages over traditional learning methods. Project-based learning contributes to the long-term preservation and development of knowledge, skills, and abilities [34]. It also contributes to a better understanding of the educational material, increases self-motivation, activates the learning process, and develops creativity [78]. Intellectual skills are formed: skills in the subject field of expertise (subject competencies), intellectual skills of critical thinking (search for information, awareness, analysis, synthesis, application, and evaluation), and communication skills (skills of joint activities, the ability to lead a discussion, and make decisions [2].

According to traditional teaching methods, students must memorise a certain amount of knowledge. Instead, using PjBL, the student should understand the basic principles of projecting from the stage of its development to implementation and public defence and learn to apply them to various situations in life and study. Thus, we are talking about possessing tools that will help solve new, unknown tasks in the future.

At the same time, using project-based learning as the main educational technology is very rare. The writing of term papers and qualifications can be mentioned, while the academic disciplines are mostly taught using more traditional approaches. They are mainly aimed at mastering some specific issues, some of which have a technical orientation, and others are socially oriented [132]. Usually, there is a lack of disciplines that would consolidate knowledge at a higher, generalising level.

Such a situation contradicts the requirements of training specialists to work in conditions of sustainable development when awareness of the task at different levels comes to the fore. It requires appropriate technological, ecological and economic knowledge and skills in landscape restoration and nature-based solutions [69]. Moreover, even when subjects that seem aimed at forming a more general picture are considered, the situation remains similar [4].

The educational programs are fully loaded and scheduled. Therefore, to improve them, for example, using project-based learning, the tactics of incorporating short dedicated modules into existing educational components is most promising. Initially, such modules are temporarily integrated into existing academic disciplines. In the first attempts, they are used for experiments to optimise the training content. When stable positive results are obtained, new modules will be permanently introduced into the curriculum.

There is a problem with conducting research using project-based learning in a real educational process when experimental tasks “fit” into the program on the go and do not have particular preferences. The need to intervene in the existing educational process imposes several restrictions. First, the investigation must be balanced over time because the integrity of the student’s perception of the material is preserved. Secondly, the experimental module, which is not foreseen in the curriculum, must be short enough to avoid interfering with the implementation of the current curriculum. Thirdly, it is most suitable to organise experimental training within hours-long disciplines to minimise relative changes in the curriculum. The most likely fact is the small sample of involved respondents since the limited investigation time allows us to work with students of only one group.

Thus, it is critical to develop a small experimental block of lessons that must be conducted quickly. When implementing experimental training, one needs to constantly assess the effectiveness of the intervention and monitor the students’ daily progress. Operational feedback is essential for success when content optimisation and corresponding performance research are conducted in the real learning process and in real-time. Such an educational technology technique allows one to work out the changes required in the content of academic disciplines in the conditions of the actual educational process. It does not require sophisticated electronic resources and is not limited to the context used. Moreover, it can be extended for more in-depth research and has the potential for further development and expansion.

The research goal is to develop a technology for the organisation of experimental training to form project competence using the project method in the conditions of the ongoing educational process. The proposed approach is based on the development, execution and defence of students’ projects in waste management. Two types of specially developed questionnaires using Google Forms will provide either daily feedback to correct learning content quickly or evaluation of general training effectiveness at the end of the training compared to its beginning. The evaluated degrees of mastering the seven basic elements of the design project indicate the formation of project competence.

2 Research methods

2.1 The main elements of the project design

The main task of the PjBL module is to master the principles on which it is built. Another essential aspect of mastering PjBL is learning while formulating and implementing projects.

There are many definitions regarding the principles of PjBL. An educational organisation, PBL Works, created a comprehensive model and formulated seven primary project design elements, known as the “gold standard of projecting” [23]. Mastering these elements, briefly listed in table 1, is necessary for successfully using the PjBL method. Therefore, the idea of familiarising students with the seven “golden” design elements and mastering them in implementing projects developed personally is the basis of the educational material in the experiment.


Table 1: The main elements of the project design taken over from [23].



No

Full and
abbreviated name

What the element describes
and its brief description



1

Challenging Problem or Question (CP)

Process. A challenging and exciting challenge makes learning more meaningful. Students do not just acquire knowledge to memorise but learn with a real need for knowledge to use in the future




2

Sustained Inquiry (SI)

Mode of action. When faced with a complex problem, students ask questions, find the resources needed to answer them, and then ask deeper questions until the answer is found




3

Authenticity and Originalities (AO)

The nature of the task. The project should have a real context – solve real problems, and have an impact on solving real issues




4

Student Voice and Choice (SV)

Students’ skills. Having a voice gives students a sense of ownership and improves project stewardship. Otherwise, students feel like they are just doing an exercise or following the instructions of others




5

Reflection and Thoughtfulness (RT)

Deliberation. Students and teachers must consider what they are learning and how and why during the project. Reflecting on the content of the acquired knowledge helps students consolidate this knowledge




6

Critique and Revision (CR)

Teamwork. Thoughtful criticism and refinement are the keys to high quality. Constructive feedback with peers, teachers, and experts improves project processes and work products




7

Public Product (PP)

Product characteristics. Publicising the results of project work is an effective way to increase motivation, performance quality and communication with others





2.2 The scheme of the experiment

A significant drawback of the existing curriculum is insufficient attention to forming professional design competence, lack of appropriate methodology and clearly defined expected learning outcomes. A complicating factor is a need to eliminate the listed shortcomings in the conditions of the educational processing process, which does not provide time and conditions for experimentation. Under such conditions, the most reliable option is to incorporate a relatively short academic module into the curriculum to ensure the necessary knowledge and skills mastery.

Students in the fourth year of the Faculty of Fashion and Arts of Kyiv National University of Technologies and Design participated in the experiment. The group consisted of 8 people: students specialising in professional education in technologies. In the fourth year, the discipline “Creative Learning Technologies” is studied, which contains 180 hours (6 ECTS credits). The developed experimental module dedicated to PjBL comprises lectures and practical classes for 18 hours.

The proposed scheme of the experiment includes three main stages (figure 1). The first two stages are characteristic of pedagogical experiments. They are called ascertaining and formative stages; the third focuses on analysing the results.


PIC

Fig. 1: The scheme of the experiment.


The purpose of the ascertaining stage was to determine the level of student knowledge regarding PjBL as of the beginning of the experimental study. First, students listened to lectures and practical classes (8 hours total), where they got acquainted with the basic concepts and components of problem-based and project-based learning. Before these lessons, students had yet to meet using any projecting. Then, students were surveyed using a questionnaire specially developed by the authors to assess their initial level of knowledge.

The second formative stage included experimental training using the developed methodology. A new educational module of 18 hours (9 lessons) was incorporated into the existing discipline. The module’s topic is “Mastering the PjBL Methodology using the Example of Individual Projects “Decorative Painting of Fabrics as a Technology of Textile Upcycling”.

The independent creation of student projects was preceded by practising the skills of working on individual elements of the project methodology under the guidance of a teacher. In the experiment, students were not limited in choosing design objects within the specified subject. It was considered mandatory to apply the idea of recycling used items and use one of the upcycling techniques for this, namely decorative fabric painting. During the development and implementation of the project, the teacher mainly played the role of a consultant.

Table 2 illustrates the main stages of the performance of experimental training.


Table 2: Stages of the experimental training by PjBL.



Stages

Activity

Time, h



1. Problem-targeted

To justify the purpose and choice of the design object. Determine the intended purpose of the future product. Formulate ergonomic, aesthetic, technological, economic, and environmental requirements. Describe the expected results.

2



2. Development of the technical task

Develop the design part, determine the technological sequence of manufacturing the product, justify the selection of materials and tools, examine the techniques of product execution and processing methods, make the necessary economic calculations, and draw up a planned schedule for project implementation.

2



3.1-3.4 Practical implementation

Produce a design object and follow the rules for using tools and materials. Since all projects were individual, it is impossible to determine a single time frame for the sequence of works.

8



4. Preliminary defence

Discuss the work results with the teacher and fellow students, and correct the identified shortcomings.

2



5. Public presentation of results & summary

Present the design result, analyse the achievement of the set goal and reflect on the success of the chosen ways of solving the set tasks.

2




In experimental training, students had to master the main elements of project design. A short, special survey was conducted after each class to operationally correct the content of new classes and the applied methodology. The survey results were processed to correct the educational materials the next day. The content of the daily surveys varied depending on the training content. The second stage ended with a second survey using the same methodology as before the start of the experimental training.

The third stage aims to assess the effectiveness of the experimental training. For this purpose, the results of surveys after the first and second stages were compared. Individual learning styles of students were also determined using the Felder-Soloman method. Determining students’ preferred learning styles allowed one to reveal the existing educational advantages of the group. Adjustment of training based on the results of daily surveys was aimed at the optimal use of training resources in experimental training and the development of training benefits that would contribute to a better assimilation of the principles of project activity.

2.3 Survey methods

The study’s primary method of obtaining quantitative data was the survey method, which was applied to three objects. Three different questionnaires were used to conduct the survey. The authors developed two types of questionnaires, and the third type, known from the literature as the Learning Styles Tool, was available for free use.

Very short deadlines for conducting the research and the need to constantly maintain feedback between teachers and students throughout the experiment led to a specific form of questionnaire implementation. The specificity consisted of using Google Forms to prepare questionnaires [22]. The forms were stored on the university’s Google Drive, and their access was implemented through QR codes. This significantly expanded the students’ ability to use various devices (tablets, smartphones, and others) to perform research tasks and promptly transfer their impressions to the teacher. Ensuring quick response of students to teachers’ demands in wartime conditions was only possible with this.

An example of a filled-in Google Form related to questions about understanding a Challenging Problem or Question (CP) is shown in figure 2. The more often feedback is used, the greater immersion in work and increased motivation among students. Simultaneous assessments by the teacher and student self-assessment of educational results are carried out. Surveys to adjust the content of individual educational blocks were repeated before each lesson. Depending on the purpose of the block, their structure (types and number of questions) differed. Although Google Forms is used only as a well-known technical toolkit in this study, it is an indispensable foundation for the developed technology. It is an excellent example of the use of information technology in pedagogical research.


PIC

Fig. 2: A completed Google Form for determining the degree of mastery of project elements (an example to master a Challenging Problem or Question (CP)).


Thus, surveys with the help of Google Forms can be components of educational technology when students become, as they were, organisers of the process, realise the goals of studying a particular topic and are responsible for the result. It can encourage students to better reflect on each stage of their activity. The teacher can store statistical data in Google spreadsheets and process them using data analysis and visualisation methods [11]. The survey at the beginning and the end of the experimental lessons allows one to assess the experiment’s effectiveness and adjust the teaching method promptly.

The questionnaire developed in the paper to determine the degree of design element mastery was used twice in ascertaining and formative experiments. The questionnaire covered all seven design elements and consisted of 49 questions. It is aimed at assessing how well students understand the essence and importance of individual design elements of the project. Seven questions or statements were formulated for each element, and the respondents had to express their attitude to each problem. Seven answer options were available, from which only one had to be chosen.

Such a result evaluation scheme corresponds to a 7-point Likert scale [19]. The points available for answers range from 1 to 7 and have the following meaning: 1 – strongly disagree with the statement in the question, 2 – disagree, 3 – somewhat disagree, 4 – no clear opinion (that is, it is a neutral or zero position), 5 – somewhat agree, 6 – agree, 7 – agree entirely with the statement in the question. Some statements were formulated in direct order (as explained above), and some in reverse order to avoid simple guessing without understanding the question’s essence. For the reverse statement, 1 point corresponds to the option to agree entirely and 7 – to completely disagree. In further analysis of the results, the scale for reverse questions was converted into a scale for direct questions.

The model of R. Felder and L. Silverman [12] was used to determine students’ preferred learning styles. For the practical application of the model, R. Felder and B. Soloman developed a tool called Indices of Learning Style [13]. This tool helps determine the individual preferences that respondents have in education. Four dimensions determine the division of learning styles, each combining two oppositely directed styles.

The first dimension is the perception of information, which includes a sensing (sen) style – oriented to facts, specifics, and practice, and opposite an intuitive (int) style, oriented to intuition, theories, and abstract concepts.

The second concerns sensors for information perception. They are based on the visual (vis) style, which gives preference to illustrations, and the verbal (vrb) style, which prioritises verbal explanation (both oral and written).

Comprehension of information includes an active (act) style – orientation to activity in work and experimentation, and reflective (ref) – based on reflection and observations.

The learning pattern includes sequential (seq), which is aimed at step-by-step understanding, convergent thinking, analysis, and global (glo) style, which is focused on systemic thinking and synthesis.

The Felder-Soloman method helps determine the direction and degree of educational preferences of students according to the above styles with the help of particular questionnaires, in our case, via Google Forms. The questionnaires contain 44 questions, 11 for each dimension, which help to determine educational preferences in four areas: sen/int; vis/vrb; act/ref; seq/glo. Each answer to the question adds +1 or -1 point for this dimension, and their sum determines the presence of one or another preference.

2.4 Statistical treatment of the results

Statistical methods were used to analyse the obtained results. All calculations were made using the statistical package IBM SPSS Statistics, version 21 [27]. The statistical study aimed to solve two main problems: assessing the reliability of the developed survey methods and comparative evaluation of the figures demonstrated by students at the first and second stages of the experiment.

One of the survey methods used in the research, namely, the Felder-Soloman method, was developed by other authors and repeatedly used in many published studies [1424]. Accordingly, the level of reliability of this technique was checked many times. The reliability, expressed through Cronbach’s alpha coefficients, varies between 0.55-0.76 according to the results of different authors [1424]. Such a value demonstrates a sufficient level of reliability of the Felder-Soloman model, taking into account the extensive database of verified questionnaires (more than 1000 units). Therefore, the model itself does not require additional checks.

The results of daily surveys, aimed at prompt correction of teaching methods, were not used for formulating general conclusions but for quick application in the educational process. Therefore, the most critical criterion for the reliability of this technique is the type of changes in the educational process that result from the practice of conducting classes.

The third questionnaire method, developed by the authors, aims to determine students’ mastery of the main design elements. The reliability of this technique was assessed by calculating Cronbach’s alpha [27, p. 246–254]. This characteristic evaluates the reliability of the method as the internal consistency of the test. In other words, it measures how partial measurements (answers to individual questions regarding the same element of the project) agree. The Cronbach’s alpha coefficient value ranges between 0 and 1 if the test is unreliable and reliable, respectively. The border between reliable and unreliable tests is often the subject of scientific debate. However, in most cases, a value of 0.55-0.6 and above is considered the reliability limit.

The reliability testing results are given in table 3 based on the results of applying the developed questionnaire at the beginning and after the end of the experimental training.


Table 3: The results of Cronbach’s alpha test based on student’s answers in the ascertaining (1) and formative (2) experiments.




NoElementCronbach’s alpha (1)Cronbach’s alpha (2)




1 CP 0.011 0.704




2 CI -0.174 0.785




3 AO 0.701 0.847




4 SV 0.721 0.791




5 RT 0.581 0.811




6 CR 0.75 0.801




7 PP 0.606 0.862





In the first testing, before the start of PjBL, students often did not fully understand the questions, affecting the results obtained. Out of 7 project design elements, the results of Cronbach’s alpha calculations met the reliability criterion in only 5 cases – they were higher than 0.58 (Cronbach’s alpha index (1) in table 3). However, the results improved significantly after the students received the experimental training (Cronbach’s alpha (2) in table 3). In this test, Cronbach’s alpha exceeded 0.7 for all elements examined. Such a result allows one to say that the developed questionnaire provides reliable results when the respondents already have a stable idea about the nature of the investigated elements.

Another statistical analysis task focuses on a statistically significant difference between the sets of respondents’ answers at the stages of ascertaining and formative experiments. The answer to this question allows one to assess the proposed teaching technology’s effectiveness quantitatively. In addition, it is essential to determine the presence of a statistically significant difference between learning styles among different groups of respondents. To choose the optimal statistical model, one should consider the characteristics of the studied data array:

1.
All survey results are rank values.
2.
As the Kolmogorov–Smirnov test results show, not all samples follow a normal distribution (table 4). It especially applies to the first stage results showing no normality in all 7 cases.
3.
Because the questionnaires were anonymous and not all students participated in the two surveys being compared, it is necessary to use statistics for independent samples.
4.
The work examines small samples (a small group of students was interviewed). Therefore, it is necessary to use statistics that provide the best results specifically for analysing small samples.


Table 4: The results of the Kolmogorov-Smirnov test (*) for the presence of normal data distribution.









Elements CP SI AO SV RT CR PP








Ascertaining experiment








Kolmogorov-Smirnov Z 1.2051.0231.1041.318 .9661.3441.266








Asymptotic significance (2-tailed), p .110 .246 .174 .062 .308 .054 .081








Formative experiment








Kolmogorov-Smirnov Z 1.3271.1911.5341.6391.6411.6621.788








Asymptotic significance (2-tailed), p .059 .117 .018 .009 .009 .008 .003








(*) The null hypothesis to be tested is a sample under study has a normal distribution


Summarising points 1-4 above, one can conclude that the Mann-Whitney U criterion best meets the experiment conditions and the nature of the data obtained [27, p. 152]. Namely, this test is designed to analyse rank variables in small independent samples. The number of elements in the compared samples may differ, but not less than three. The Mann-Whitney test is non-parametric. Therefore, this method does not require the presence of a normal distribution and the equality of variances.

3 Results and discussion

3.1 Study of project design elements

All students coped with successfully designing, implementing, and defending the developed projects. Therefore, the proposed educational technology has prospects for further development. As mentioned earlier, the student’s understanding of the main project design elements was examined twice through a questionnaire – before the beginning and after the end of the experimental study. The results in the form of ratings of average scores are shown in figure 3.


PIC

Fig. 3: Average scores for individual design elements based on the results of ascertaining and formative experiments.


In the first stage, the respondents’ average scores exceeded 5 points (Somewhat agree) only for four out of seven elements. For two elements, SI and CP, the respondents’ opinions varied between 4 (I do not have a clear view) and 5 points. For these two elements, Cronbach’s alpha, calculated based on the first stage results, was far from the level characteristic of a reliable scale. This fact may confirm our assumption that respondents did not fully understand the nature and role of these elements but only tried to guess adequate answers.

The situation was significantly different according to the survey results at the second stage of work (figure 3). For all elements, the average scores confidently exceeded 5 points. For two elements (CR, CP), they came close to 6 points (Agree); for SV, they even transcended this line. It should also be noted that the most progress in promotion in the rating was characteristic of the elements of CP (previously the most problematic) and SV. Conversely, the SI element was the only one that remained at the bottom of the rating. All other elements stayed again in the middle of the rating.


PIC

Fig. 4: Comparison of the average scores of individual design elements based on the results of ascertaining and formative experiments.


The values of average scores increased after the respondents underwent experimental training for all elements, without exception (figure 4). If the average scores ranged between 4.75 and 5.4 during the initial testing, the values of all scores increased to 5.54–6.23 after training.

At the qualitative evaluation level of the results, the training seemed to improve the respondents’ understanding of the essence of the project activity in all directions. However, the data given is not enough to formulate final conclusions. An open question remains whether the obtained difference reflects a real improvement in the perception of project elements or whether the observed difference results from statistical error. Special statistical testing is necessary to solve this issue.

3.2 Comparative analysis of the results of ascertaining and formative experiments

The degree of understanding and ability to use individual design elements are indicators of project competence formation. The applied approach improved the knowledge of PjBL and clarified its importance and individual element roles. The degree of progress achieved differs for different project design elements. It may be related to teaching deficiencies and students’ perceptions of various educational materials. Determining the availability of statistically significant improvement remains essential because it may not occur in all cases.

The Mann-Whitney U test was used to compare survey results at the first and second experimental stages. The null hypothesis of the Mann-Whitney U test is that the differences between the samples for the ascertaining and formative experiments are random (or do not exist at all). An alternative statistical hypothesis is that such differences exist. Acceptance of the alternative hypothesis gives reason to believe that the differences between the groups are statistically significant at the p <0.05 level. The results of calculating the asymptotic values (2-tailed) of p are given in table 5.


Table 5: Mann-Whitney U test statistics (*) to compare average scores for individual design elements calculated in the ascertaining and formative experiments.









Elements

CP SI AO SV RT CR PP








Mann-Whitney U test statistics

457.500559.000560.000387.000541.500566.000526.500








Z

-2.913 -1.844 -1.845 -3.695 -2.050 -1.804 -2.214








Asymptotic significance (2-tailed), p

.004 .065 .065 .000 .040 .071 .027








(*) The null hypothesis of the test is that the differences between the two samples are not significant


For CP, SV, RT and PP elements, the Mann-Whitney test predicts the presence of a significant difference between the results of ascertaining and formative experiments (figure 5). For the other three elements, SI, AO and CR, the calculated p parameter slightly exceeds the value of 0.05, which is accepted as a threshold value. For example, for SI and AO, the p-value is 0.065; the probability that the compared groups are identical is 6.5%, slightly higher than the accepted threshold of 5%.


PIC

Fig. 5: The difference between the results of the formative and ascertaining experiments expressed in the average ranks for each design element with the determination of the level of statistical significance p.


The content of experimental learning can cause significant progress in understanding the role of different design elements. After completing the training, individual training lessons were analysed from the point of view of how many and which design elements were considered. The analysis results in a diagram are shown in figure 6. We see a significantly different load on design elements from the point of view of their use at different stages of experimental projecting. Thus, the SV element in one form or another was considered in all eight practical lessons (4 stages consisted of 1 lesson each, and 1 stage covered four lessons). We do not consider the first of 9, the introductory lesson. In contrast, attention was paid to the SI element in only one lesson. The SV element took first place in the rating at the stage of the formative experiment, and the SI element dropped from the penultimate to the last place (figure 3).


PIC

Fig. 6: Study of individual design elements at stages 1-5 of the experimental study: stage 1 – problem-targeted, 2 – technical task, 3.1-3.4 – practical implementation, 4 – pre-defence, 5 – product presentation & summary.


Obviously, allocating a certain minimum of time is necessary for successfully mastering the element. Later, with increasing time (within the conducted short-term experiment), the level of understanding increases in most cases. However, this condition is insufficient, and two elements fall out of this pattern. The elements of RT and CR (used in five and four lessons) show slightly less progress than expected. In both cases, one needs to find logical explanations for this fact.

Only the classes of the third stage (table 2), during which the product was manufactured, are devoted to the CR element. It should be emphasised that all projects were individual. At the same time, as we know, the CR element is focused on teamwork. It includes constructive feedback from peers, teachers and experts, which improves project processes and work products. However, this is impossible when implementing individual projects; students can only exchange opinions with the teacher. The dotted connections for CR in figure 6 illustrate their incomplete performance. Thus, restrictions on the progress of the CR element were already designed in the structure of the experimental curriculum, and they were reflected in the obtained results.

The RT element, by default, implies deliberation. Students and teachers must consider what they are learning and how and why during the project. Reflecting on the content of the acquired knowledge helps students consolidate this knowledge. On the one hand, developed reflective thinking is the key to successfully mastering the element of RT. On the other hand, it is impossible to say a priori how much reflective thinking is developed among the respondents. Studying students ’ learning styles can provide additional information on this issue. For example, within the Felder-Soloman model of learning styles, students use active or reflective styles of understanding information.

3.3 Individual learning styles

The learning style preferences among all participating students were determined using the Felder-Soloman method. Separately, similar research was done for three teachers related to teaching the discipline in which the experiment was conducted. The average profiles for the student group and teachers are shown in figure 7.


PIC

Fig. 7: Average learning profiles of students and teachers who participated in the experiment. For clarity, the red line separating shaded inside and unshaded outside areas corresponds to the balance of styles in all four dimensions (5.5 points on a scale from 0 to 11 points).


The Mann-Whitney U test was applied again (table 6) to clarify in which cases the difference in styles is statistically significant. A significant difference between the teaching styles of teachers and students was observed only in the act/ref dimension (p = 0.035). In other dimensions, the value of p is far from the threshold value of 0.05.


Table 6: Mann-Whitney U-test (*) statistics for comparing the learning styles of teachers and students.










Parameters

Act Sen Vis Seq Ref Int Vrb Glo









Mann-Whitney U-test statistics

6.500 16.00015.50021.000 6.500 16.00015.50021.000









Z-value

-2.104 -.796 -.857 -.134 -2.104 -.796 -.857 -.134









Asymptotic significance (2-tailed), p

.035 .426 .391 .893 .035 .426 .391 .893









(*) The null hypothesis to be tested is that the two populations being compared have identical distributions


The perception of the concept of learning styles is still debatable due to the complexity of the object itself. On the one hand, some authors deny the value of using learning styles for the educational process [28]. On the other hand, most authors positively perceive the idea of educational preferences [5]. The situation is complicated by many style models that do not always agree with each other [5]. Active discussions concern the presence [3840] or absence [2933] of a correlation between styles and the level of academic achievement.

It is known from the literature that acquired educational advantages are pretty stable and, therefore, practically do not change during undergraduate studies [7]. At the same time, student profiles become more balanced when transitioning from bachelor’s to master’s [20]. Many publications emphasise a significant difference between the learning styles of students of different study fields [7] and nationalities [16], and students and teachers [3639].

Usually, teachers are more reflective and have global learning patterns than more active and sequential students. The inconsistency of styles is partly related to possible differences in the attitude of both parties to the educational process [7]. As already mentioned, the preference for an active learning style is typical of all undergraduate engineering majors. Further, in the master’s degree – postgraduate studies – teachers series, accents gradually shift towards the preference of the reflective style. The difference contributes to the efficiency of the assimilation of the material.

Typically, models of learning styles do not divide learning preferences into good and bad. Nevertheless, a certain balance between learning and teaching styles will contribute to the material’s effectiveness mastering. The optimal organisation of the educational process should be aimed at balancing teaching and learning styles rather than achieving absolute consistency between each teacher’s actions and students’ educational preferences. The balance creates conditions for students to develop the necessary learning styles in a group and does not lead to cognitive overload, discomfort in learning, fatigue, negative emotions, etc.

The current study confirmed teachers’ higher level of reflectiveness compared to students. Comprehension of information was the only statistically significant among the four dimensions of learning styles. The other three dimensions, including perception of information, sensors for information perception, and learning patterns, show no difference. As already mentioned, the preference for an active learning style is typical of all undergraduate engineering majors. However, the difference may also appear in other dimensions when studying larger samples.

Several works [31018] prove the existence of a direct correlation just between the level of reflectivity and academic achievements. Thus, the importance of meaningful reflection for academic achievement is recognised. Therefore, excessive activity and, at the same time, an undeveloped ability to reflect on the results obtained and planned, as well as failures in the implementation of projects, inhibit the formation of project competence. The relatively restrained progress achieved in the study of the RT element is probably related to the weak development of reflective thinking in bachelor students, which was found experimentally. It indicates an underestimation of this factor’s possible influence on currently active and developed experimental programs.

It is necessary to develop educational programs to strengthen students’ reflection in the learning process, improve academic success, and better prepare them for the future profession. To enhance the reflexivity of active students during training, one needs to apply reflective exercises: 1) survey after each stage of projecting to understand one’s activity; 2) keeping a diary in which observations of positive and negative moments in the design process are entered, to find solutions and improve the work; 3) at the beginning of each design stage, repeat the knowledge gained at the previous stages; 4) living project activity on one’s own experience, which solves a real problem.

4 Conclusions

1.
Educational technology has been developed to mould students’ project competence. The technology is based on a short-term training course (18 hours), organised as an academic module and incorporated into one of the active disciplines. During training, students independently choose a project topic from waste management, develop, execute and defend it. The final effectiveness of the approach is evaluated through two surveys using specially designed questionnaires at the beginning and end of the training.
2.
The proposed technology focuses on mastering the seven basic elements of PjBL, which are considered indicators of project competence formation. An essential part of the module is the organisation of regular and prompt feedback between students and teachers using short daily surveys via Google Forms. This tool allows teachers to adjust the training content and accents in time, depending on each lesson’s effectiveness. The proposed methodology significantly improved students’ understanding of the basic principles of PjBL. Statistically significant improvement was observed for four of the seven elements (CP, SV, RT and PP); for the other three (SI, AO and CR), progress was also observed but did not exceed the 0.05 significance level.
3.
The general trend of progress in learning was that the more classes were devoted to one or another projecting element, the more remarkable improvement was observed. In other words, minimal but sufficient attention to each element is necessary for achieving progress in its study. The SI element in the developed module was not given enough attention, so this element ended up in the last place in the progress rating.
4.
However, a certain level of attention seems necessary but insufficient for achieving appreciable progress in learning individual project elements. Progress was less than expected for two elements, which received quite a high level of attention during training. The non-ideal organisation of project learning and the unaccounted feature of the student’s learning style inhibited the mastery of some design elements, namely CR and RT.
5.
Mastery of the CR element includes active discussion among all participants. However, the chosen format of individual projects was not entirely favourable to broad discussions, limiting the possibilities of exchanging ideas in contrast to group projects. Accordingly, implementing group projects should contribute to a better understanding of the CR element.
6.
Studies of learning styles have shown the advantage of students’ active style of information comprehension over a reflective one. Developing reflective thinking is vital for mastering the RT element. This factor has become the reason for slowing down progress in mastering RT. The development of reflexivity due to the implementation of the proposed special exercises in the educational process is promising for eliminating the identified deficiency.
7.
The developed educational technology is not limited to the studied topic and has the potential for further development and expansion. Subsequent studies will aim to eliminate the shortcomings identified and listed in the previous item. In future, more respondents will be involved in the analysis to obtain more precise results.

References

[1]    Beloshenko, V., Voznyak, Y., Voznyak, A., Savchenko, B.: New approach to production of fiber reinforced polymer hybrid composites. Composites Part B: Engineering 112(1), 22–30 (mar 2017), https://doi.org/10.1016/j.compositesb.2016.12.030

[2]    Bhan, T., Otroshko, T., Shyshko, I.: Project technology as a key element of the formation of competence of an educational institution graduate. In: Ponomarova, G.F., Kharkivska, A.A., Petrychenko, L.O. (eds.) Enrichment of the Pedagogical Cluster of Educational Services: International Collective Monograph, pp. 362–379, OKTAN PRINT s. r. o., Kharkiv-Praha (2022), https://doi.org/10.46489/EOTPCOES-17-17-17

[3]    Cavilla, D.: The effects of student reflection on academic performance and motivation. SAGE Open 7(3), 1–13 (2017), https://doi.org/10.1177/2158244017733790

[4]    Chouprina, N.V.: Characteristics of “fast fashion” concept in fashion industry. Vlakna a Textil 21(1), 31–36 (mar 2014), URL https://er.knutd.edu.ua/handle/123456789/8999

[5]    Coffield, F., Moseley, D., Hall, E., Ecclestone, K.: Learning Styles and Pedagogy in Post 16 Education: A Critical and Systematic Review. The Learning and Skills Research Centre, London (Jan 2004)

[6]    Derkach, T., Khomenko, V.: Essential and toxic microelements in the medicinal remedy Hyperichi herba by different producers. Research Journal of Pharmacy and Technology 11(2), 466–474 (2018), https://doi.org/10.5958/0974-360X.2018.00086.0

[7]    Derkach, T.M., Shuhailo, Y.V.: Project-based learning for undergraduate engineering students minoring in textile technology and design. Journal of Physics: Conference Series 1840(1), 012042 (mar 2021), https://doi.org/10.1088/1742-6596/1840/1/012042

[8]    Derkach, T.M., Shuhailo, Y.V.: Adapting engineering education to challenges of sustainable development. IOP Conference Series: Earth and Environmental Science 1049(1), 012021 (jun 2022), https://doi.org/10.1088/1755-1315/1049/1/012021

[9]    Derkach, T.M., Starikova, O.O.: Variation of chemical composition of medicinal herbs of different producers. Journal of Chemistry and Technologies 27(1), 79–91 (2019), URL https://doi.org/10.15421/091909

[10]    Džubur, A., Lisica, D., Abdulahović, D., Ejubović, M.: Impact of social and psychological factors on academic achievement of university students. Med Glas (Zenica) 17(1), 224–238 (2020), https://doi.org/10.17392/1080-20

[11]    Dybkova, L.N.: The means of quick feedback implementation in the information educational environment. Information Technologies and Learning Tools 77(3), 130–144 (2013), https://doi.org/10.33407/itlt.v77i3.2828

[12]    Felder, R.M., Silverman, L.K.: Learning and teaching styles in engineering education. Engineering Education 78(7), 674–681 (jan 1988), URL https://www.researchgate.net/publication/257431200

[13]    Felder, R.M., Soloman, B.A.: Index of Learning Styles Questionnaire (2024), URL https://learningstyles.webtools.ncsu.edu/

[14]    Felder, R.M., Spurlin, J.: Applications, reliability and validity of the index of learning styles. International Journal of Continuing Engineering Education and Life-Long Learning 21(1), 103–112 (Jan 2005), URL https://www.researchgate.net/publication/279894244

[15]    Fu, B., Wang, S., Zhang, J., Hou, Z., Li, J.: Unravelling the complexity in achieving the 17 sustainable-development goals. National Science Review 6(3), 386–388 (03 2019), https://doi.org/10.1093/nsr/nwz038

[16]    Gryshchenko, I.M., Jin, L., Derkach, T.M., Tang, S.: Experience in teaching analytical chemistry in a joint English-language educational project of Chinese and Ukrainian universities. Journal of Physics: Conference Series 1946(1), 012008 (2021), https://doi.org/10.1088/1742-6596/1946/1/012008

[17]    Halkos, G., Gkampoura, E.C.: Where do we stand on the 17 Sustainable Development Goals? An overview on progress. Economic Analysis and Policy 70, 94–122 (2021), https://doi.org/10.1016/j.eap.2021.02.001

[18]    Hur, Y., Kim, S.: Different outcomes of active and reflective students in problem-based learning. Medical Teacher 29(1), e18–e21 (2007), https://doi.org/10.1080/01421590601045007

[19]    Jamieson, S.: Likert scales: how to (ab) use them. Medical Education 38(12), 1217–1218 (2004), https://doi.org/10.1111/j.1365-2929.2004.02012.x

[20]    Jiraporncharoen, W., Angkurawaranon, C., Chockjamsai, M., Deesomchok, A., Euathrongchit, J.: Learning styles and academic achievement among undergraduate medical students in Thailand. Journal of Educational Evaluation for Health Professions 12(38), 1–7 (jul 2015), https://doi.org/10.3352/jeehp.2015.12.38

[21]    Kalogiannakis, M., Papadakis, S., Zourmpakis, A.I.: Gamification in Science Education. A Systematic Review of the Literature. Education Sciences 11(1), 22 (2021), URL https://doi.org/10.3390/educsci11010022

[22]    Kovalchuk, V.N.: ICT tools application in pedagogical experiment: online questionnaire. Information Technologies and Learning Tools 35(3), 135–152 (2013), https://doi.org/10.33407/itlt.v35i3.832

[23]    Larmer, J., Mergendoller, J., Boss, S.: Setting the Standard for Project Based Learning. ASCD, Alexandria, USA (2015)

[24]     Litzinger, T.A., Lee, S.H., Wise, J.C., Felder, R.M.: A Psychometric Study of the Index of Learning Styles. Journal of Engineering Education 96(4), 309–319 (oct 2007), https://doi.org/10.1002/j.2168-9830.2007.tb00941.x

[25]    Melnykov, O.Y.: Automated information system for processing standards and developing educational programs for higher educational institutions. Information Technologies and Learning Tools 84(4), 302–321 (2021), https://doi.org/10.33407/itlt.v84i4.3584

[26]    Mensah, J.: Sustainable development: Meaning, history, principles, pillars, and implications for human action: Literature review. Cogent Social Sciences 5(1), 1653531 (2019), https://doi.org/10.1080/23311886.2019.1653531

[27]    Nasledov, A.: SPSS 20 Statistics & AMOS. Piter, St-Petersburg (2013)

[28]    Newton, P.M.: The learning styles myth is thriving in higher education. Frontiers in Psychology 6 (2015), https://doi.org/10.3389/fpsyg.2015.01908

[29]    Norasyikin, O., Mohamad, M.M., Paimin, A.N.: Dimension of learning styles and students’ academic achievement. Procedia - Social and Behavioral Sciences 204, 172–182 (Aug 2015), https://doi.org/10.1016/j.sbspro.2015.08.130

[30]    Papadakis, S., Kalogianakis, M., Sifaki, E., Monnier, A.: Editorial: The Impact of Smart Screen Technologies and Accompanied Apps on Young Children Learning and Developmental Outcomes. Frontiers in Education 6 (2021), https://doi.org/10.3389/feduc.2021.790534

[31]    Papadakis, S., Kiv, A.E., Kravtsov, H.M., Osadchyi, V.V., Marienko, M.V., Pinchuk, O.P., Shyshkina, M.P., Sokolyuk, O.M., Mintii, I.S., Vakaliuk, T.A., Azarova, L.E., Kolgatina, L.S., Amelina, S.M., Volkova, N.P., Velychko, V.Y., Striuk, A.M., Semerikov, S.O.: Unlocking the power of synergy: the joint force of cloud technologies and augmented reality in education. In: Semerikov, S.O., Striuk, A.M. (eds.) Joint Proceedings of the 10th Workshop on Cloud Technologies in Education, and 5th International Workshop on Augmented Reality in Education (CTE+AREdu 2022), Kryvyi Rih, Ukraine, May 23, 2022, CEUR Workshop Proceedings, vol. 3364, pp. 1–23, CEUR-WS.org (2022), URL https://ceur-ws.org/Vol-3364/paper00.pdf

[32]    Pashkevich, K.L., Kolosnichenko, M.V., Ostapenko, N.V.: Research of some physical and mechanical characteristics of suiting fabrics for designing the clothes. Vlakna a Textil 23(1), 3–8 (2016), URL https://er.knutd.edu.ua/handle/123456789/2016

[33]    Sabag, N., Trotskovsky, E.: An examination of the relationship between instructional strategies to learning styles distance and students’ achievements. The International Journal of Engineering Education 32(5(A)), 1903–1910 (2016), URL https://www.researchgate.net/publication/309671999

[34]    Strobel, J., van Barneveld, A.: When is PBL more effective? A Meta-synthesis of metaanalyses comparing PBL to conventional classrooms. Interdisciplinary Journal of Problem-Based Learning 3(1), 44–58 (spr 2009), https://doi.org/10.7771/1541-5015.1046

[35]    Swain, R.B., Yang-Wallentin, F.: Achieving sustainable development goals: predicaments and strategies. International Journal of Sustainable Development & World Ecology 27(2), 96–106 (2020), https://doi.org/10.1080/13504509.2019.1692316

[36]    Tulbure, C.: Learning styles, teaching strategies and academic achievement in higher education: A crosssectional investigation. Procedia - Social and Behavioral Sciences 33, 398–402 (2012), https://doi.org/10.1016/j.sbspro.2012.01.151

[37]    Tzagaraki, E., Papadakis, S., Kalogiannakis, M.: Teachers’ Attitudes on the Use of Educational Robotics in Primary School. In: Papadakis, S., Kalogiannakis, M. (eds.) STEM, Robotics, Mobile Apps in Early Childhood and Primary Education: Technology to Promote Teaching and Learning, p. 257—283, Springer Nature, Singapore (2022), https://doi.org/10.1007/978-981-19-0568-1_13

[38]    Widaningrum, D.L., Chie, H.H.: Felder-Silverman learning style model and the relationship with academic performance. Anima, Indonesian Psychological Journal 30(2), 88–100 (Jan 2015), https://doi.org/10.24123/aipj.v30i2.538

[39]    Willems, J.: Using learning styles data to inform e-learning design: A study comparing undergraduates, postgraduates and e-educators. Australasian Journal of Educational Technology 27(6), 863–880 (2011), https://doi.org/10.14742/ajet.917

[40]    Zagulova, D., Boltunova, V., Katalnikova, S., Prokofyeva, N., Synytsya, K.: Personalized e-learning: relation between Felder-Silverman model and academic performance. Applied Computer Systems 24(1), 25–31 (May 2019), https://doi.org/10.2478/acss-2019-0004

[41]    Zourmpakis, A.I., Kalogiannakis, M., Papadakis, S.: Adaptive Gamification in Science Education: An Analysis of the Impact of Implementation and Adapted Game Elements on Students’ Motivation. Computers 12(7), 143 (2023), https://doi.org/10.3390/computers12070143