Assessing the effectiveness  of research and academic staff's scientific activity: definition of criteria and indicators

Serhiy O. Semerikov; Oleg M. Spirin; Tetiana A. Vakaliuk; Iryna S. Mintii; Svitlana M. Ivanova; Oleksandr M. Shymon

doi:10.31812/ed.605

Authors

Serhiy O. Semerikov Kryvyi Rih State Pedagogical University https://orcid.org/0000-0003-0789-0272
Oleg M. Spirin Institute for Digitalisation of Education of the NAES of Ukraine https://orcid.org/0000-0002-9594-6602
Tetiana A. Vakaliuk Zhytomyr Polytechnic State University https://orcid.org/0000-0001-6825-4697
Iryna S. Mintii Institute for Digitalisation of Education of the NAES of Ukraine https://orcid.org/0000-0002-3613-9202
Svitlana M. Ivanova Institute for Digitalisation of Education of the NAES of Ukraine https://orcid.org/0000-0002-3613-9202
Oleksandr M. Shymon Institute for Digitalisation of Education of the NAES of Ukraine https://orcid.org/0000-0001-7009-2682

DOI:

https://doi.org/10.31812/ed.605

Keywords:

scientific activity, effectiveness assessment, criteria and indicators, project-competition, scientific-publication, scientometrics, altmetrics, expert contributions, representative science, academic community

Abstract

This paper highlights the crucial need to establish clear criteria and indicators for assessing the effectiveness of the scientific activity of research and academic staff. By carefully examining different dimensions of research activity, this study identifies and describes a comprehensive set of criteria and relevant indicators. These include involvement in project competitions (preparation, participation, and student engagement), scholarly publication outputs (journals, conferences, monographs, and student mentoring), scientometric measures (indexing and citations), altmetric considerations (digital repositories, online presence, and social media impact), expert contributions (peer review, committee participation, editorial roles, and conference organization), and representative scholarly achievements (dissertation defense, academic and honorary titles, awards, supervision, international engagement, and language skills). It is emphasized that these identified criteria can be categorized into publication and dissemination, exploitation, and impact on the academic community. By using this comprehensive framework, educational institutions can effectively assess the quality, impact, and contribution of the activities of research and academic staff, thereby promoting research excellence and educational progress.

Downloads

Download data is not yet available.

Abstract views: 459 / PDF (Ukrainian) views: 239

References

Stratehiia rozvytku vyshchoi osvity v Ukraini na 2021-2031 roky [Strategy for the Development of Higher Education in Ukraine for 2021-2031] (2020), URL https://mon.gov.ua/storage/app/media/rizne/2020/09/25/rozvitku-vishchoi-osviti-v-ukraini-02-10-2020.pdf

Rozporiadzhennia No 04 vid 25 travnia 2021 roku [Order No 04 of May 25, 2021] (2021), URL https://kdpu.edu.ua/pro-nas/dokumenty-universytetu/rozporiadzhennia/15832-rozporiadzhennia-04-vid-25-travnia-2021-roku.html DOI: https://doi.org/10.2307/j.ctv24650hx.6

Bykov, V.Y., Spirin, O.M., Ivanova, S.M., Vakaliuk, T.A., Mintii, I.S., Kilchenko, A.V.: Scientometric indicators for evaluating the effectiveness of pedagogical research of scientific institutions and educational institutions. Information Technologies and Learning Tools 86(6), 289–312 (2021), https://doi.org/10.33407/itlt.v86i6.4656 DOI: https://doi.org/10.33407/itlt.v86i6.4656

Cao, R., Geng, Y., Xu, X., Wang, X.: How does duplicate tweeting boost social media exposure to scholarly articles? Journal of Informetrics 16(1) (2022), https://doi.org/10.1016/j.joi.2022.101249 DOI: https://doi.org/10.1016/j.joi.2022.101249

Masic, I., Begic, E.: Evaluation of scientific journal validity, it’s articles and their authors. Studies in Health Technology and Informatics 226, 9–14 (2016), https://doi.org/10.3233/978-1-61499-664-4-9

Moral-Muñoz, J.A., Herrera-Viedma, E., Santisteban-Espejo, A., Cobo, M.J.: Software tools for conducting bibliometric analysis in science: An up-to-date review. Profesional De La Informacion 29(1) (2020), https://doi.org/10.3145/epi.2020.ene.03 DOI: https://doi.org/10.3145/epi.2020.ene.03

Morze, N.V., Buinytska, O.P., Smirnova, V.A.: Designing a rating system based on competencies for the analysis of the university teachers’ research activities. CTE Workshop Proceedings 9, 139–153 (Mar 2022), https://doi.org/10.55056/cte.109 DOI: https://doi.org/10.55056/cte.109

Salinas, M., Giorgi, D., Ponchio, F., Cignoni, P.: ReviewerNet: A visualization platform for the selection of academic reviewers. Computers and Graphics 89, 77–87 (2020), https://doi.org/10.1016/j.cag.2020.04.006 DOI: https://doi.org/10.1016/j.cag.2020.04.006

Shirazi, M.S., Valinejadi, A.: Investigating of Association between Altmetrics Activity Indicators and Citation Quality Indicators in Iranian Medical Journals. International Journal of Preventive Medicine 12(1), 156 (2021), URL https://www.ijpvmjournal.net/article.asp?issn=2008-7802;year=2021;volume=12;issue=1;spage=156;epage=156;aulast=Shirazi;type=0

Singh, V.K., Srichandan, S.S., Lathabai, H.H.: ResearchGate and Google Scholar: How much do they differ in publications, citations and different metrics and why? Scientometrics 127(3), 1515–1542 (2022), https://doi.org/10.1007/s11192-022-04264-2 DOI: https://doi.org/10.1007/s11192-022-04264-2

Spirin, O.M., Vakaliuk, T.A.: Criteria of open web-operated technologies of teaching the fundamentals of programs of future teachers of informatics. Information Technologies and Learning Tools 60(4), 275–287 (2017), https://doi.org/10.33407/itlt.v60i4.1815 DOI: https://doi.org/10.33407/itlt.v60i4.1815

Vakaliuk, T.A., Ivanova, S.M., Kilchenko, A.V.: Electronic portfolio as a tool of reflecting the results of scientific and pedagogical activities of teachers of higher education institutions. Scientific Bulletin of Uzhhorod University. Series: «Pedagogy. Social Work» 1(48), 53–58 (2021), https://doi.org/10.24144/2524-0609.2021.48.53-58 DOI: https://doi.org/10.24144/2524-0609.2021.48.53-58

Vakaliuk, T.A., Mintii, I.S., Hamaniuk, V.A., Ivanova, S.M.: Recording the practice of teaching staff in project activities and the preparation of winners in competitions and olympiads. Scientific innovations and advanced technologies 4(6), 22–34 (2022), https://doi.org/10.52058/2786-5274-2022-4(6)-22-34 DOI: https://doi.org/10.52058/2786-5274-2022-4(6)-22-34

Vakaliuk, T.A., Spirin, O.M., Mintiy, I.S., Ivanova, S.M., Novytska, T.L.: Scientometric indicators for evaluating the effectiveness of pedagogical research of scientists and research and teaching staff. Modern Information Technologies and Innovation Methodologies of Education in Professional Training Methodology Theory Experience Problems 60, 167–184 (2021), https://doi.org/10.31652/2412-1142-2021-60-167-184 DOI: https://doi.org/10.31652/2412-1142-2021-60-167-184

Wiechetek, L., Pastuszak, Z.: Academic social networks metrics: An effective indicator for university performance? Scientometrics 127(3), 1381–1401 (2022), https://doi.org/10.1007/s11192-021-04258-6 DOI: https://doi.org/10.1007/s11192-021-04258-6

Xu, Z., Yang, D., Chen, B.: Career difficulties that Chinese academic journal editors face and their causes. Journal of Scholarly Publishing 52(4), 212–232 (2021), https://doi.org/10.3138/JSP.52.4.02 DOI: https://doi.org/10.3138/jsp.52.4.02

Yu, X., Wu, S., Chen, W., Zheng, W., Huang, M., Yang, L., Zhou, S.: Exploring the associations between perceived organizational support and job burnout among chinese academic journal editors: A moderated mediation model. International Journal of Environmental Research and Public Health 18(22) (2021), https://doi.org/10.3390/ijerph182212167 DOI: https://doi.org/10.3390/ijerph182212167