Development of an Algorithm and Module for Automatic Evaluation of Student Papers Based on Semantic Analysis of Text

Research objective. The main objectives of developing a module for automatic assessment of students’ papers based on the Russian grading scale system are:
– Increased grading efficiency: the automatic system can process a larger number of papers in less time than a lecturer, reducing the time spent on checking;
– Grading objectivity: the automated system is not subject to bias and other human factors, which ensures more objective grading of papers;
– Standardization of grading: the automated system provides a uniform approach to grading, making it more transparent and comparable;
– Reduced workload for lecturers: freeing lecturers from the routine work of checking papers allows them to devote more time to individual work with students.
Materials and methods. Various methods can be used to develop a module for automatic assessment of students’ work, such as:
Machine learning techniques: these techniques allow the module to learn from a set of examples where lecturers have already assessed students’ papers and automatically grade new papers.
Natural Language Processing (NLP) methods: these methods allow the module to understand the meaning of text and evaluate it against given criteria.
Expert systems methods: these methods allow the module to utilize the knowledge of experts in assessing students’ papers. For this project, we have chosen a combination of Levenshtein and Jaro index algorithms based on the method of assessing students’ knowledge in the Russian system.
Results. The following main results were obtained in the course of this research. The module of automatic evaluation of student papers has been successfully developed and implemented on the basis of the Russian system of evaluation scales. Test evaluations of papers showed high accuracy of the module’s predictions and reliability of its work. Comparison of the module with manual assessment confirmed its ability to give comparable results. The module proved to be useful for lecturers, providing them with the ability to quickly and objectively assess students’ papers. Integration of the module with existing learning management systems facilitates its implementation in the educational environment. All the obtained results testify to the high efficiency and prospectivity of the developed module.
Conclusion. As a result of this work, a module for automatic evaluation of students’ papers based on the Russian system of evaluation scales was developed and implemented. The main results of the work are that the module has been successfully tested and demonstrated high accuracy and reliability in the assessment of papers. The module has also demonstrated its usefulness for lecturers, providing them with an opportunity to quickly and objectively assess students’ papers. Due to its integration with existing learning management systems, the module can be easily implemented into the educational environment. In general, the results of the work confirm the effectiveness and prospects of using automatic assessment of students’ papers based on the Russian system of assessment scales.

1. Lapteva Ye.S. Ispol’zovaniye iskusstvennogo intellekta v obrazovatel’nykh onlayn-platformakh = Use of artificial intelligence in online educational platforms. 2022. (In Russ.)

2. Medvedeva N.A., Malkov N.G. et al. Professional and Public Accreditation as An Assessment of Agricultural Educational Program Quality in Russia. Journal of University Education. 2021.

3. Maksimova Ye.A, Nikitina G.A., Shchilova S.A. Realization of the linguodidactic potential of artificial intelligence methods. Izvestiya Saratovskogo universiteta. Novaya seriya. Seriya: Akmeologiya obrazovaniya. Psikhologiya razvitiya = News of Saratov University. New episode. Series: Acmeology of education. Developmental psychology. 2023; 12; 2(46): 114-122. (In Russ.)

4. Odintsova S.V. Development and implementation of a neural network algorithm for assessing learning outcomes in the discipline of mathematics. Simvol nauki = Symbol of Science. 2023. (In Russ.)

5. Babomuradov O. Pre-processing mechanism when analyzing text data. Miasto Przyszłości. 2023.

6. Karpenko O.M., Fokina V.N, Shirokova M.Ye. Didactics of the educational process. Edukologiya = Educology. 2020. (In Russ.)

7. El Mostafa H., Benabbou F. A deep learning based technique for plagiarism detection: a comparative study. Artificial Intelligence. 2020.

8. Saeed A.A.M., Taqa A.Y. A proposed approach for plagiarism detection in Article documents. Sinkron: jurnal dan penelitian teknik informatika. 2022; 1: 64-73.

9. Belikova K.M. Does Russia need a fixed percentage of originality and the originality of scientific works itself: reflections of a lawyer. Yuridicheskiye issledovaniya = Legal Research. 2023. (In Russ.)

10. Ivanovna B.S. Sovershenstvovaniye protsessa otsenki proizvedeniy iskusstva ekspertno-zakupochnoy komissiyey gosudarstvennogo muzeya = Improving the process of assessing works of art by the expert purchasing commission of the state museum. 2021. (In Russ.)

11. Bakhteyev O.Yu., Gafarov F.M., Grinshkun V.V. Digital platform for education. Rossiyskiy fond fundamental’nykh issledovaniy = Russian Foundation for Basic Research. 2022. (In Russ.)

12. Popova O.V. Intellektual’naya sistema upravleniya protsessami podderzhaniya parametrov tekhnicheskikh system = Intelligent process control system for maintaining the parameters of technical systems. 2022. (In Russ.)

13. Burnashev R.F., Murzamuratova U.B. Application of computer linguistics technologies in social networks and Internet marketing. Universum Universum. 2023. (In Russ.)

14. Iogolevich N.I., Lobodenko Ye.I Academic dishonesty of technical university students: the scale of the problem and solutions. Pedagogika. Voprosy teorii i praktiki = Pedagogy. Questions of theory and practice. 2020. (In Russ.)

15. Satayev M.Yu. Student, Faculty of Investigator Training, Moscow Academy of the Investigative Committee of the Russian Federation. Tekhnologii XXI veka v yurisprudentsii = Technologies of the 21st century in jurisprudence. 2023. (In Russ.)

16. Guo H., Li Y., Liu Y., Li W., Li X. Evaluation of string comparators for record linkage in Chinese environment. International Journal of Wavelets, Multiresolution and Information Processing. 2022.

17. Aghaee, M., Khoshgozaran A. Automatic essay scoring using a combination of lexical, syntactic and semantic features. International Journal of Artificial Intelligence and Education. 2018; 28(2): 185-202.

18. Zhil’tsova N.V. Razgranicheniye polisemii i omonimii diskursivnogo slova vot s pomoshch’yu metodov mashinnogo obucheniya na materiale bimodal’nogo korpusa ustnoy = Distinguishing between polysemy and homonymy of a discursive word using machine learning methods based on the material of a bimodal oral corpus. 2022. (In Russ.)