AUTOMATIC SCORING OF SHORT-TEXT ANSWERS USING NATURAL LANGUAGE PROCESSING (NLP)
Keywords:
natural language processing, NLP, automatic scoring, short-text answers, artificial intelligence, machine learning, BERT, semantic analysis.Abstract
This article examines the possibilities of automatically scoring short-text answers provided by school pupils and university students in the educational process through the use of Natural Language Processing (NLP) technologies. Traditional assessment methods involve several challenges, including the considerable amount of time required from teachers, the emergence of subjectivity during the assessment process, and variations in the way answers are expressed. NLP technologies make it possible to reduce these challenges, identify the semantic relevance of answers, and accelerate the assessment process. The article analyses the mechanisms of text preprocessing, semantic similarity detection, the BERT architecture, and the use of large language models.
References
1.Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 4171–4186. https://doi.org/10.18653/v1/N19-1423
2.Bexte, M., Horbach, A., & Zesch, T. (2024). Strengths and weaknesses of automated scoring of free-text student answers. Informatik Spektrum, 47, 78–86. https://doi.org/10.1007/s00287-024-01573-z
3.Jiang, L., & Bosch, N. (2024). Short answer scoring with GPT-4. Proceedings of the Eleventh ACM Conference on Learning @ Scale, 438–442. https://doi.org/10.1145/3657604.3664685
4.Clauser, B. E., Yaneva, V., Baldwin, P., Ha, L. A., & Mee, J. (2024). Automated scoring of short-answer questions: A progress report. Applied Measurement in Education, 37(3), 209–224. https://doi.org/10.1080/08957347.2024.2386945
5.Ormerod, C., Lottridge, S., Harris, A. E., Patel, M., van Wamelen, P., Kodeswaran, B., Woolf, S., & Young, M. (2023). Automated short answer scoring using an ensemble of neural networks and latent semantic analysis classifiers. International Journal of Artificial Intelligence in Education, 33(3), 467–496. https://doi.org/10.1007/s40593-022-00294-2
6.Zesch, T., Horbach, A., & Zehner, F. (2023). To score or not to score: Factors influencing performance and feasibility of automatic content scoring of text responses. Educational Measurement: Issues and Practice, 42, 44–58. https://doi.org/10.1111/emip.12544