DEVELOPMENT OF AN INTELLIGENT ASSISTANT FOR AN EDUCATIONAL INSTITUTION

V. O. Eliseev1, V. I. Bondarenko2, A. Y. Maksimova1

About the authors

V. O. Eliseev1, V. I. Bondarenko2, A. Y. Maksimova1

Елисеев Вадим Олегович – стажер-исследователь лаборатории интеллектуальных систем Федерального государственного бюджетного научного учреждения Институт прикладной математики и механики. Область научных интересов: искусственный интеллект, машинное обучение, нейронные сети, обработка естественного языка, генеративные и большие языковые модели.

Бондаренко Виталий Иванович – кандидат технических наук, доцент кафедры компьютерных технологий физико-технического факультета, Федерального государ- ственного бюджетного образовательного учреждения высшего образования "Донецкого Государственного Университета". Область научных интересов: искусственный интеллект, интеллектуальный анализ данных, машинное обучение, математическое моделирование гидро- и теплофизических процессов, разработка пользовательских интерфейсов для прикладных программ моделирования.

Максимова Александра Юрьевна – кандидат технических наук, заведующий лабораторией интеллектуальных систем, Федерального государственного бюджетного научного учреждения Институт прикладной математики и механики. Область научных интересов: машинное обучение, анализ данных, распознавание образов, миварный логический вывод, нейросетевые модели.

Download PDF

UDC: 004.912
DOI: 10.24412/2413-7383-2025-4-39-36-48
Language: Russian

Abstract: The article proposes the method for development intelligent assistants for educational institutions based on Large Language Models (LLMs). This enables users to receive informational responses about an institution within a min ute, as opposed to the hours or days typically required. The method was implemented to create an assistant for Donetsk State University (DonSU). The study involved selecting a method for integrating corporate data into an LLM, followed by the direct collection and preprocessing of this data. The final system is a standard Retrieval-Augmented Generation (RAG) pipeline, which utilizes a fine-tuned LLaMA-3.1-8b model as the generator and Giga-Embeddings-instruct as the retriever. To evaluate the performance of the generator model, we used token-level similarity metrics and assessed the semantic similarity between the assistant's responses and the target answers for a set of predefined scenarios. The source code for all implemented steps and the final version of the DonSU assistant are open-source what makes the proposed approach applicable for other educational institutions.

Keywords: large language model, natural language processing, question answering, Retrieval-Augmented Generation, LLAMA, fine-tuning.

References

1. Wu S. et al. Bloomberggpt: A large language model for finance //arXiv preprint arXiv:2303.17564. – 2023.
2. Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. Don’t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964, 2020.
3. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen-tau Yih, Tim Rockt ¨ aschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474, 2020.
4. Hu, E. Shen, Y. Wallis, P. Allen-Zhu, Z. Li, Y. Wang, S. Wang, L. & Chen, W. Lora: Low-rank adaptation of large language models, International Conference On Learning Representations. (2021).
5. Andrievskaya N. K. Hybrid Intelligent Measure for Assessing Semantic Similarity // Problems of Artificial Intelligence. – 2021. – No. 1(20). – Pp. 4-17.
6. Scott Barnett, Stefanus Kurniawan, Srikanth Thudumu, Zach Brannelly, and Mohamed Abdelrazek. Seven failure points when engineering a retrieval augmented generation system. In Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering-Software Engineering for AI, pp. 194–199, 2024.
7. Ouyang, L. Wu, J. Jiang, X. Almeida, D. Wainwright, C. Mishkin, P. Zhang, C. Agarwal, S. Slama, K. Ray, A. & Others Training language models to follow instructions with human feedback, Advances In Neural Information Processing Systems. 35 pp. 27730-27744 (2022).
8. Eliseev, V. Maksimova, A. Bondarenko, V. Transforming raw corporate texts into instruction dataset for fine-tuning generator within a RAG system. System Informatics 2025, 27, 77-92.
9. Zhang T. et al. Raft: Adapting language model to domain specific rag //First Conference on Language Modeling. – 2024.
10. Liu W. et al. RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions //arXiv preprint arXiv:2501.00353. – 2024.
11. Methodical Guidelines: Methodical Guidelines for the Presentation of Information about a Higher- Education Institution in Open Sources, Taking into Account Compliance with Education-Legislation Requirements (effective 01.09.2024) // Federal Service for Supervision in Education and Science, 2024.
12. Eliseev Vadim. Website Crawler [Электронный ресурс]. – URL: https://github.com/EliseevVadim/WebsiteCrawler.
13. Team G. et al. Gemma 2: Improving open language models at a practical size //arXiv preprint arXiv:2408.00118. – 2024.
14. Eliseev Vadim. Texts Estimator [Electronic resource]. – URL: https://github.com/EliseevVadim/TextsEstimator.
15. Lee M. H. J. Examining the Robustness of Homogeneity Bias to Hyperparameter Adjustments in GPT-4 //arXiv preprint arXiv:2501.02211. – 2025.
16. Eliseev Vadim. Instructions Generator [Electronic resource]. – URL: https://github.com/EliseevVadim/InstructionsGenerator.
17. Giga-Embeddings-instruct | HuggingFace [Электронный ресурс]. – URL: https://huggingface.co/ai- sage/Giga-Embeddings-instruct.
18. Kaplan J. et al. Scaling laws for neural language models //arXiv preprint arXiv:2001.08361. – 2020.
19. Rodrigues J. Branco A. Meta-prompting Optimized Retrieval-Augmented Generation //EPIA Conference on Artificial Intelligence. – Cham: Springer Nature Switzerland, 2024. – С. 203-214.
20. Huerta-Enochian M. Ko S. Y. Instruction Fine-Tuning: Does Prompt Loss Matter? //arXiv preprint arXiv:2401.13586. – 2024.
21. Dettmers T. et al. Qlora: Efficient finetuning of quantized llms //Advances in neural information processing systems. – 2023. – Т. 36. – С. 10088-10115.
22. Reiter E. A structured review of the validity of BLEU //Computational Linguistics. 2018. Т. 44. №. 3. – С. 393-401.
23. Lin C. Y. Rouge: A package for automatic evaluation of summaries //Text summarization branches out. – 2004. – С. 74-81.
24. Eliseev Vadim. DonSU LLaMa Family [Electronic resource]. – URL: https://huggingface.co/collections/insidious316/donsu-llama-family.
25. Zhao E. et al. Quantitative Analysis of Performance Drop in DeepSeek Model Quantization //arXiv preprint arXiv:2505.02390. – 2025.
26. Kavathekar I. et al. Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling //arXiv preprint arXiv:2504.19277. – 2025.
27. Eliseev Vadim. DonSU Intelligent Assistant [Electronic resource]. URL:

Issue: 4(39)'2025
Section: ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING
How to cite: V. O. Eliseev1, V. I. Bondarenko2, A. Y. Maksimova1. DEVELOPMENT OF AN INTELLIGENT ASSISTANT FOR AN EDUCATIONAL INSTITUTION // Problems of Artificial Intelligence. - 2025. - № 4 (39). - P. 36-48. - https://paijournal.guiaidn.ru/en/2025/4(39)-4.html

Back to issue 4(39)'2025