ASSESSMENT OF SEMANTIC SIMILARITY OF SENTENCES USING PROBABILISTIC MEAUSRES BASED ON EMBEDDINGS

K. A. Nikitenko, A. V. Zviagintseva

About the authors

K. A. Nikitenko, A. V. Zviagintseva

Никитенко Кирилл Андреевич – аспирант кафедры компьютерных технологий ФГБОУ ВО «ДонГУ», ул. Университетская, 24, Донецк, 283001, Россия. Область научных интересов: компьютерное зрение, машинное обучение, нейронные сети. Число научных публикаций – 5.

Звягинцева Анна Викторовна – д.т.н. доцент, профессор кафедры компьютерных технологий ФГБОУ ВО «ДонГУ », ул. Университетская, 24, Донецк, 283001, Россия. Область научных интересов: системный анализ, событийная и комплексная оценка; безопасность и управление социально- экономическими и техногенными системами; информационно -аналитические системы; обработка и анализ данных. Число научных публикаций – более 150.

Download PDF

UDC: 004.8+004.912
DOI: 10.24412/2413-7383-2025-4-39-229-240
Language: Russian

Abstract: The article addresses the problem of evaluating the semantic similar ity of sentences using probabilistic measures of word embeddings. An method is proposed based on comparing the probabilities of independent and d ependent events corresponding to the words within a sentence. The methods for constructing attention matrices a nd digitizing words are analyzed, and the choice of probabilistic embeddings as the core of the semantic representation is justified. A procedure is proposed for comparing probabilities derived from embeddings and attention weights, which enables formalization of semantic relations between words and sentences.

Keywords: semantic similarity, probabilistic model, attention matrix, embedding, probabilistic text analysis.

References

1. Minaee S. Kalchbrenner N. Cambria E. Nikzad N. Chenaghlu M. Gao J. Deep Learning-Based Text Classification: A comprehensive review // ACM Computing Surveys. no 54(3), 2021: 1–40.
2. Averin G.V. O veroyatnostnoi prirode smyslov v diskretnykh yazykovykh edinitsakh [On the Probabilistic Nature of Meanings in Discrete Linguistic Units]. Sistemnyi analiz i informatsionnye tekhnologii v naukakh o prirode i obshchestve. no 1(12)–2(13), 2017: 11–18.
3. Andrievskaya N.K. Gibridnaya intellektual'naya mera ocenki semanticheskoj blizosti // Problemy iskusstvennogo intellekta. №1(20), 2021: 4–17.
4. Kryukova K.V. L.A. Pankova, V.A. Pronina, V.S. Sukhoverov, L.B. Shipilina. Mery semanticheskoy blizosti v ontologii [Measures of semantic proximity in ontology] Probl. upravl. 2010. Issue 5: 2–14.
5. Erk K. The probabilistic turn in semantics and pragmatics // Annu. Rev. Linguist. 2022. 8:101–21.
6. Zheng Z. Wang Y. Huang Y. Song S. Yang M. Tang B. Xiong F. Li Z. Attention Heads of Large Language Models: A Survey // arXiv – 2024 – arXiv:2409.03752.
7. Sobchishen A.S. Zviagintseva A.V. Veroyatnostno-smyslovye modeli otsifrovannykh tekstovykh dannykh [Probabilistic-Semantic Models of Digitized Text Data]. Materialy konferentsii “Matematicheskoe i komp'yuternoe modelirovanie v ekonomike, strakhovanii i upravlenii riskami”. no 9, 2024: 55–59. Оценка семантической схожести предложений… Н
8. Reimers N. Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks // arXiv – 2019 – arXiv:1908.10084.
9. Gao T. Yao X. Chen D. SimCSE: Simple Constrastive Learning of Sentence Embeddings // arXiv – 2021 – arXiv:2104.08821.
10. Shen L. Jiang H. Liu L. Shi S. Sen2Pro: A Probabilistic Perspective to Sentence Embedding from Pre- trained Language Models // arXiv – 2023 – arXiv:2306.02247.
11. Vaswani A. Shazeer N. Parmar N. Uszkoreit J. Jones L. Gomez A. Kaiser L. Polosukhin I. Attention is all you need // arXiv – 2017 – arXiv:1706.03762.
12. Yoda S. Tsukagoshi H. Sasano R. Takeda K. Sentence Representations via Gaussian Embedding // arXiv – 2023 – arXiv:2305.12990v2.
13. Sait proekta “Otkrytyi korpus” (OpenCorpora) russkogo yazyka [Project website “OpenCorpora” of the Russian language]. – Elektronnyi resurs. URL: https://opencorpora.org/ (accessed November 1, 2025).
14. Chun S. Joon S. Sampaio R. Kalantidis Y. Larlus D. Probabilistic Embeddings for Cross-Modal Retrieval // arXiv – 2021 – arXiv:2101.05068.
15. Abdelwahab A. Landwehr N. Deep distributional sequence embeddings based on a Wasserstein loss // arXiv – 2019 – arXiv:1912.01933.
16. Botha J. Probabilistic modeling of morphologically rich languages // arXiv – 2015 – arXiv:1508.04271.
17. Bondarenko V.I. Eliseev V.O. Ermolenko T.V. Analiz effektivnosti glubokikh yazykovykh modelei dlya zadachi opredeleniya tonal'nosti russkoyazychnykh tekstov [Analysis of the Effectiveness of Deep Language Models for Sentiment Analysis of Russian Texts]. Problemy iskusstvennogo intellekta. no 1(32), 2025: 51–62.
18. Nikitenko K.A. Zviagintseva A.V. Interpretiruemost' neirosemanticheskikh modelei pri ikh primenenii v

Issue: 4(39)'2025
Section: SYSTEM ANALYSIS, CONTROL AND INFORMATION PROCESSING, STATISTICS
How to cite: K. A. Nikitenko, A. V. Zviagintseva. ASSESSMENT OF SEMANTIC SIMILARITY OF SENTENCES USING PROBABILISTIC MEAUSRES BASED ON EMBEDDINGS // Problems of Artificial Intelligence. - 2025. - № 4 (39). - P. 229-240. - https://paijournal.guiaidn.ru/en/2025/4(39)-20.html

Back to issue 4(39)'2025