Central Asian Research Index

Polytechnic Bulletin. Intellect, Innovations, Investments Series, 2026

ON THE ALGORITHM FOR DEVELOPING A PRIVATE SET OF N-GRAMS FROM TEXT ELEMENTS

Mirshafi Gafurov, Ibrohimi Yusuf

Tajik Technical University named after academic M.S.Osimi

Аннотация · Abstract

This article describes the process of analyzing and processing textual information by developing a private set of n- grams (unigrams, bigrams, and trigrams) from text elements. This method is of key importance in fields such as Natural Language Processing (NLP), linguistic modeling, and cryptography for the transformation of text objects. The article presents algorithms for dividing these elements, in which unigrams - each text character as a separate unit - while bigrams and trigrams are divided into groups of two or three characters, respectively. If there are not enough characters to create n-grams, a special "zero place" character (ø) is added to the end of the text object. The C# programming language is used for the practical implementation of these algorithms, and examples of its program code are provided. A significant portion of the work is devoted to the creation of sets of unique n-grams. It is noted that the best method for this is the use of "hash sets" (HashSet), since it is more efficient in terms of execution speed and ease of implementation.

Сведения · Details