Central AsianResearch Index

IMPROVING THE VOICE RECOGNITION SYSTEM FOR CHILDREN IN KAZAKH THROUGH ADDITIONAL TRAINING (FINE-TUNING)

This article describes the process of fine-tuning (retraining) the Whisper-tiny model, proposed by OpenAI, to enhance an automatic speech recognition (ASR) system adapted for children's speech in Kazakh. The subject of the study is an audio corpus consisting of words and short phrases in Kazakh spoken by children aged 5 to 12 years. A two-stage training strategy was employed: the first stage used a dataset of 235 audio samples, and the second stage used an expanded dataset of 1936 samples. The data was resampled to a frequency of 16 kHz and converted to mono format. Transcriptions were cleaned according to standardized rules, and audio-text alignment was ensured through Excel metadata and automatic verification (using difflib). The fine-tuning of the model was performed based on the openai/whisper-tiny architecture, using a feature extractor based on mel-spectrograms, 4-layer encoder-decoder transformer blocks, and special tokens <|kk|> and <|transcribe|>. The training was conducted in the Google Colab environment with the assistance of an NVIDIA T4 GPU, utilizing the AdamW optimizer and cross-entropy loss. The performance of the model was evaluated using the Word Error Rate (WER) and Accuracy metrics. As a result, the WER decreased from 0.80 to 0.42, showing an improvement of approximately 47.5%, while the accuracy increased from 20.43% to 57.95% (a 183% improvement). For single-word samples, the exact match accuracy reached 66.73%, while the error rate remained high for multi-word phrases. This work empirically demonstrates the effectiveness of the two-stage fine-tuning strategy for low-resource Kazakh, particularly in the domain of children's speech recognition, and lays the foundation for further improvements through dataset expansion, testing larger Whisper models, and incorporating data augmentation and regularization methods. Keywords: Whisper-tiny, fine-tuning, Kazakh language, automatic speech recognition (ASR), children's speech, Word Error Rate (WER), accuracy, deep learning, transfer learning, chalk spectrogram, dataset expansion. Financing. This study was funded by the grant project «Study of automatic generation of parallel speech corpora of Turkic languages and their use for neural models» (grant number IRN AP23488624) of the Ministry of Science and Higher Education of the Republic of Kazakhstan.

Сведения

Опубликовано
2026 · № 2
Язык
Английский
ISSN
1991-346X · 2518-1726
Издатель
Central Asian Academic Research Center LLP