DEEP MULTIMODAL FUSION OF RESPIRATORY ACOUSTICS AND METADATA FOR AUTOMATED CRACKLE AND WHEEZE CLASSIFICATION
Automated interpretation of adventitious respiratory sounds is complicated by short, nonstationary acoustic events, heterogeneous stethoscope hardware, variable chest locations, and pronounced class imbalance. This study presents a reproducible deep multimodal architecture that jointly models respiratory acoustics and acquisition-context metadata for crackle and wheeze detection. Respiratory cycles are obtained from the open ICBHI 2017 Respiratory Sound Database, which contains 6,898 expert-annotated cycles from 920 recordings and 126 subjects. Each cycle is resampled, amplitude-normalized, pre-emphasized, decomposed by harmonic-percussive source separation, and mapped over 80-2,000 Hz into a two-channel 64-bin per-channel energy normalized Mel representation. A convolutional branch produces a 128-dimensional audio embedding, while device identity, grouped auscultation location, cycle duration, and relative temporal position form a 14-dimensional metadata representation. Late fusion yields two sigmoid probabilities, one for crackle and one for wheeze; threshold pairs map these event probabilities to the four conventional classes: normal, crackle, wheeze, and both. The supplied development traces show a reduction in training loss from approximately 0.575 to 0.377, an increase in training precision from 0.468 to 0.748, and an increase in training AUROC from 0.630 to 0.887 over ten epochs. Training recall increased from 0.155 to 0.650 through epoch nine. These trajectories establish learnability but do not constitute patient-independent test evidence. Accordingly, the manuscript defines a leakage-resistant subject-level evaluation, calibration, ablation, and metadata-shift protocol required for confirmatory reporting. The proposed factorized output and explicit acquisition-context branch provide an interpretable foundation for robust respiratory event classification while preserving a clear boundary between preliminary training evidence and clinical generalization.
Сведения
- Журнал
- Вестник КазАТК
- Опубликовано
- 2026 · Т. 4, № 145, С. 334-347
- Язык
- Английский
- ISSN
- 1609-1817 · 2790-5802
- Издатель
- JSC Academy of Logistics and Transport