TY - GEN
T1 - Class-Level Error Analysis and Systematic Evaluation of CNN-Based Models for French Mispronunciation Detection
AU - Alva Alvarado, Alberto
AU - De-la-Torre, Miguel
AU - Akber Dewan, M. Ali
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2027.
PY - 2027
Y1 - 2027
N2 - Accurate pronunciation assessment remains a critical challenge in second language learning, particularly for French, where subtle phonological differences can significantly affect intelligibility. Although deep learning approaches have shown promising results in mispronunciation detection, the impact of architectural and hyperparameter variations on model behavior is not yet fully understood, especially at the class level. This study presents a systematic experimental analysis of convolutional neural network (CNN) configurations for French mispronunciation detection using a balanced dataset of Spanish-speaking learners annotated into four pronunciation categories. The analysis includes controlled variations in hyperparameters, the incorporation of Batch Normalization, increased model capacity, label smoothing, and extended training up to 200 epochs. Results show that the baseline CNN achieves strong and stable performance, while most modifications do not produce consistent improvements in macro F1-score. Batch Normalization enhances training stability but has limited impact on generalization, and increasing model capacity yields only marginal gains. In contrast, label smoothing degrades performance. Extended training experiments further reveal early convergence and clear overfitting behavior. A detailed class-level analysis indicates that the primary limitation lies in the confusion between phonetically similar error categories, particularly Error Class 1 and Error Class 2. These findings suggest that improving mispronunciation detection requires focusing on feature representation and class discrimination rather than increasing model complexity.
AB - Accurate pronunciation assessment remains a critical challenge in second language learning, particularly for French, where subtle phonological differences can significantly affect intelligibility. Although deep learning approaches have shown promising results in mispronunciation detection, the impact of architectural and hyperparameter variations on model behavior is not yet fully understood, especially at the class level. This study presents a systematic experimental analysis of convolutional neural network (CNN) configurations for French mispronunciation detection using a balanced dataset of Spanish-speaking learners annotated into four pronunciation categories. The analysis includes controlled variations in hyperparameters, the incorporation of Batch Normalization, increased model capacity, label smoothing, and extended training up to 200 epochs. Results show that the baseline CNN achieves strong and stable performance, while most modifications do not produce consistent improvements in macro F1-score. Batch Normalization enhances training stability but has limited impact on generalization, and increasing model capacity yields only marginal gains. In contrast, label smoothing degrades performance. Extended training experiments further reveal early convergence and clear overfitting behavior. A detailed class-level analysis indicates that the primary limitation lies in the confusion between phonetically similar error categories, particularly Error Class 1 and Error Class 2. These findings suggest that improving mispronunciation detection requires focusing on feature representation and class discrimination rather than increasing model complexity.
KW - Class-level analysis
KW - Convolutional neural networks
KW - French mispronunciation detection
KW - Hyperparameter study
KW - Mel-spectrograms
UR - https://www.scopus.com/pages/publications/105053567578
U2 - 10.1007/978-3-032-37444-8_4
DO - 10.1007/978-3-032-37444-8_4
M3 - Published Conference contribution
AN - SCOPUS:105053567578
SN - 9783032374431
T3 - Communications in Computer and Information Science
SP - 46
EP - 58
BT - Artificial Intelligence – COMIA 2026 - 18th Mexican Congress, 2026, Proceedings
A2 - Ponce, Hiram
A2 - Escalante, Hugo Jair
A2 - Vazquez, Roberto A.
T2 - 18th Mexican Congress on Artificial Intelligence, COMIA 2026
Y2 - 25 May 2026 through 29 May 2026
ER -