Improving End-To-End Modeling for Mispronunciation Detection with Effective Augmentation Mechanisms

Tien Hong Lo, Yao Ting Sung, Berlin Chen

研究成果: 書貢獻/報告類型會議論文篇章

2 引文 斯高帕斯(Scopus)

摘要

Recently, end-to-end (E2E) models, which allow to take spectral vector sequences of L2 (second-language) learners' utterances as input and produce the corresponding phone-level sequences as output, have attracted much research attention in developing mispronunciation detection (MD) systems. However, due to the lack of sufficient labeled speech data of L2 speakers for model estimation, E2E MD models are prone to overfitting in relation to conventional ones that are built on DNN-HMM acoustic models. To alleviate this critical issue, we in this paper propose two modeling strategies to enhance the discrimination capability of E2E MD models, each of which can implicitly leverage the phonetic and phonological traits encoded in a pretrained acoustic model and contained within reference transcripts of the training data, respectively. The first one is input augmentation, which aims to distill knowledge about phonetic discrimination from a DNN-HMM acoustic model. The second one is label augmentation, which manages to capture more phonological patterns from the transcripts of training data. A series of empirical experiments conducted on the L2-ARCTIC English dataset seem to confirm the efficacy of our E2E MD model when compared to some top-of-the-line E2E MD models and a classic pronunciation-scoring based method built on a DNN-HMM acoustic model.

原文英語
主出版物標題2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2021 - Proceedings
發行者Institute of Electrical and Electronics Engineers Inc.
頁面1049-1055
頁數7
ISBN(電子)9789881476890
出版狀態已發佈 - 2021
事件2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2021 - Tokyo, 日本
持續時間: 2021 12月 142021 12月 17

出版系列

名字2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2021 - Proceedings

會議

會議2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2021
國家/地區日本
城市Tokyo
期間2021/12/142021/12/17

ASJC Scopus subject areas

  • 人工智慧
  • 電腦視覺和模式識別
  • 訊號處理
  • 儀器

指紋

深入研究「Improving End-To-End Modeling for Mispronunciation Detection with Effective Augmentation Mechanisms」主題。共同形成了獨特的指紋。

引用此