An effective end-to-end modeling approach for mispronunciation detection

Tien Hong Lo, Shi Yan Weng, Hsiu Jui Chang, Berlin Chen

研究成果: 雜誌貢獻會議論文同行評審

1 引文 斯高帕斯(Scopus)

摘要

Recently, end-to-end (E2E) automatic speech recognition (ASR) systems have garnered tremendous attention because of their great success and unified modeling paradigms in comparison to conventional hybrid DNN-HMM ASR systems. Despite the widespread adoption of E2E modeling frameworks on ASR, there still is a dearth of work on investigating the E2E frameworks for use in computer-assisted pronunciation learning (CAPT), particularly for mispronunciation detection (MD). In response, we first present a novel use of hybrid CTC-Attention approach to the MD task, taking advantage of the strengths of both CTC and the attention-based model meanwhile getting around the need for phone-level forced-alignment. Second, we perform input augmentation with text prompt information to make the resulting E2E model more tailored for the MD task. On the other hand, we adopt two MD decision methods so as to better cooperate with the proposed framework: 1) decision-making based on a recognition confidence measure or 2) simply based on speech recognition results. A series of Mandarin MD experiments demonstrate that our approach not only simplifies the processing pipeline of existing hybrid DNN-HMM systems but also brings about systematic and substantial performance improvements. Furthermore, input augmentation with text prompts seems to hold excellent promise for the E2E-based MD approach.

原文英語
頁(從 - 到)3027-3031
頁數5
期刊Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
2020-October
DOIs
出版狀態已發佈 - 2020
事件21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020 - Shanghai, 中国
持續時間: 2020 十月 252020 十月 29

ASJC Scopus subject areas

  • 語言與語言學
  • 人機介面
  • 訊號處理
  • 軟體
  • 建模與模擬

指紋

深入研究「An effective end-to-end modeling approach for mispronunciation detection」主題。共同形成了獨特的指紋。

引用此