Improving the Intelligibility of Speech for Simulated Electric and Acoustic Stimulation Using Fully Convolutional Neural Networks

Natalie Yu Hsien Wang, Hsiao Lan Sharon Wang, Tao Wei Wang, Szu Wei Fu, Xugan Lu, Hsin Min Wang, Yu Tsao*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

15 Citations (Scopus)


Combined electric and acoustic stimulation (EAS) has demonstrated better speech recognition than conventional cochlear implant (CI) and yielded satisfactory performance under quiet conditions. However, when noise signals are involved, both the electric signal and the acoustic signal may be distorted, thereby resulting in poor recognition performance. To suppress noise effects, speech enhancement (SE) is a necessary unit in EAS devices. Recently, a time-domain speech enhancement algorithm based on the fully convolutional neural networks (FCN) with a short-Time objective intelligibility (STOI)-based objective function (termed FCN(S) in short) has received increasing attention due to its simple structure and effectiveness of restoring clean speech signals from noisy counterparts. With evidence showing the benefits of FCN(S) for normal speech, this study sets out to assess its ability to improve the intelligibility of EAS simulated speech. Objective evaluations and listening tests were conducted to examine the performance of FCN(S) in improving the speech intelligibility of normal and vocoded speech in noisy environments. The experimental results show that, compared with the traditional minimum-mean square-error SE method and the deep denoising autoencoder SE method, FCN(S) can obtain better gain in the speech intelligibility for normal as well as vocoded speech. This study, being the first to evaluate deep learning SE approaches for EAS, confirms that FCN(S) is an effective SE approach that may potentially be integrated into an EAS processor to benefit users in noisy environments.

Original languageEnglish
Article number9281097
Pages (from-to)184-195
Number of pages12
JournalIEEE Transactions on Neural Systems and Rehabilitation Engineering
Publication statusPublished - 2021


  • Electric and acoustic stimulation (EAS)
  • cochlear implant
  • fully convolutional neural network
  • speech enhancement

ASJC Scopus subject areas

  • Internal Medicine
  • General Neuroscience
  • Biomedical Engineering
  • Rehabilitation


Dive into the research topics of 'Improving the Intelligibility of Speech for Simulated Electric and Acoustic Stimulation Using Fully Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this