NAaLOSS: Rethinking the Objective of Speech Enhancement

Kuan Hsun Ho; En Lun Yu; Jeih Weih Hung; Berlin Chen

doi:10.1109/MLSP55844.2023.10285948

NAaLOSS: Rethinking the Objective of Speech Enhancement

Kuan Hsun Ho^*, En Lun Yu, Jeih Weih Hung, Berlin Chen

^*Corresponding author for this work

Department of Computer Science and Information Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

Reducing noise interference is crucial for automatic speech recognition (ASR) in a real-world scenario. However, most single-channel speech enhancement (SE) generates 'processing artifacts' that negatively affect ASR performance. Hence, in this study, we suggest a Noise- and Artifacts-aware loss function, NAaLoss, to ameliorate the influence of artifacts from a novel perspective. NAaLoss considers the loss of estimation, de-artifact, and noise ignorance, enabling the learned SE to individually model speech, artifacts, and noise. We examine two SE models (simple/advanced) learned with NAaLoss under various input scenarios (clean/noisy) using two configurations of the ASR system (with/without noise robustness). Experiments reveal that NAaLoss significantly improves the ASR performance of most setups while preserving the quality of SE toward perception and intelligibility. Furthermore, we visualize artifacts through waveforms and spectrograms, and explain their impact on ASR.

Original language	English
Title of host publication	Proceedings of the 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing, MLSP 2023
Editors	Danilo Comminiello, Michele Scarpiniti
Publisher	IEEE Computer Society
ISBN (Electronic)	9798350324112
DOIs	https://doi.org/10.1109/MLSP55844.2023.10285948
Publication status	Published - 2023
Event	33rd IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2023 - Rome, Italy Duration: 2023 Sept 17 → 2023 Sept 20

Publication series

Name	IEEE International Workshop on Machine Learning for Signal Processing, MLSP
Volume	2023-September
ISSN (Print)	2161-0363
ISSN (Electronic)	2161-0371

Conference

Conference	33rd IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2023
Country/Territory	Italy
City	Rome
Period	2023/09/17 → 2023/09/20

Keywords

noise-robust speech enhancement
processing artifacts
single-channel speech enhancement

ASJC Scopus subject areas

Human-Computer Interaction
Signal Processing

Access to Document

10.1109/MLSP55844.2023.10285948

Cite this

Ho, K. H., Yu, E. L., Hung, J. W., & Chen, B. (2023). NAaLOSS: Rethinking the Objective of Speech Enhancement. In D. Comminiello, & M. Scarpiniti (Eds.), Proceedings of the 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing, MLSP 2023 (IEEE International Workshop on Machine Learning for Signal Processing, MLSP; Vol. 2023-September). IEEE Computer Society. https://doi.org/10.1109/MLSP55844.2023.10285948

NAaLOSS: Rethinking the Objective of Speech Enhancement. / Ho, Kuan Hsun; Yu, En Lun; Hung, Jeih Weih et al.
Proceedings of the 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing, MLSP 2023. ed. / Danilo Comminiello; Michele Scarpiniti. IEEE Computer Society, 2023. (IEEE International Workshop on Machine Learning for Signal Processing, MLSP; Vol. 2023-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ho, KH, Yu, EL, Hung, JW & Chen, B 2023, NAaLOSS: Rethinking the Objective of Speech Enhancement. in D Comminiello & M Scarpiniti (eds), Proceedings of the 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing, MLSP 2023. IEEE International Workshop on Machine Learning for Signal Processing, MLSP, vol. 2023-September, IEEE Computer Society, 33rd IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2023, Rome, Italy, 2023/09/17. https://doi.org/10.1109/MLSP55844.2023.10285948

Ho KH, Yu EL, Hung JW, Chen B. NAaLOSS: Rethinking the Objective of Speech Enhancement. In Comminiello D, Scarpiniti M, editors, Proceedings of the 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing, MLSP 2023. IEEE Computer Society. 2023. (IEEE International Workshop on Machine Learning for Signal Processing, MLSP). doi: 10.1109/MLSP55844.2023.10285948

Ho, Kuan Hsun ; Yu, En Lun ; Hung, Jeih Weih et al. / NAaLOSS : Rethinking the Objective of Speech Enhancement. Proceedings of the 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing, MLSP 2023. editor / Danilo Comminiello ; Michele Scarpiniti. IEEE Computer Society, 2023. (IEEE International Workshop on Machine Learning for Signal Processing, MLSP).

@inproceedings{020f2d02c9304b43bb49e6ee791af720,

title = "NAaLOSS: Rethinking the Objective of Speech Enhancement",

abstract = "Reducing noise interference is crucial for automatic speech recognition (ASR) in a real-world scenario. However, most single-channel speech enhancement (SE) generates 'processing artifacts' that negatively affect ASR performance. Hence, in this study, we suggest a Noise- and Artifacts-aware loss function, NAaLoss, to ameliorate the influence of artifacts from a novel perspective. NAaLoss considers the loss of estimation, de-artifact, and noise ignorance, enabling the learned SE to individually model speech, artifacts, and noise. We examine two SE models (simple/advanced) learned with NAaLoss under various input scenarios (clean/noisy) using two configurations of the ASR system (with/without noise robustness). Experiments reveal that NAaLoss significantly improves the ASR performance of most setups while preserving the quality of SE toward perception and intelligibility. Furthermore, we visualize artifacts through waveforms and spectrograms, and explain their impact on ASR.",

keywords = "noise-robust speech enhancement, processing artifacts, single-channel speech enhancement",

author = "Ho, {Kuan Hsun} and Yu, {En Lun} and Hung, {Jeih Weih} and Berlin Chen",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 33rd IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2023 ; Conference date: 17-09-2023 Through 20-09-2023",

year = "2023",

doi = "10.1109/MLSP55844.2023.10285948",

language = "English",

series = "IEEE International Workshop on Machine Learning for Signal Processing, MLSP",

publisher = "IEEE Computer Society",

editor = "Danilo Comminiello and Michele Scarpiniti",

booktitle = "Proceedings of the 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing, MLSP 2023",

}

TY - GEN

T1 - NAaLOSS

T2 - 33rd IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2023

AU - Ho, Kuan Hsun

AU - Yu, En Lun

AU - Hung, Jeih Weih

AU - Chen, Berlin

PY - 2023

Y1 - 2023

N2 - Reducing noise interference is crucial for automatic speech recognition (ASR) in a real-world scenario. However, most single-channel speech enhancement (SE) generates 'processing artifacts' that negatively affect ASR performance. Hence, in this study, we suggest a Noise- and Artifacts-aware loss function, NAaLoss, to ameliorate the influence of artifacts from a novel perspective. NAaLoss considers the loss of estimation, de-artifact, and noise ignorance, enabling the learned SE to individually model speech, artifacts, and noise. We examine two SE models (simple/advanced) learned with NAaLoss under various input scenarios (clean/noisy) using two configurations of the ASR system (with/without noise robustness). Experiments reveal that NAaLoss significantly improves the ASR performance of most setups while preserving the quality of SE toward perception and intelligibility. Furthermore, we visualize artifacts through waveforms and spectrograms, and explain their impact on ASR.

AB - Reducing noise interference is crucial for automatic speech recognition (ASR) in a real-world scenario. However, most single-channel speech enhancement (SE) generates 'processing artifacts' that negatively affect ASR performance. Hence, in this study, we suggest a Noise- and Artifacts-aware loss function, NAaLoss, to ameliorate the influence of artifacts from a novel perspective. NAaLoss considers the loss of estimation, de-artifact, and noise ignorance, enabling the learned SE to individually model speech, artifacts, and noise. We examine two SE models (simple/advanced) learned with NAaLoss under various input scenarios (clean/noisy) using two configurations of the ASR system (with/without noise robustness). Experiments reveal that NAaLoss significantly improves the ASR performance of most setups while preserving the quality of SE toward perception and intelligibility. Furthermore, we visualize artifacts through waveforms and spectrograms, and explain their impact on ASR.

KW - noise-robust speech enhancement

KW - processing artifacts

KW - single-channel speech enhancement

UR - http://www.scopus.com/inward/record.url?scp=85177194419&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85177194419&partnerID=8YFLogxK

U2 - 10.1109/MLSP55844.2023.10285948

DO - 10.1109/MLSP55844.2023.10285948

M3 - Conference contribution

AN - SCOPUS:85177194419

T3 - IEEE International Workshop on Machine Learning for Signal Processing, MLSP

BT - Proceedings of the 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing, MLSP 2023

A2 - Comminiello, Danilo

A2 - Scarpiniti, Michele

PB - IEEE Computer Society

Y2 - 17 September 2023 through 20 September 2023

ER -

NAaLOSS: Rethinking the Objective of Speech Enhancement

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this