Modulation spectrum factorization for robust speech recognition

This paper presents a novel approach to improving the noise robustness of speech features built on top of nonnegative matrix factorization (NMF). To do this, we employ NMF to extract a common set of basis spectral vectors that cover the intrinsic temporal structure inherent in the modulation spectra of clean training speech features. The new modulation spectra of the speech features, constructed by mapping the original modulation spectra into the space spanned by these basis vectors, are demonstrated with good noise-robust capabilities. All experiments were conducted using the Aurora-2 database and task. The results show that the proposed NMF-based approach, together with mean and variance normalization (MVN), can provide average error reduction rates of over 65% and 12% relative as compared with the baseline MFCC system and that using the MVN method alone, respectively.