多模式语音端点检测

在语音信号处理系统中,基于帧能量的语音端点检测(voiceactivitydetection,VAD)往往受到语音段能量不平稳及噪声的影响,为了提高语音端点检测的性能和鲁棒性,引入视觉信息。该文提出采用基于数据驱动的线性变换生成视觉特征,在提出一个基于统计的VAD一般模型的基础上构建两个单模式的VAD系统,通过两步式的融合方法,得到了多模式的VAD系统。实验表明:同时利用音频和视觉信息的多模式VAD比基于帧能量的听觉VAD在帧错误率上有55.0%的相对下降,在断句错误率上有98.5%的相对下降。这一结果说明多模式VAD方法基本可以避免断句错误,也能够显著改善帧检测性能,是一种相当有效的方法。

Multimodal voice activity detection

In speech recognition systems, the frame energy-based voice activity detection (VAD) method may be affected by interferance from background noise and non-stationary characteristics of the frame energy in the voice segment. This paper presents a model to improve the performance and robustness of VAD by introducing visual information. Data driven linear transformations are used for visual feature extraction with a general statistical VAD model and a two-stage fusion strategy in a multimodal VAD system. Experiments show a 55.0% reduction in the frame error rate and a 98.5% reduction in sentence breaking error rate with the multimodal VAD as compared to the frame energy-based audio VAD. The results show that multimodal method eliminates most sentence breaking errors, and improves frame detection performance.