Simple and Effective Speech Enhancement for Visual Microphone

Visual microphone is a technique that recovers the sound from a silent video. The simplest way to improve sound recovery performance of the visual microphone is by applying the traditional speech enhancement algorithms which are based on complicated filter designs or sound models. This paper proposes a simple and effective speech enhancement for visual microphone (SEVM) that suppress spectrum components with small amplitude than a predefined threshold value, which exploits the unique properties that the sound spectrum recovered from the visual microphone is relatively high and the noise spectrum generated motion estimation error and damped oscillation is relatively low. The proposed SEVM method can also be easily extended to a multichannel case that multiple speech signals are recovered from multiple cameras. Experimental results show the proposed SEVM method better performance than the traditional speech enhancement algorithms in terms of log-likelihood ratio (LLR), signal to noise ratio (SNR), segmental SNR (SegSNR) and cepstral distance (CEP). From these results, we convince that the proposed SEVM method that is adapted to the visual microphone is really simple and effective than the traditional speech enhancement methods that are just extended to the visual microphone as a post-processing.