Improving the Quality of Sound Recovered Using the Visual Microphone with Frame-wise Image Denoising Preprocessing

<jats:title>Abstract</jats:title>
<jats:p>The visual microphone is a passive remote sound recovery technique by analysing subtle vibrations in a video of an object vibrating due to sound. Multiple research endeavours have been done in improving the quality of the recovered sound, but so far none has been done on investigating the effect of denoising the videos frame by frame prior to the sound recovery using the visual microphone. This work fills in that gap, and evaluates the quality of the recovered sound based on human intelligibility metrics and simple signal-to-noise ratio metrics. Our best results indicated performing image denoising before sound recovery can reduce the noise power by up to 47.16 %, increase the intelligibility by 9.11 % and signal-to-noise ratio by 17.97 %. It is important to perform image denoising before sound recovery when the recorded videos are very noisy.</jats:p>