Learning the Direction of a Sound Source Using Head Motions and Spectral Features

Abstract

In this paper we address the problem of localizing a sound-source by combining binaural or monaural spectral features with head movements. Based on a number of psychophysical and behavioral studies suggesting that the problem of spatial hearing is both listener-dependent and dynamic, we propose to address the problem at hand within the framework of unsupervised learning. More precisely, our method is able to retrieve an intrinsic low-dimensional parameterization from the high-dimensional spectral representation of the acoustic input. We address both binaural and monaural spatial localization with both static and dynamic cues. We show that the recovered low-dimensional representations are homeomorphic to the two-dimensional manifold associated with the motor states of a robotic head with two rotational degrees of freedom. We describe the experimental setup and protocols allowing us to gather acoustic data sets with ground truth for both the emitter-to-listener directions and precise head motions. We validate our method using extensive experiments that consist in classifying acoustic vectors from a test set, based on manifold learning with a different training set. Our method strongly contrasts with current approaches in sound localization because it puts forward the role of learning.