A multimodal corpus for modeling turn management in multi-party conversations

Abstract

Spoken interactions usually have accurate timing and alignment between interlocutors: turn-taking and topic flow are managed in a manner that provides conversational fluency and smooth progress of the interaction. Turn-taking and topic flow are also important in applications such as robot companions that interact with a user in real time. The creation of a multimodal conversational corpus for modeling turn management in multi-party conversations is described. The relation between the interlocutors' spoken utterances and eye-gaze based on the corpus is investigated.