Towards Deep End-of-Turn Detection for Situated Spoken Dialogue Systems

Abstract

We address the challenge of improving live end-of-turn detection for situated spoken dialogue systems. While traditionally silence thresholds have been used to detect the user’s end-of-turn, such an approach limits the system’s potential fluidity in interaction, restricting it to a purely reactive paradigm. By contrast, here we present a system which takes a predictive approach. The user’s end-of-turn is predicted live as acoustic features and words are consumed by the system. We compare the benefits of live lexical and acoustic information by feature analysis and testing equivalent models with different feature sets with a common deep learning architecture, a Long Short-Term Memory (LSTM) network. We show the usefulness of incremental enriched language model features in particular. Training and testing onWizard-of-Oz data collected to train an agent in a simple virtual world, we are successful in improving over a reactive baseline in terms of reducing latency whilst minimising the cut-in rate.