A dialogue manager for multimodal human‐robot interaction and learning of a humanoid robot

Purpose – This paper aims to give an overview of a dialogue manager and recent experiments with multimodal human‐robot dialogues. Design/methodology/approach – The paper identifies requirements and solutions in the design of a human‐robot interface. The paper presents essential techniques for a humanoid robot in a household environment and describes their application to representative interaction scenarios that are based on standard situations for a humanoid robot in a household environment. The presented dialogue manager has been developed within the German collaborative research center SFB‐588 on “Humanoid Robots – Learning and Cooperating Multimodal Robots”. The dialogue system is embedded in a multimodal perceptual system of the humanoid robot developed within this project. The implementation of the dialogue manager is geared to requirements found in the explored scenarios. The algorithms include multimodal fusion, reinforcement learning, knowledge acquisition and tight coupling of dialogue manager and speech recognition. Findings – Within the presented scenarios several algorithms have been implemented and show improvements of the interactions. Results are reported within scenarios that model typical household situations. Research limitations/implications – Additional scenarios need to be explored especially in real‐world (out of the lab) experiments. Practical implications – The paper includes implications for the development of humanoid robots and human‐robot interaction. Originality/value – This paper explores human‐robot interaction scenarios and describes solutions for dialogue systems.