EmoTrustBot: Affective Intelligence and Trust-Aware Adaptation for Emotionally Aligned Human-Robot Interaction

Abstract

This study presents a new approach to affective reinforcement learning (ARL) designed to enable socially assistive (SAR) robots to detect and respond to human emotional states in real time. The system implements multimodal emotional detection, combining identification of facial expressions with recognition of changes in speech tone, using a personalized emotional response policy that was trained using deep reinforcement learning. This research presents a contribution to ARL by adding an emotion-reward shaping component so the robot can learn to associate emotional information and engage with human-like context-aware behavior. Through the part of the ARL model, the robot learns to perform behavior which is socially acceptable and emotionally valid, ultimately earning user trust in the interaction, which also likely contributes to long-term engagement. The proposed model was analyzed via experiments in a simulated caregiving context using a Pepper humanoid robot and MELD (Multimodal EmotionLines Dataset) using public dataset. The results from the study demonstrated that an ARL based model is more effective at enabling the robot to learn more empathetic human interaction, using different measures to show higher satisfaction, better perception of emotional intelligence of the robot in the experience, and more effective task completion than a baseline which used a non-affective policy. Overall, these are promising findings for the use of emotionally intelligent robots in real-world caregiving and service contexts.