Neural Network-Based Emotion Classification in Medical Robotics: Anticipating Enhanced Human–Robot Interaction in Healthcare

Abstract

This study advances artificial intelligence by pioneering the classification of human emotions (for patients) with a healthcare mobile robot, anticipating human–robot interaction for humans (patients) admitted in hospitals or any healthcare environment. This study delves into the challenge of accurately classifying humans emotion as a patient emotion, which is a critical factor in understanding patients’ recent moods and situations. We integrate convolutional neural networks (CNNs), recurrent neural networks (RNNs), and multi-layer perceptrons (MLPs) to analyze facial emotions comprehensively. The process begins by deploying a faster region-based convolutional neural network (Faster R-CNN) to swiftly and accurately identify human emotions in real-time and recorded video feeds. This includes advanced feature extraction across three CNN models and innovative fusion techniques, which strengthen the improved Inception-V3 for superior accuracy and replace the improved Faster R-CNN feature learning module. This valuable replacement aims to enhance the accuracy of face detection in our proposed framework. Carefully acquired these datasets in a simulated environment. Validation on the EMOTIC, CK+, FER-2013, and AffectNet datasets all showed impressive accuracy rates of 98.01%, 99.53%, 99.27%, and 96.81%, respectively. These class-wise accuracy rates show that it has the potential to advance the medical environment and measures in the intelligent manufacturing of healthcare mobile robots.