A multi-modal human robot interaction framework based on cognitive behavioral therapy model

Abstract

According to recent statistics, depression and suicide are on a rise in the United States and elsewhere. To resolve this is- sue, synonymous to various current approaches, we propose a multi-modal robot interaction framework, which will act as an extension to current Human Robot Interaction systems to further identify studied signs of depression from various data-acoustic features, like images, video, speech, text, and in general, multi-modal data. One of the recent technologies that we plan to introduce in our resolution, is the use of social-humanoid robots (Pepper by SoftBank) to detect early signs of depression via the power of Natural Language and Multi-Modal Interactions. Rather than solely relying on the interaction between professionals and patients/individuals for treatment, the current HRI framework, offers to lower the entry barrier for potential mental health diagnosis and providing medical treatments in convenience of ones reach. To assure the psychological safety of conversation there is also a "psychological safety module" to provide professional assistance/aid for episodic-cognitive behavioral therapy. Our Multi-modal Robot Interaction (MRI) architecture contains of five modules: Multi-modal Data, Social Robot & dialogue system, psycho-linguistic feature extraction, Machine Learning & NLP methods, and Psychological Safety Feedback/Suggestion to end user and experts.