Multimodal Emotion Recognition from Voice and Video Signals

Abstract

A promising area of research and development that can significantly increase the efficacy and accuracy of mental health assessments is the use of artificial intelligence (AI) and machine learning algorithms to analyse simultaneously voice and facial expressions in a video stream. More studies are required to completely comprehend the capabilities and limitations of these technologies and guarantee their ethical and effective usage in clinical settings. Collaborative robots (cobots) have the potential to completely change how mental evaluations of autistic children are approached. ChatGPT is an effective language model that can understand and produce human-like text. When used in conjunction with the Cobot, this technology enables children with autism to interact and communicate in a way that is natural to them. In this article, we introduce a novel method for analysing emotional detection using voice analysis and facial recognition that has been tested on the IEMOCAP database. The outcomes session, which illustrates the tool's potential use in healthcare, concludes the paper.