Bangla Speech-based Emotion Detection using a Hybrid CNN-Transformer Approach

Abstract

Robots are now widely employed in various scenarios to interact with humans. It is vital that the robots understand the speaker's emotion and respond accordingly. Humans possess innate abilities to recognize emotions from speech, but it is challenging to provide machines with the same capabilities to predict emotions from human speech. Recently machine learning and deep learning techniques are employed for emotion recognition tasks. Due to emotions such as angry, surprise etc. being predominant for only few time frames of an example audio of that class, it becomes difficult to detect them and models often have a hard time for these classes. We employ a hybrid CNN-Transformer approach where we attempt to solve this problem by having both locally receptive convolutional features and globally receptive attention features contributing to final classification. We evaluate our approach along with other standard approaches in BanglaSER dataset. We report weighted class accuracy of 92.3% using our proposed approach achieving significant performance gain over current literature.