Unraveling Emotions: Multimodal Deep Learning for Fine-Grained Emotion Recognition

Abstract

In the landscape of natural language processing and artificial intelligence, sentiment analysis and emotion recognition hold crucial roles in deciphering human emotions across diverse communication channels. This study addresses a significant research gap by venturing into the promising domain of emotion identification through sentiment analysis, capitalising on the potential of multimodal deep learning. Through an exhaustive literature review, the study seeks to bridge the gap between conventional sentiment analysis methods and the intricate subtleties of human emotions, achieved by fusing data from various modalities. This integration, coupled with fine-grained emotion recognition, strives to heighten the precision of emotion comprehension. Anchored by a robust conceptual framework encompassing variables like information integration, information variability and model type, this research probes the interplay of these factors in shaping emotion consistency. The research methodology involves a two-tailed t-test, a potent statistical tool for hypothesis testing using SmartPLS. The outcomes shed light on the intricate interplay among these variables. While information integration may not exert a significant impact on information consistency, information variability and model type surface as critical factors, each distinctively contributing to the enhancement of information consistency. These insights offer a deeper comprehension of the complexities within this domain, charting a path towards refined insights into the examined relationships.

Keywords: Multimodal Deep Learning, Emotion Recognition, Sentiment Analysis, Fine-Grained Emotions, Information Integration, Model Type

View PDF