A Method of Identifying Sentiment from Consumer Multimodality Reviews

Abstract

The rapid expansion of e-commerce and digital media has led to an increasing reliance on online consumer reviews, which play a crucial role in shaping purchasing decisions. Traditional sentiment analysis methods often focus solely on textual data, overlooking the multimodal nature of consumer reviews, which frequently include emoticons and other non-textual elements. This study addresses the challenge of sentiment classification in multimodal consumer reviews by integrating text and emoticons to improve classification accuracy. The primary objective of this research is to develop a sentiment classification method capable of identifying sentiment from movie reviews containing both text and emoticons. Two classification tasks were considered: binary sentiment classification (positive and negative) and multiclass sentiment classification (positive, neutral, and negative). The dataset was collected from the Douban Film and Television Network, comprising Chinese-language movie reviews with embedded emoticons. Textual data was represented using five embedding techniques—Word2Vec, GloVe, FastText, Ada-002, and BERT—while emoticons were encoded using one-hot encoding. The fusion of textual and emoticon features was performed using concatenation. To evaluate model performance, machine learning classifiers such as Support Vector Machine (SVM) and Random Forest, as well as deep learning models like Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM), were trained using 10-fold cross-validation. Experimental results demonstrated that deep learning models, particularly LSTM and CNN, outperformed traditional classifiers when combined with contextual embeddings such as BERT and Ada-002. In binary classification, CNN with Ada-002 achieved the highest accuracy, while LSTM with BERT exhibited superior performance in multiclass classification. The inclusion of emoticons enhanced classification results, particularly in deep learning models. For example, in experiments using the first dataset, the Word2Vec + CNN model achieved an accuracy of 0.80 with text alone, which increased to 0.83 when emoticons were included. Similarly, the GLOVE + CNN model improved from 0.80 to 0.83 with the addition of emoticons. Furthermore, two datasets were used to validate the model's performance under different conditions, testing whether the developed model can generalize and maintain good performance when encountering different data.This study highlights the importance of multimodal fusion in sentiment analysis, demonstrating that integrating emoticons with advanced text representations significantly improves sentiment classification accuracy. These findings provide valuable insights for enhancing sentiment analysis techniques in consumer review analysis.

-