A Performance Analysis of Emotion Recognition Systems Driven by Physiological Signals
Abstract:
Emotion recognition using
physiological signals has emerged as a pivotal area in affective computing,
enabling machines to interpret human emotional states with enhanced accuracy.
This study conducts a comprehensive performance evaluation of five machine
learning models—Support Vector Machine (SVM), Random Forest (RF), Convolutional
Neural Network (CNN), Bidirectional Long Short-Term Memory (BiLSTM), and
K-Nearest Neighbors (KNN)—applied to two key physiological datasets: a
univariate Heart Rate Emotion dataset and a multimodal VR Emotion dataset
encompassing EEG, galvanic skin response, and motion signals. Implemented
within a modular Java-based framework, these models were assessed on accuracy,
precision, recall, F1-score, and confusion matrices. The BiLSTM model
consistently outperformed others, achieving peak accuracies of 85.2% on heart
rate data and 89.7% on multimodal VR data, underscoring the critical role of
temporal modeling and multimodal integration in emotion recognition. The
findings provide insights for developing robust, real-time, and scalable
emotion-aware systems in domains such as healthcare, virtual reality, and
human-computer interaction.
References:
[1]. Koelstra, S., 2012, DEAP: A Dataset for Emotion Analysis Using Physiological and Audiovisual Signals: IEEE Trans. Affective Comput., vol. 3, no. 1, pp. 18–31.
[2]. Zhao, R., 2021, ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors: IEEE Trans. Affective Comput., vol. 12, no. 3, pp. 552–566.
[3]. Subasi, M., 2020, Practical Machine Learning for Data Analysis Using Python, Academic Press.
[4]. Zhang, J., and Liu, B., Li, Y., 2020, EEG Emotion Recognition Based on BiLSTM Networks with Attention: IEEE Access, vol. 8, pp. 183162–183173.
[5]. Panahi, F., and Rashidi, S., 2021, Application of Fractional Fourier Transform in ECG-Based Emotion Detection: Biomed. Signal Process. Control, vol. 66.
[6]. Zheng, W., and Lu, B.-L., 2016, Personalizing EEG-Based Affective Models with Transfer Learning: Proc. IJCAI, pp. 2732–2738.
[7]. Abadi, M., 2016, TensorFlow: A System for Large-Scale Machine Learning: Proc. OSDI, pp. 265–283.
[8]. Breiman, L., 2001, Random Forests: Machine Learning, vol. 45, no. 1, pp. 5–32.
[9]. Chang, C.-C., and Lin, C.-J., 2011, LIBSVM: A Library for Support Vector Machines: ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, pp. 1–27.
[10]. Alhagry, A., Aly, A., and El-Khoribi, R., 2017, Emotion Recognition Based on EEG Using LSTM Recurrent Neural Network: Int. J. Adv. Comput. Sci. Appl., vol. 8, no. 10, pp. 355–358.
[11]. Krill, T., 2022, Emotion Recognition Using Wearable EEG: A Systematic Literature Review: IEEE Access, vol. 10, pp. 101732–101747.
[12]. Kumar, D., and Singh, M., 2021, Feature-Level Fusion of Multimodal Signals for Emotion Classification: Multimedia Tools Appl., vol. 80, pp. 27859–27879.
[13]. Chanel, G., Kronegg, J., Grandjean, D., and Pun, T., 2003, Emotion Assessment: Arousal Evaluation Using EEG’s and Peripheral Physiological Signals: Int. J. Human-Computer Studies, vol. 59, no. 1–2, pp. 89–112.
[14]. Alarcao, S., and Fonseca, M. J., 2019, Emotions Recognition Using EEG Signals: A Survey: IEEE Trans. Affective Comput., vol. 10, no. 3, pp. 374–393.
[15]. Ranjan, A., and Prasad, P., 2022, Deep Learning Approach for EEG-Based Emotion Classification Using LSTM and Attention Mechanisms: Comput. Biol. Med., vol. 144.
[16]. Picard, R. W., 2000, Toward Computers that Recognize and Respond to User Emotion: IBM Syst. J., vol. 39, no. 3–4, pp. 705–719.
[17]. Li, M., Xu, H., Zhao, Q., & Zhang, Y., 2023, Multimodal Emotion Recognition Using EEG and Eye-Tracking Signals with Deep Neural Networks: IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31, 905–914.
[18]. Gupta, R., & Yadav, A., 2023, Emotion Detection from ECG Using Hybrid CNN-LSTM Model: Biomedical Signal Processing and Control, 84, 104740.
[19]. Khan, M. A., Akram, T., & Sharif, M., 2022, A Deep Learning Framework for Emotion Recognition Using EEG Signals: Computers in Biology and Medicine, 142, 105210.
[20]. Luo, Y., Lin, Y., Li, Z., & Liu, Y., 2022, A Lightweight BiLSTM-Based Emotion Recognition Model for Wearable Devices: Sensors, 22(4), 1432.
