A Transformer-Enhanced CNN Framework for EEG Emotion Detection with Lightweight Gray Wolf Optimization and SHAP Analysis

Nattavut Sriwiboon; Songgrod Phimphisan

doi:10.18196/jrc.v6i4.26725

Authors

Nattavut Sriwiboon Kalasin University
Songgrod Phimphisan Kalasin University

DOI:

https://doi.org/10.18196/jrc.v6i4.26725

Keywords:

EEG, Dual-Stream Deep Learning, Transformer Encoder, SHAP, Lightweight Gray Wolf Optimization

Abstract

Emotion recognition from electroencephalogram (EEG) signals has been recognized as critical for enhancing human–computer interaction and mental health monitoring. In this paper, an explainable and real-time dual-stream deep learning framework has been proposed for EEG-based emotion classification. The model integrates a 1D convolutional neural network (1D-CNN) for local feature extraction and a transformer encoder for global dependency modeling, with multi-head attention used for feature fusion. Lightweight Gray Wolf Optimization (LGWO) has been employed for selecting optimal features, and an ensemble of lightweight classifiers has been applied to improve robustness. Experiments conducted on DEAP, SEED, BrainWave, and INTERFACE datasets have demonstrated superior performance, achieving accuracies of 96.90%, 94.25%, 93.70%, and 92.80%, respectively. An average inference delay of 5.2 milliseconds per trial has confirmed real-time applicability. Furthermore, SHAP analysis has been incorporated to interpret the model’s decision-making process by identifying influential EEG channels and frequency components. The results have validated the proposed model as a robust, accurate, and explainable solution for EEG-based emotion recognition, establishing a new benchmark for future research in affective computing and clinical applications.

References

H. Berger, “Über das Elektrenkephalogramm des Menschen,” Archiv für Psychiatrie und Nervenkrankheiten, vol. 87, pp. 527-570, 1929.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436-444, 2015.

P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.

S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.

X. Xie, G. Cheng, J. Wang, K. Li, X. Yao, and J. Han, “Oriented R-CNN and Beyond,” International Journal of Computer Vision, vol. 132, no. 7, pp. 2420-2442, 2024.

A. Bouguettaya and H. Zarzour, “CNN-based hot-rolled steel strip surface defects classification: a comparative study between different pre-trained CNN models,” The International Journal of Advanced Manufacturing Technology, vol. 132, no. 1, pp. 399-419, 2024.

Y. Nie, Y. Chen, J. Guo, S. Li, Y. Xiao, W. Gong, and R. Lan, “An improved CNN model in image classification application on water turbidity,” Scientific Reports, vol. 15, no. 1, p. 11264, 2025.

R. Du, T. Li, G. Meng, and F. Liu, “CNN-AC algorithm for hybrid precoding in millimeter-wave massive MIMO systems,” Wireless Networks, pp. 1-11, 2025.

W. R. Murdhiono, H. Riska, N. Khasanah, Hamzah, and P. Wanda, “Mentalix: stepping up mental health disorder detection using Gaussian CNN algorithm,” Iran Journal of Computer Science, pp. 1-10, 2025.

T. Al-Shehari et al., “Comparative evaluation of data imbalance addressing techniques for CNN-based insider threat detection,” Scientific Reports, vol. 14, no. 1, p. 24715, 2024.

L. Yang, L. Lu, C. Liu, J. Zhang, K. Guo, N. Zhang, F. Zhou, and Y. Zhao, “Interactive exploration of CNN interpretability via coalitional game theory,” Scientific Reports, vol. 15, no. 1, p. 9261, 2025.

P. Cai et al., “Enhancing quantum approximate optimization with CNN-CVaR integration,” Quantum Information Processing, vol. 24, no. 2, p. 37, 2025.

S. K. Dewangan, S. Choubey, J. Patra, and A. Choubey, “IMU-CNN: implementing remote sensing image restoration framework based on Mask-Upgraded Cascade R-CNN and deep autoencoder,” Multimedia Tools and Applications, vol. 83, no. 27, pp. 69049-69081, 2024.

R. Bhargava, N. Arivazhagan, and K. S. Babu, “Hybrid RMDL-CNN for speech recognition from unclear speech signal,” International Journal of Speech Technology, vol. 28, no. 1, pp. 195-217, 2025.

S. B. M K and M. Kalra, “Leveraging CNN and Fundus Imaging for Enhanced Glaucoma Detection,” SN Computer Science, vol. 5, no. 8, p. 1137, 2024.

K.-A. C. Quan, V.-T. Nguyen, T. V. Nguyen, and M.-T. Tran, “Unified ViT-CNN for few-shot object counting,” Signal, Image and Video Processing, vol. 19, no. 3, p. 221, 2025.

J. Zhu et al., “Realization of normal temperature detection through visible light images by Retinex-CNN,” Journal of Optics, pp. 1-10, 2025.

P. Dutta and N. B. Muppalaneni, “OCR Advancement with Pixel-Focused CNN for Handwritten Characters: A Journey with AsTel Dataset,” Arabian Journal for Science and Engineering, pp. 1-17, 2025.

S. Prakash and K. Sangeetha, “Systems classification of air pollutants using Adam optimized CNN with XGBoost feature selection,” Analog Integrated Circuits and Signal Processing, vol. 122, no. 3, p. 35, 2025.

C. Liu, “Landslide susceptibility mapping using CNN models based on factor visualization and transfer learning,” Stochastic Environmental Research and Risk Assessment, vol. 39, no. 1, pp. 231-249, 2025.

N.-L. Pham, Q.-B. Ta, T.-C. Huynh, and J.-T. Kim, “CNN federated learning for vibration-based damage identification of submerged structure-foundation system,” Journal of Civil Structural Health Monitoring, pp. 1-26, 2025.

B. Chakravarthi, S. Ng, M. Ezilarasan, and M. Leung, “EEG-based emotion recognition using hybrid CNN and LSTM classification,” Frontiers in Systems Neuroscience, vol. 16, pp. 1-9, 2022.

H. Sun, H. Wang, R. Wang, Y. Gao, “Emotion recognition based on EEG source signals and dynamic brain function network,” Journal of Neuroscience Methods, vol. 415, 2025.

J. Tian and X. Luo, “Emotion classification based on EEG wavelet features and LSTM network”, Proceedings of the Fifth International Conference on Signal Processing and Computer Science (SPCS 2024), vol. 13442, pp. 87-95, 2025.

N. Ahmadzadeh, N. Cavus, P. Esmaili, B. Sekeroglu, and S. Aşır, “Detecting emotions through EEG signals based on modified convolutional fuzzy neural network,” Scientific Reports, vol. 14, 2024.

W. Tang, L. Fan, X. Lin, and Y. Gu, “EEG emotion recognition based on efficient-capsule network with convolutional attention, Biomedical Signal Processing and Control,” Biomedical Signal Processing and Control, vol. 103, 2025.

M. Li, P. Yu, L. Zhang, and Y. Shen, “A spatial and temporal transformer-based EEG emotion recognition in VR environment,” Frontiers in Neuroscience, vol. 19, 2025.

Z. Wang and Y. Wang, “Emotion recognition based on multimodal physiological electrical signals,” Frontiers in Neuroscience, vol. 19, p. 1512799, 2025.

J. Yan, “Spatio-temporal graph Bert network for EEG emotion recognition,” Biomedical Signal Processing and Control, vol. 104, 2025.

J. A. Cruz-Vazquez, J. Y. Montiel-Pérez, R. Romero-Herrera, and E. Rubio-Espino, “Emotion recognition from EEG signals using advanced transformations and deep learning,” Mathematics, vol. 13, no. 2, 2025.

R. Singh and M. Sharma, “Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals,” Heliyon, vol. 11, no. 2, 2025.

V. Doma and M. Pirouz, “A comparative analysis of machine learning methods for emotion recognition using EEG and peripheral physiological signals,” Journal of Big Data, vol. 7, 2025.

S. Adhikari et al., “Analysis of frequency domain features for the classification of evoked emotions using EEG signals,” Experimental Brain Research, vol. 243, no. 3, p. 65, 2025.

D. K. Saha and T. D. Nath, “A lightweight CNN-based ensemble approach for early detecting Parkinson’s disease with enhanced features,” International Journal of Speech Technology, pp. 1-15, 2025.

F. Zhang, B. Zhang, S. Guo, and X. Zhang, “MFCC-CNN: A patient-independent seizure prediction model,” Neurological Sciences, vol. 45, no. 12, pp. 5897-5908, 2024.

G. Wang, H. Zhang, M. Gao, W. Ding, and Y. Qian, “Identification and classification of power quality disturbances using CNN-transformer,” Journal of Electrical Engineering & Technology, 2025.

JS. Kiranyaz, T. Ince, O. Abdeljaber, O. Avci, and M. Gabbouj, “1-D Convolutional Neural Networks for Signal Processing Applications,” Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8360–8364, 2019.

S. Mirjalili, “Grey Wolf Optimizer,” Advances in Engineering Software, vol. 69, pp. 46-61, 2014.

Y. Yan, and W. Liu, “Topical collections on machine learning based semantic representation and analytics for multimedia application,” Neural Computing and Applications, vol. 34, no. 15, pp. 12239-12240, 2022.

S. M. Lundberg and S. I. Lee, “A Unified Approach to Interpreting Model Predictions.” Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 4768 - 4777, 2027.

K. Merabet et al., “Predicting water quality variables using gradient boosting machine: global versus local explainability using SHapley Additive Explanations (SHAP),” Earth Science Informatics, vol. 18, no. 3, p. 298, 2025.

A. Koushik, M. Manoj, and N. Nezamuddin, “SHapley Additive exPlanations for Explaining Artificial Neural Network Based Mode Choice Models,” Transportation in Developing Economies, vol. 10, no. 1, p. 12, 2024.

K. D. Bathe and N. S. Patil, “ConvExNet: Deep learning-based flood detection utilizing Shapley additive explanations,” Journal of Earth System Science, vol. 134, no. 2, p. 99, 2025.

E. Çetin, C. Barrado, E. Salamí, and E. Pastor, “Analyzing deep reinforcement learning model decisions with Shapley additive explanations for counter drone operations,” Applied Intelligence, vol. 54, no. 23, pp. 12095-12111, 2024.

H. H. Nguyen, J.-L. Viviani, and S. Ben Jabeur, “Bankruptcy prediction using machine learning and Shapley additive explanations,” Review of Quantitative Finance and Accounting, pp. 1-42, 2023.

S. Heddam, "Explainability of Machine Learning Using Shapley Additive exPlanations (SHAP): CatBoost, XGBoost and LightGBM for Total Dissolved Gas Prediction," Machine Learning and Granular Computing: A Synergistic Design Environment, pp. 1-25, 2024.

L. Chen, Z. He, Q. Ni, Q. Zhou, X. Long, W. Yan, Q. Sui, and J. Liu, “Dual-radiomics based on SHapley additive explanations for predicting hematologic toxicity in concurrent chemoradiotherapy patients,” Discover Oncology, vol. 16, no. 1, p. 541, 2025.

C. Kirabo, S. Murindanyi, N. P. Kirabo, K. M. Hasib, and G. Marvin, "SHapley Additive exPlanations for Machine Emotion Intelligence in CNNs," International Conference on Computational Intelligence, pp. 657-671, 2023.

N. Sriwiboon, “Efficient and lightweight CNN model for COVID-19 diagnosis from CT and X-ray images using customized pruning and quantization techniques,” Neural Computing and Applications, vol. 37, no. 18, pp. 13059-13078, 2025.

N. K. Mishra, P. Singh, A. Gupta, and S. D. Joshi, “PP-CNN: probabilistic pooling CNN for enhanced image classification,” Neural Computing and Applications, vol. 37, no. 6, pp. 4345-4361, 2025.

N. Kaur, S. Pandey, and N. Kalra, “MFR-CNN: A modified faster R-CNN approach based on bounding box and reliable score for cloth image retrieval,” Multimedia Tools and Applications, pp. 1-29, 2024.

R. Saffarini, F. Khamayseh, Y. Awwad, M. Sabha, and D. Eleyan, “Dynamic generative R-CNN,” Neural Computing and Applications, vol. 37, no. 10, pp. 7107-7120, 2025.

C. Gao and H. Ge, “I-CNN-LSTM: An Improved CNN-LSTM for Transient Stability Analysis of More Electric Aircraft Power Systems,” Arabian Journal for Science and Engineering, vol. 50, no. 8, pp. 5683-5696, 2025.

H. Aouani and Y. Ben Ayed, “Deep facial expression detection using Viola-Jones algorithm, CNN-MLP and CNN-SVM,” Social Network Analysis and Mining, vol. 14, no. 1, p. 65, 2024.

S. Davoudi and K. Roushangar, “Innovative approaches to surface water quality management: advancing nitrate (NO3) forecasting with hybrid CNN-LSTM and CNN-GRU techniques,” Modeling Earth Systems and Environment, vol. 11, no. 2, p. 80, 2025.

Pranav and R. Katarya, “Effi-CNN: real-time vision-based system for interpretation of sign language using CNN and transfer learning,” Multimedia Tools and Applications, vol. 84, no. 6, pp. 3137-3159, 2025.

H. Dehnavi, M. Dehnavi, and S. H. Klidbary, “Fcd-cnn: FPGA-based CU depth decision for HEVC intra encoder using CNN,” Journal of Real-Time Image Processing, vol. 21, no. 4, p. 105, 2024.

I. Linck, A. T. Gómez, and G. Alaghband, “SVG-CNN: A shallow CNN based on VGGNet applied to intra prediction partition block in HEVC,” Multimedia Tools and Applications, vol. 83, no. 30, pp. 73983-74001, 2024.

M. Telmem, N. Laaidi, Y. Ghanou, S. Hamiane, and H. Satori, “Comparative study of CNN, LSTM and hybrid CNN-LSTM model in amazigh speech recognition using spectrogram feature extraction and different gender and age dataset,” International Journal of Speech Technology, vol. 27, no. 4, pp. 1121-1133, 2024.

S. Esteki and A. R. Naghsh-Nilchi, “SW/SE-CNN: semi-wavelet and specific image edge extractor CNN for Gaussian image denoising,” Neural Computing and Applications, vol. 36, no. 10, pp. 5447-5469, 2024.

M. Asfand-e-yar, Q. Hashir, A. A. Shah, H. A. M. Malik, A. Alourani, and W. Khalil, “Multimodal CNN-DDI: using multimodal CNN for drug to drug interaction associated events,” Scientific Reports, vol. 14, no. 1, p. 4076, 2024.

K. G. Panchbhai, M. G. Lanjewar, V. V. Malik, and P. Charanarur, “Small size CNN (CAS-CNN), and modified MobileNetV2 (CAS-MODMOBNET) to identify cashew nut and fruit diseases,” Multimedia Tools and Applications, vol. 83, no. 42, pp. 89871-89891, 2024.

M. Kaddes, Y. M. Ayid, A. M. Elshewey, and Y. Fouad, “Breast cancer classification based on hybrid CNN with LSTM model,” Scientific Reports, vol. 15, no. 1, p. 4409, 2025.

E. Pintelas, I. E. Livieris, V. Tampakas, and P. Pintelas, “Feature augmentation-based CNN framework for skin-cancer diagnosis,” Evolving Systems, vol. 16, no. 1, p. 34, 2025.

J. Mishra and R. K. Sharma, “Optimized FPGA Architecture for CNN-Driven Voice Disorder Detection,” Circuits, Systems, and Signal Processing, vol. 44, no. 6, pp. 4455-4467, 2025.

R. Nambiar and R. N, “A Comprehensive Review of AI and Deep Learning Applications in Dentistry: From Image Segmentation to Treatment Planning,” Journal of Robotics and Control (JRC), vol. 5, p. 2024, 2024.

T.-V. Dang and L. Tran, “A Secured, Multilevel Face Recognition based on Head Pose Estimation, MTCNN and FaceNet,” Journal of Robotics and Control (JRC), vol. 4, pp. 431-436, 2023.

T. Admassu, T. Suresh, R. Purushothaman, S. Ganesan, and K. K. Napa, “Early Prediction of Gestational Diabetes with Parameter-Tuned K-Nearest Neighbor Classifier,” Journal of Robotics and Control (JRC), vol. 4, pp. 452-457, 2023.

S. Phimphisan and N. Sriwiboon, “A Customized CNN Architecture with CLAHE for Multi-Stage Diabetic Retinopathy Classification,” Engineering, Technology & Applied Science Research, vol. 14, no. 6, pp. 18258-18263, 2024.

N. Sriwiboon and S. Phimphisan, “Efficient COVID-19 Detection using Optimized MobileNetV3-Small with SRGAN for Web Application,” Engineering, Technology & Applied Science Research, vol. 15, no. 2, pp. 20953-20958, 2025.

I. Uluocak and M. Bilgili, “Daily air temperature forecasting using LSTM-CNN and GRU-CNN models,” Acta Geophysica, vol. 72, no. 3, pp. 2107-2126, 2024.

T. Li, J. Shu, and Y. Wang, “Deformation prediction of underground engineering support structures via the ST-CNN-LSTM model,” Journal of Civil Structural Health Monitoring, pp. 1-19, 2025.

E. R. Coutinho, J. G. F. Madeira, D. G. F. Borges, M. V. Springer, E. M. de Oliveira, and A. L. G. A. Coutinho, “Multi-Step Forecasting of Meteorological Time Series Using CNN-LSTM with Decomposition Methods,” Water Resources Management, pp. 1-26, 2025.

V. Singh, S. K. Sahana, and V. Bhattacharjee, “A novel CNN-GRU-LSTM based deep learning model for accurate traffic prediction,” Discover Computing, vol. 28, no. 1, p. 38, 2025.

A. Shaik, S. S. Dutta, I. M. Sawant, S. Kumar, A. Balasundaram, and K. De, “An attention based hybrid approach using CNN and BiLSTM for improved skin lesion classification,” Scientific Reports, vol. 15, no. 1, p. 15680, 2025.

X. Bai, L. Zhang, Y. Feng, H. Yan, and Q. Mi, “Multivariate temperature prediction model based on CNN-BiLSTM and RandomForest,” The Journal of Supercomputing, vol. 81, no. 1, p. 162, 2024.

Q. Tian, R. Cai, Y. Luo, and G. Qiu, “DOA Estimation: LSTM and CNN Learning Algorithms,” Circuits, Systems, and Signal Processing, vol. 44, no. 1, pp. 652-669, 2025.

M. Miao, J. Liang, Z. Sheng, W. Liu, B. Xu, and W. Hu, “ST-SHAP: A hierarchical and explainable attention network for emotional EEG representation learning and decoding,” Journal of Neuroscience Methods, vol. 414, 2025.

S. Koelstra et al., "DEAP: A Database for Emotion Analysis ;Using Physiological Signals," in IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18-31, Jan.-March 2012.

W. Lu, T.-P. Tan, and H. Ma, “Bi-Branch Vision Transformer Network for EEG Emotion Recognition,” IEEE Access, vol. 11, pp. 36233-36243, 2023.

P. Dutta, S. Paul, K. Cengiz, R. Anand, and A. Kumar, "A predictive method for emotional sentiment analysis by deep learning from EEG of brainwave dataset," Artificial intelligence for neurological disorders, pp. 25-48, 2023.

G. Y. Choi et al., “EEG Dataset for the Recognition of Different Emotions Induced in Voice User Interaction,” Scientific Data, vol. 11, no. 1, 2024.

A. Howard et al., "Searching for mobilenetv3," Proceedings of the IEEE/CVF international conference on computer vision, pp. 1314-1324, 2019.

T. Fawcett, “An introduction to ROC analysis,” Pattern Recognition Letters, vol. 27, no. 8, pp. 861-874, 2006.

J. Tian and X. Luo, “Emotion classification based on EEG wavelet features and LSTM network,” Proceedings of the Fifth International Conference on Signal Processing and Computer Science (SPCS 2024), vol. 13442, pp. 87-95, 2025.