基于脑电和眼动信号的不同文化情绪识别研究

doi:10.12267/j.issn.2096-5931.2026.06.008

信息通信技术与政策 ›› 2026, Vol. 52 ›› Issue (6): 48-56.doi: 10.12267/j.issn.2096-5931.2026.06.008

基于脑电和眼动信号的不同文化情绪识别研究

Emotion recognition across different cultures based on EEG and eye-tracking signals

罗锦宏¹, 徐劲龙², 姜忠义², 邹凌²

¹ 常州信息职业技术学院, 常州 213164
² 常州大学, 常州 213159

收稿日期:2026-05-10 出版日期:2026-06-25 发布日期:2026-07-06
作者简介:
罗锦宏,常州信息职业技术学院电子工程学院副教授,主要从事生物电磁信号分析及电子信息相关教学与科研工作
徐劲龙,常州大学硕士研究生在读,主要从事情绪识别等方面的研究工作
姜忠义,常州大学计算机与人工智能学院副教授,主要从事最优化理论、数据分析及智能算法应用等方面的研究工作
邹凌,常州大学医学与健康工程学院教授,主要从事信号处理、机器学习、神经工程和脑机接口等方面的研究工作

LUO Jinhong¹, XU Jinlong², JIANG Zhongyi², ZOU Ling²

¹ School of Electronic Engineering, Changzhou College of Information Technology, Changzhou 213164, China
² Changzhou University, Changzhou 213159, China

Received:2026-05-10 Online:2026-06-25 Published:2026-07-06

摘要/Abstract

摘要：

针对不同文化情绪识别中多模态特征挖掘不足与泛化困难问题,提出了“层级多尺度分支残差变换-典型相关注意力融合网络”,其中包含多尺度特征提取、典型相关分析增强和注意力加权融合模块。基于上海交通大学情绪脑电数据集(SJTU Emotion Electroencephalography Dataset,SEED)的中国、德国和法国子集(分别为SEED-CHN、SEED-GER和SEED-FRA),开展了文化内被试相关、文化内被试无关以及跨文化被试无关试验。试验结果表明,该方法在三类试验结果中整体优于多种基线方法,说明该方法在不同文化场景下具有较好的情绪识别性能与稳定性。

关键词: 不同文化情绪识别, 脑电信号, 眼动信号, 多尺度特征提取, 注意力融合

Abstract:

To address the problems of insufficient multimodal feature mining and limited generalization in emotion recognition across different cultures, this paper proposes a Hierarchical Multi-scale Branch Residual Transformation-Canonical Correlation Attention Fusion Network, which consists of multi-scale feature extraction, canonical correlation analysis-based enhancement, and attention-weighted fusion modules. Based on the Chinese, German, and French subsets of the SJTU emotion electroencephalography dataset (SEED), namely SEED-CHN, SEED-GER, and SEED-FRA, intra-cultural subject-dependent, intra-cultural subject-independent, and cross-cultural subject-independent experiments were conducted. The experimental results show that, in the intra-cultural subject-dependent experiments, the proposed method outperforms several baseline methods overall, indicating that it has good emotion recognition performance and stability in different cultural scenarios.

Key words: cross-cultural emotion recognition, electroencephalography signals, eye movement signals, multiscale feature extraction, attention fusi

中图分类号:

罗锦宏, 徐劲龙, 姜忠义, 邹凌. 基于脑电和眼动信号的不同文化情绪识别研究[J]. 信息通信技术与政策, 2026, 52(6): 48-56.

LUO Jinhong, XU Jinlong, JIANG Zhongyi, ZOU Ling. Emotion recognition across different cultures based on EEG and eye-tracking signals[J]. Information and Communications Technology and Policy, 2026, 52(6): 48-56.

导出引用管理器 EndNote|Ris|BibTeX

链接本文:

http://ictp.caict.ac.cn/CN/10.12267/j.issn.2096-5931.2026.06.008

http://ictp.caict.ac.cn/CN/Y2026/V52/I6/48

图/表 8

参考文献 10

[1]	LIU W, ZHENG W L, LI Z, et al. Identifying similarities and differences in emotion recognition with EEG and eye movements among Chinese, German, and French people[J]. Journal of Neural Engineering, 2022, 19(2):026012. doi: 10.1088/1741-2552/ac5c8d
[2]	ZHENG W L, LIU W, LU Y, et al. Emotionmeter: a multimodal framework for recognizing human emotions[J]. IEEE Transactions on Cybernetics, 2018, 49(3):1110-1122. doi: 10.1109/TCYB.6221036 URL
[3]	LIU Z, WANG Y, VAIDYA S, et al. Kan: kolmogorov-arnold networks[J]. arXiv Preprint, arXiv:240419756, 2024.
[4]	KHALED A. BCN: batch channel normalization for image classification[C]//International Conference on Pattern Recognition. Cham: Springer, 2025: 295-308.
[5]	HOTELLING H. Relations between two sets of variates[M]. Breakthroughs in Statistics:Methodology and Distribution. New York: Springer, 1992: 162-190.
[6]	LU Y, ZHENG W L, LI B, et al. Combining eye movements and EEG to enhance emotion recognition[C]//Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2015:1170-1176.
[7]	GONG X, DONG Y, ZHANG T. CoDF-Net: coordinated-representation decision fusion network for emotion recognition with EEG and eye movement signals[J]. International Journal of Machine Learning and Cybernetics, 2024, 15(4):1213-1226. doi: 10.1007/s13042-023-01964-w
[8]	GONG X, CHEN C P, ZHANG T. Cross-cultural emotion recognition with EEG and eye movement signals based on multiple stacked broad learning system[J]. IEEE Transactions on Computational Social Systems, 2023, 11(2):2014-2025. doi: 10.1109/TCSS.2023.3298324 URL
[9]	VAN DER MAATEN L, HINTON G. Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9:11.
[10]	KE G, MENG Q, FINLEY T, et al. Lightgbm: a highly efficient gradient boosting decision tree[J]. Advances in Neural Information Processing Systems, 2017:30.

	SEED-CHN		SEED-GER		SEED-FRA
	Acc	Std↓	Acc	Std↓	Acc	Std↓
Concat	82.43%	13.67%	74.53%	20.48%	69.07%	17.32%
SUM	76.30%	15.42%	74.45%	18.84%	65.11%	18.40%
MAX	79.88%	13.29%	75.29%	20.99%	66.64%	15.61%
Fuzzy	82.36%	11.12%	81.00%	15.05%	68.45%	14.33%
BDAE	90.11%	10.34%	85.32%	14.06%	81.13%	10.56%
DCCA-AM	92.36%	8.02%	89.45%	10.65%	79.14%	13.56%
MSBLS	94.30%	8.14%	94.33%	9.14%	84.68%	9.73%
CoDF-Net	94.09%	7.34%	91.62%	9.94%	—	—
HMSB-CAF	97.41%	5.56%	95.37%	5.51%	90.59%	7.72%

	SEED-CHN		SEED-GER		SEED-FRA
	Acc	Std↓	Acc	Std↓	Acc	Std↓
Concat	82.43%	13.67%	74.53%	20.48%	69.07%	17.32%
SUM	76.30%	15.42%	74.45%	18.84%	65.11%	18.40%
MAX	79.88%	13.29%	75.29%	20.99%	66.64%	15.61%
Fuzzy	82.36%	11.12%	81.00%	15.05%	68.45%	14.33%
BDAE	90.11%	10.34%	85.32%	14.06%	81.13%	10.56%
DCCA-AM	92.36%	8.02%	89.45%	10.65%	79.14%	13.56%
MSBLS	94.30%	8.14%	94.33%	9.14%	84.68%	9.73%
CoDF-Net	94.09%	7.34%	91.62%	9.94%	—	—
HMSB-CAF	97.41%	5.56%	95.37%	5.51%	90.59%	7.72%

	SEED-CHN		SEED-GER		SEED-FRA
	Acc	Std↓	Acc	Std↓	Acc	Std↓
Concat	79.64%	8.59%	82.43%	11.93%	68.52%	7.91%
SUM	66.64%	13.84%	74.98%	9.92%	79.86%	7.10%
MAX	79.88%	13.29%	64.52%	12.44%	66.64%	10.23%
Fuzzy	79.63%	8.56%	80.51%	8.65%	71.14%	8.42%
BDAE	82.49%	8.62%	79.35%	8.26%	68.33%	9.13%
DCCA-AM	82.36%	7.93%	81.55%	7.90%	77.54%	8.23%
MSBLS	90.16%	6.40%	84.34%	7.43%	84.67%	4.85%
CoDF-Net	87.04%	6.58%	83.87%	7.47%	—	—
HMSB-CAF	91.67%	6.08%	88.21%	6.89%	86.96%	5.63%

	SEED-CHN		SEED-GER		SEED-FRA
	Acc	Std↓	Acc	Std↓	Acc	Std↓
Concat	79.64%	8.59%	82.43%	11.93%	68.52%	7.91%
SUM	66.64%	13.84%	74.98%	9.92%	79.86%	7.10%
MAX	79.88%	13.29%	64.52%	12.44%	66.64%	10.23%
Fuzzy	79.63%	8.56%	80.51%	8.65%	71.14%	8.42%
BDAE	82.49%	8.62%	79.35%	8.26%	68.33%	9.13%
DCCA-AM	82.36%	7.93%	81.55%	7.90%	77.54%	8.23%
MSBLS	90.16%	6.40%	84.34%	7.43%	84.67%	4.85%
CoDF-Net	87.04%	6.58%	83.87%	7.47%	—	—
HMSB-CAF	91.67%	6.08%	88.21%	6.89%	86.96%	5.63%

	CHN				GER				FRA
	GER		FRA		CHN		FRA		CHN		GER
	Acc	Std↓	Acc	Std↓	Acc	Std↓	Acc	Std↓	Acc	Std↓	Acc	Std↓
Concat	53.64%	13.11%	62.32%	10.25%	52.11%	13.61%	61.23%	13.54%	55.17%	15.23%	60.94%	9.82%
SUM	55.12%	8.63%	57.95%	8.74%	49.43%	15.32%	61.45%	11.22%	54.81%	16.29%	55.72%	11.20%
MAX	56.53%	8.14%	54.56%	8.91%	50.43%	15.14%	63.65%	10.68%	55.14%	16.52%	56.83%	11.09%
Fuzzy	60.03%	9.91%	60.82%	10.31%	55.28%	13.14%	62.14%	10.01%	64.32%	14.09%	58.64%	11.67%
BDAE	70.05%	6.89%	70.45%	8.04%	68.14%	8.82%	71.46%	7.11%	72.92%	9.26%	72.54%	7.54%
DCCA-AM	71.67%	11.01%	72.03%	8.15%	80.14%	9.01%	75.74%	6.92%	78.93%	8.83%	75.44%	8.82%
MSBLS	77.31%	6.34%	77.68%	5.40%	86.06%	6.48	83.40%	5.44%	85.76%	8.19%	82.83%	4.57%
HMSB-CAF	79.13%	5.87%	78.68%	5.15%	90.53	5.87	84.04%	5.03%	89.27%	7.34%	84.19%	5.44%

基于脑电和眼动信号的不同文化情绪识别研究

Emotion recognition across different cultures based on EEG and eye-tracking signals

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 10

相关文章 2

编辑推荐

Metrics

本文评价

试验组	GUFN	KAN	DW	BFN	MSCP	SEED	SEED-FRA	SEED-GER
Baseline(DCCA-AM)	×	×	×	×	×	92.36%	79.14%	89.45%
GUFN	√	×	×	×	×	94.42%	85.80%	93.45%
GUFN+K	√	√	×	×	×	96.44%	88.19%	93.58%
GUFN+KD	√	√	√	×	×	96.87%	88.61%	94.11%
GUFN+KDB	√	√	√	√	×	97.13%	89.89%	95.17%
HMSB-CAF	√	√	√	√	√	97.41%	90.59%	95.37%

[1]	刘玉婷, 吴彤, 任祥辉. 基于Nginx负载均衡的大规模脑电信号处理集群架构设计与实现[J]. 信息通信技术与政策, 2025, 51(3): 16-24.
[2]	胡碧荣, 王晓东, 王浩冲, 史改革. 面向脑机接口测试的高精度脑电信号模拟发生系统设计与实现^*[J]. 信息通信技术与政策, 2024, 50(5): 34-40.