Abstract:

To address the problems of insufficient multimodal feature mining and limited generalization in emotion recognition across different cultures, this paper proposes a Hierarchical Multi-scale Branch Residual Transformation-Canonical Correlation Attention Fusion Network, which consists of multi-scale feature extraction, canonical correlation analysis-based enhancement, and attention-weighted fusion modules. Based on the Chinese, German, and French subsets of the SJTU emotion electroencephalography dataset (SEED), namely SEED-CHN, SEED-GER, and SEED-FRA, intra-cultural subject-dependent, intra-cultural subject-independent, and cross-cultural subject-independent experiments were conducted. The experimental results show that, in the intra-cultural subject-dependent experiments, the proposed method outperforms several baseline methods overall, indicating that it has good emotion recognition performance and stability in different cultural scenarios.

Key words: cross-cultural emotion recognition, electroencephalography signals, eye movement signals, multiscale feature extraction, attention fusi