基于注意力机制多尺度网络的自然场景情绪识别

情绪识别作为计算机视觉的一项基本课题已经取得很大进展,然而在无约束自然场景中的情绪识别仍具挑战性.现有方法主要是利用人脸、姿态以及场景信息识别情绪,但是忽略了人物个体在场景中的不确定性,以及不能很好地挖掘场景中的情绪线索.针对现有研究存在的问题,提出了基于人物与场景线索的双分支网络结构,两个分支独立学习,通过早期融合得到情绪分类结果.对于人物在场景中的不确定性,引入身体注意力机制预判人物情绪置信度进而获得人体的特征表示,场景中引入空间注意力机制和特征金字塔以便充分获得场景中不同粒度的情绪线索.实验结果表明,此方法有效融合人物与场景信息,在EMOTIC数据集下能够明显提高情绪识别率.

Emotion recognition of the natural scenes based on attention mechanism and multi-scale network

Emotion recognition as a fundamental topic in computer vision has made tremendous progress, yet emotion recognition in unconstrained environments is still challenging. Existing methods mainly use face, posture, and scene information to recognize emotions, but these methods ignore the uncertainty of individuals in the context, and do not tap the emotional cues in the scene well. Aiming at the problems in existing research, a dual-branch network structure based on body and context cues is proposed. Two branches learning independently, then obtain the result of emotion classification through early fusion. For uncertainties of person in context, the body gesture attention mechanism is utilized to estimate the confidence coefficient and obtain the feature representation of body. For context branch, spatial attention mechanism and feature pyramid network are employed to fully obtain the emotional cues of different granularities in the scene. The experiment results demonstrated that the effectiveness of the proposed method in the EMOTIC dataset.