| 一种基于多任务学习的多模态情感识别方法 |
| |
| 引用本文： | 林子杰,龙云飞,杜嘉晨,徐睿峰.一种基于多任务学习的多模态情感识别方法[J].北京大学学报(自然科学版),2021,57(1):7-15. |
| |
| 作者姓名： | 林子杰 龙云飞 杜嘉晨 徐睿峰 |
| |
| 作者单位： | 1. 哈尔滨工业大学(深圳)计算机科学与技术学院, 深圳 518055
2. School of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ |
| |
| 基金项目： | 深圳市技术攻关项目;国家自然科学基金;广东省新冠肺炎疫情防控科研专项;深圳市基础研究学科布局项目 |
| |
| 摘 要： | 为了通过设置辅助任务学习到更具有情感倾向性的视频和语音表示,进而提升模态融合的效果,提出一种基于多任务学习的多模态情感识别模型,使用多模态共享层来学习视觉和语音模型的情感信息.在MOSI数据集和MOSEI数据集上的实验表明,添加两个辅助的单模态情感识别任务后,模型可以学习到更有效的单模态情感表示,并且在两个数据集上的情...
|
| 关 键 词： | 多模态信息 情感识别 模态融合 多任务学习 |
| 收稿时间： | 2020-06-08 |
A Multi-modal Sentiment Recognition Method Based on Multi-task Learning |
| |
| LIN Zijie,LONG Yunfei,DU Jiachen,XU Ruifeng.A Multi-modal Sentiment Recognition Method Based on Multi-task Learning[J].Acta Scientiarum Naturalium Universitatis Pekinensis,2021,57(1):7-15. |
| |
| Authors: | LIN Zijie LONG Yunfei DU Jiachen XU Ruifeng |
| |
| Institution: | 1. School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen 518055
2. School of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ |
| |
| Abstract: | In order to learn more emotionally inclined video and speech representations through auxiliary tasks, and improve the effect of multi-modal fusion, this paper proposes a multi-modal sentiment recognition method based on multi-task learning. A multimodal sharing layer is used to learn the sentiment information of the visual and acoustic modes. The experiment on MOSI and MOSEI data sets shows that adding two auxiliary single-modal sentiment recognition tasks can learn more effective single-modal sentiment representations, and improve the accuracy of sentiment recognition by 0.8% and 2.5% respectively. |
| |
| Keywords: | multi-modal information sentiment recognition multi-modal fusion multi-task learning |
| 本文献已被 CNKI 万方数据 等数据库收录！ |
| 点击此处可从《北京大学学报(自然科学版)》浏览原始摘要信息 |
| 点击此处可从《北京大学学报(自然科学版)》下载免费的PDF全文 |
