注意力机制下双模态交互融合的目标跟踪网络

针对当前目标跟踪难以适应低光照、运动模糊、目标快速移动等挑战,提出了空间通道注意力下的红外与可见光双模态交互融合跟踪网络.首先,红外图像与可见光图像通过backbone三层卷积提取分层特征,并降维至统一分辨率,之后级联三层特征形成各模态特征.其次,多模态特征通过所设计的空间通道自注意力模块和跨模态交互注意力模块使得模态...

Target tracking network based on dual-modal interactive fusion under attention mechanism

Aiming at the challenges of current object tracking that is difficult to low illusion,motion blur,and fast motion,a dual-modal interacive fusion tracking network of infrared and visible under spatial channel attention is proposed.First,the infrared and RGB images are extracted through the backbone three-layer convalution to extract layered features which are normalized to the same resolution via dimension reduction.The three-layer features are cascaded to form each modal feature.Then the features are sent to the designed spatial channel self-attention module and the cross-module interactive attention module which lead network focus on global spatial features and high-response channels and therefore improve the complementarity of the dual-modal information.The interacted features of the dual-modal are cascaded for the fusion and finally sent to three fully connected layers to complete the target tracking.The experimental results of the largest RGB-Themeral(RGB-T)tracking data set RGBT234 show that the proposed network can effectively extract dual-modal interactive features and improve target tracking accuracy.Its Precision/Success Rateis improced by 5.3%and 4.2%,respectively,compared with the baseline network.