基于复合型2S网络的红外与可见光图像配准研究

针对传统图像配准方法在红外图像与可见光图像配准任务中效果较差的问题。提出了一种复合型2S网络即Superpoint+Superglue相结合的特征匹配法用于红外与可见光图像配准。方法中首先使用Superpoint独特的特征提取方法，充分提取红外图像与可见光图像之间的共性特征。其次利用Superglue特征匹配方法中增加匹配约束和使用注意力机制的思想，发挥神经网络的优势，提高匹配效率。在训练阶段通过使用自建数据集的方法，以提高神经网络的泛化性与准确性。结果表明：传统配准方法在三组实验图像上的特征点提取重复性评分与准确性评分分别为：（0.0067，0.0061）、（0.0010，0.0008）、（0，0），特征点正确匹配对数为：7对、1对、0对，平均数量低于估计变换矩阵所需要的最少四对匹配点对。而基于Superpoint+Superglue的红外与可见光图像配准方法的各项评分为：（0.2402，0.2625）、（0.1939，0.1722）、（0.2630，0.2644），特征点正确匹配对数为：252对、165对、252对，特征点提取评价指标与特征点对正确匹配数量相较于传统方法均大幅度提升，可以较好的完成配准任务。

Research on infrared and visible image registration based on compound 2S network

Aiming at the problem that traditional Image registration methods have poor effect in infrared and visible Image registration tasks. A composite 2S network, Superpoint+Superglue, is proposed for infrared and visible Image registration. The method first uses Superpoint''s unique feature extraction method to fully extract common features between infrared and visible light images. Secondly, the idea of adding matching constraints and using attention mechanism in Superlube feature matching method is used to give full play to the advantages of neural network and improve the matching efficiency. In the training phase, the method of using self built datasets is used to improve the generalization and accuracy of the neural network. The results show that the repeatability and accuracy scores of traditional registration methods for feature point extraction on three sets of experimental images are (0.0067, 0.0061), (0.0010, 0.0008), and (0, 0), respectively. The correct matching logarithms of feature points are: 7 pairs, 1 pair, and 0 pairs, with an average number lower than the minimum four matching point pairs required to estimate the transformation matrix. The scores of infrared and visible Image registration methods based on Superpoint+Superglue are (0.2402, 0.2625), (0.1939, 0.1722), (0.2630, 0.2644), and the correct matching logarithms of feature points are 252, 165, and 252 pairs. The evaluation index of feature point extraction and the number of correct matching of feature point pairs are significantly increased compared with traditional methods, which can better complete the registration task.