一种融合标签间强相关性的多标签图像分类方法

为了将标签间的语义相关性引入多标签图像分类模型中,传统的方法例如 ML-GCN 通过设置单阈值将标 签条件概率矩阵二值化为标签共现矩阵,然而,仅设置单阈值很难归纳所有的标签语义关系情况。 针对这一问题, 提出一种融合标签间强相关性的多标签图像分类方法—MGAN(Multiple Graph Convolutional Attention Networks), 通过设置多个阈值,将传统的标签条件概率矩阵按照不同的相关性程度分割为多个子图;同时,为了提升多标签分 类性能,也引入图像区域空间相关性。 另外,针对传统的“CNN+GCN”方法将标签与特征的融合张量视为预测分数 缺乏可解释性问题,将标签与特征的融合张量视为注意力分数;在 MS-COCO 和 PASCAL VOC 数据集上与其他主 流多标签图像分类方法进行了对比实验,平均准确率分别达到了 94. 9%和 83. 7%,相较于经典 ML-GCN 模型,分 别获得了 0. 9%和 0. 8%准确率提升,且在“Binary”和“Re-weighted”邻接矩阵模式下,MGAN 都有较好的表现,验证 了新的融合方法可以缓解图卷积神经网络过平滑问题对多标签图像分类的影响。

A Method of Multi-label Image Classification with Fusing Powerful Semantic Correlation

In order to introduce semantic correlation between labels into multi-label image classification model traditional methods such as ML-GCN transform label conditional probability matrix into label co-occurrence matrix by using single threshold value. However it is difficult to sum up all semantic relationships of all labels by using single threshold value. To solve this problem a method of multi-label image classification with fusing powerful semantic correlation MGAN was proposed. By setting multiple thresholds the traditional conditional probability matrix of labels was divided into multiple subgraphs according to different degrees of correlation. Meanwhile in order to improve the performance of multi-label classification image region spatial correlation was also introduced. In addition the traditional ??CNN +GCN method regards the fusion tensor of label and feature as the lack of interpretability of the predicted fraction. To solve this problem MGAN regards the labels and feature?? s fusion tensor as the attention score. Compared with other mainstream multi-label image classification methods on MS-COCO and PASCAL VOC datasets the mAP were 94. 9% and 83. 7% respectively which were 0. 9% and 0. 8% higher than traditional ML-GCN model. And MGAN performed well in both ??Binary and??Re-weighted adjacency matrix mode which verified that the new fusion method can alleviate the influence of graph convolutional neural network?? s ??over smoothing problem on multi-label image classification.