多层次自适应知识蒸馏的轻量化高分遥感场景分类

针对深度神经网络模型计算量大且耗时，而轻量化模型速度快但精度低，二者均无法直接应用在嵌入式设备上的问题，本文提出一种多层次自适应知识蒸馏方法提升轻量化模型的性能。首先，针对遥感影像类别间差异程度不均衡，通过改进输出层知识蒸馏中的温度机制，提出一种自适应温度机制，促进学生模型更好地学习大而深的教师模型的输出层概率分布知识；然后，通过添加辅助卷积块以融入特征层的知识蒸馏方法，使学生模型学习教师模型的多层次知识。在两个高分遥感场景分类数据集上的实验验证本文方法的有效性。

Lightweight high-resolution remote sensing scene classification based on multi-level adaptive knowledge distillation

Aiming at the problem that the deep convolutional neural networks models are computationally expensive and time-consuming, while the lightweight models have fast speed but low accuracy, and both of them could not be directly applied to embedded devices, a multi-level adaptive knowledge distillation method is proposed to improve the performance of the lightweight model. Firstly, aiming at the uneven degree of difference between remote sensing image categories, an adaptive temperature mechanism is proposed in the knowledge distillation of the output layer, so as to promote the student model to better learn the probability distribution knowledge of the output layer of the large and deep teacher model; In addition, make the student model to learn the multilevel knowledge by adding auxiliary convolution blocks integrate into the knowledge distillation method of the feature layer from teacher model. Experiments on two high-resolution remote sensing scene classification datasets verify the effectiveness of this method.