一种量化因子自适应学习量化训练算法

深度神经网络因参数量过多而影响嵌入式部署，解决的办法之一是模型小型化（如模型量化，知识蒸馏等）。针对这一问题，提出了一种基于BN（batch normg lization）折叠的量化因子自适应学习的量化训练算法（简称为LSQ-BN算法）。采用单个CNN（convolutional neural）构造BN折叠以实现BN与CNN融合；在量化训练过程中，将量化因子设置成模型参数；提出了一种自适应量化因子初始化方案以解决量化因子难以初始化的问题。实验结果表明：8bit的权重和激活量化，量化模型的精度与FP32预制模型几乎一致；4bit的权重量化和8 bit的激活量化，量化模型的精度损失在3%以内。因此，LSQ-BN是一种优异的模型量化算法。

A Quantization Training Algorithm of Adaptive Learning Quantization Scale Fators

Deep neural network model is difficult to effectively deploy in embedded terminals due to its excessive number of components, andone of the solutions is model miniaturization (such as model quantization, knowledge distillation, etc.). To address this problem, a quantization training algorithm (referred to as LSQ-BN algorithm) based on adaptive learning of quantizationscale factors with BN folding is proposed.A single CNN (convolutional neural) is usedtoconstruct BN folding and achieve BN and CNN fusion. During the process of quantitative training,the quantization scale factors are set as model parameters. An adaptive quantizationscale factor initialization scheme is proposed to solve the problem of difficult initialization of quantizationscale factors.The experimental results show that the precision of the quantized model is almost the same as that of the FP32 prefabricated model when the weight and activation are both 8bit quantization. When the weight is 4 bit quantization and the activation is 8bit quantization, the precision loss of the quantization model is within 3%. Therefore, LSQ-BN proposed in this paper is an excellent model quantization algorithm.