一种使用log函数的新型修正激活单元LogReLU

提出一种新型校正激活函数的改进,该新型校正激活函数带有一个可变参数,使用对数函数对正区域的梯度进行矫正,解决了预测准确率较低的问题.改进的激活函数使用两个不同参数分别控制正区域和负区域的梯度.通过对两个不同数据集进行仿真实验的结果表明,新提出的两种方法效果均好于原始的修正线性单元,带有两个参数的改进使验证错误率分别降低了0.14%和5.33%.

LogReLU: A New Rectified Activation Unit Using Log Funcion

We proposed a new revised rectified activation function. The new rectified activation function has a changeable parameter, and the logarithmic function was used to rectify the gradient in positive region, which solved the problem of low accuracy of prediction. The improved activation function used two different parameters to control the gradient in positive region and negative region, respectively. The results of simulation experiments on two different data sets show that the effectiveness of the two new methods is better than that of the original rectified linear units. With the improvement of two parameters, the verification error rate is reduced by 014% and 533%, respectively.