一种自适应步长的复合梯度加速优化算法

自适应步长加速(Adam)类算法由于其计算效率高、兼容性好的特点,成为近期相关领域的研究热点.针对Adam收敛速度慢的问题,本文基于当前梯度、预测梯度以及历史动量梯度,提出一种新型Adam类一阶优化算法——复合梯度下降法(C-Adam),并对其收敛性进行了理论证明.与其他加速算法的区别之处在于,C-Adam将预测梯度与历史动量区别开,通过一次真实的梯度更新找到下一次迭代更精准的搜索方向.利用两组常用测试数据集及45钢静拉伸破坏实验的实验数据对所提算法进行验证,实验结果表明C-Adam与其他流行算法相比较具有更快的收敛速度及更小的训练损失.

A Compound Gradient Acceleration Optimization Algorithm with Adaptive Step Size

In related researches, a class of adaptive iteration step size accelerated(Adam) algorithms becomes a research hotspot because of its high computational efficiency and compatibility. To solve the problem of Adam′s low convergence rate, based on the combination of current gradient, prediction gradient and historical momentum gradient, this paper proposed a new kind of Adam algorithm named as compound gradient descent method(C-Adam), and proved its convergence. The difference between C-Adam and other acceleration algorithms is that C-Adam distinguishes the prediction gradient from the historical momentum, and finds a more accurate search direction for the next iteration through a real gradient update. Using two testing data sets and the data of 45 steel static tensile experiment to test the C-Adam, the results show that the algorithm has faster convergence speed and smaller training loss compared with other popular algorithms.