前向神经网络学习速率的自适应算法

学习速率是控制神经网络学习过程的一个重要参数,影响神经网络的稳定性和快速性。提出了一种能够满足实时性要求的神经网络学习速率的自适应算法,并证明了在该学习速率下,神经网络的学习过程是Lyapunov意义稳定的。该方法通过为神经网络的输出增加一个输出修正量来补偿多个未知因素对学习误差的影响,从而构造使学习误差快速收敛到零的学习速率自适应算法。通过对神经网络在线逼近一个非线性对象的过程进行仿真,结果证明了该方法的有效性。

Adaptive Algorithm of Learning Rate for Feedforward Neural Network

The learning rate is an important parameter for the learning process of a neural network(NN)which influents the stability and quickness of the NN.An adaptive algorithm of learning rate was proposed which satisfied the real-time requirement of the NN.The stability of the NN with such learning rate was proved in Lyapunov stability sense.By adding an amending part to the output of the NN to compensate,the influence of many unknown factors on the learning error,the method to adapt the learning rate was constructed,which could make the learning error converge quickly and stably.Simulation results show the efficiency of the algorithm.