基于Metropolis准则的多步Q学习算法与性能仿真

强化学习是目前智能体和机器学习研究的热点。针对强化学习中标准Q学习算法更新速度慢的缺点,通过引入多步信息更新策略和模拟退火中的Metropolis准则,提出了一种新颖的多步Q学习算法,称为SAMQ算法。仿真实验表明,与现有的算法相比,该算法能够有效提高收敛速度,较好地解决智能体选择动作时面临的新知识探索还是当前策略遵循的关键问题。

Metropolis Policy-based Multi-step Q Learning Algorithm and Performance Simulation

Increasing attention has been paid to reinforcement learning in Agent and machine learning community at present. By using multi-step information updating strategy and Metropolis criterion in simulated annealing,a new multi-step Q learning method, which is called simulated annealing-based multi-step Q learning(SAMQ), was proposed to compensate for the drawbacks of slow update speed in standard Q learning. Simulated experiments demonstrate that convergence speed can be effectively improved in this algorithm compared with other existing ones and the problem of new knowledge exploration or current policy exploitation that the Agent is confronted with when selecting actions can be resolved properly.