最小二乘支持向量机在强化学习系统中的应用

将连续状态空间下的Q学习构建为最小二乘支持向量机的回归估计问题,利用最小二乘支持向量机良好的泛化以及非线性逼近性能实现由系统状态-动作对到Q值函数的映射。为了保证计算速度以及适应Q学习系统在线学习的需要,最小二乘支持向量机的训练样本是窗式移动的,即在Q学习系统学习的同时获取样本数据并进行最小二乘支持向量机的训练。小车爬山控制问题的仿真结果表明该方法学习效率高,能够有效解决强化学习系统连续状态空间的泛化问题。

Application of Least Squares Support Vector Machine to Reinforcement Learning System

Q learning system for continuous state space was constructed as a regression problem for a least squares support machine (LS-SVM). This LS-SVM was used to realize a mapping from a state-action pair to a Q value function by taking full advantage of perfect generalization and nonlinear approximation properties of LS-SVM. In order to improve computation speed and satisfy the requirements of on-line Q learning system, training samples of the LS-SVM were introduced in the manner of a time-window, i.e., the training samples are obtained to train the LS-SVM at the same time as the Q learning system progresses. Simulation results of a mountain car control show that the proposed Q learning method is highly efficient and can effectively solve the generalization problem of continuous state space.