平均排队长度差最小的单交叉口在线Q学习模型

建立了以平均排队长度差最小为优化目标的在线Q学习模型．针对控制性能指标相对于临近的配时方案不敏感的特点，提出了以平均排队长度差作为基本单位重新构造奖励函数，目的是拉大各行为对应的Q值差距，提高模型的收敛速度和鲁棒性．集成ExcelVBA、Vissim、Matlab建立了在线仿真平台，作为计算环境对模型进行了计算．利用GPS数据对Vissim软件中车辆加减速度曲线进行了标定．计算结果表明以平均排队长度差作为优化目标能够优化整个交叉口的时空资源，本文建立的在线Q学习模型具有较快的收敛速度和鲁棒性，通过学习能够实现优化目标．

On-line Q Learning Model for Minimizing Average Queue Length Difference

For adapting the randomness of traffic flow, the paper builds an on-line Q learning model for minimizing average queue length difference. Because performance index is approximate at the adjacent signal timing, the paper puts forward a method of building reward function to increase the gap between different behaviours to improve the robustness and computation speed. The paper integrates VBA, Vissim, and Matlab to build a simulation platform. Signal timing optimization of a single intersection with two phases is done by the on-line Q learning model, which can optimize the time and space resources of the intersection.