基于深度强化学习的智能车辆行为决策研究

自动驾驶车辆决策系统直接影响车辆综合行驶性能，是实现自动驾驶技术需要解决的关键难题之一。基于深度强化学习算法DDPG（deep deterministic policy gradient），针对此问题提出了一种端到端驾驶行为决策模型。首先，结合驾驶员模型选取自车、道路、干扰车辆等共64维度状态空间信息作为输入数据集对决策模型进行训练，决策模型输出合理的驾驶行为以及控制量，为解决训练测试中的奖励和控制量突变问题，本文改进了DDPG决策模型对决策控制效果进行优化，并在TORCS（the open racing car simulator）平台进行仿真实验验证。结果表明本文提出的决策模型可以根据车辆和环境实时状态信息输出合理的驾驶行为以及控制量，与DDPG模型相比，改进的模型具有更好的控制精度，且车辆横向速度显著减小，车辆舒适性以及车辆稳定性明显改善。

Research on Intelligent Vehicles Behavior Decision-making Based on Deep Reinforcement Learning

Autonomous driving vehicle decision-making system has direct influence on driving performance. It is one of the key challenges to be addressed to realize fully autonomous driving. To solve this problem, a driving decision-making system based on deep reinforcement learning algorithm DDPG（deep deterministic policy gradient） is proposed in this paper. Firstly, A total of 64 dimensions of state spaces information such as ego vehicle information, road information and obstacle vehicle information on the basis of a driver model are selected as input variables of the constructed model. Then the decision-making is trained and outputs reasonable driving behaviors and control variable values. Finally, aiming at the problems of reward value and control variable values saltation, the DDPG decision model is improved to optimize decision control effect. To verify the performance of the proposed decision making model, simulation experiments are conducted on TORCS（the open racing car simulator）platform. The results show that the proposed decision-making model can output reasonable driving behaviors and accurate control quantities based on real-time state information of vehicles and environment, the improved decision-making model significantly improved driving safety and driving stability of ego vehicle.