混合动力汽车深度强化学习分层能量管理策略

为了提高混合动力汽车的燃油经济性和控制策略的稳定性,以第三代普锐斯混联式混合动力汽车作为研究对象,提出了一种等效燃油消耗最小策略(equivalent fuel consumption minimization strategy,ECMS)与深度强化学习方法(deep feinforcement learning,DRL)结合的分层能量管理策略。仿真结果证明,该分层控制策略不仅可以让强化学习中的智能体在无模型的情况下实现自适应节能控制,而且能保证混合动力汽车在所有工况下的SOC都满足约束限制。与基于规则的能量管理策略相比,此分层控制策略可以将燃油经济性提高20.83%~32.66%;增加智能体对车速的预测信息,可进一步降低5.12%的燃油消耗;与没有分层的深度强化学习策略相比,此策略可将燃油经济性提高8.04%;与使用SOC偏移惩罚的自适应等效燃油消耗最小策略(A-ECMS)相比,此策略下的燃油经济性将提高5.81%~16.18%。

Deep reinforcement learning hierarchical energy management strategy for hybrid electric vehicles

To improve the fuel economy and control strategy stability of hybrid electric vehicles (HEVs), with taking the third-generation Prius hybrid electric vehicle as the research object, a hierarchical energy management strategy is created by combining an equivalent fuel consumption minimization strategy (ECMS) with a deep reinforcement learning (DRL) method. The simulation results show that the hierarchical control strategy not only enables the agent in reinforcement learning to achieve adaptive energy-saving control without a model, but also ensures that the state of charge (SOC) of the hybrid vehicle meets the constraints under all operating conditions. Compared with the rule-based energy management strategy, this layered control strategy improves the fuel economy by 20.83% to 32.66%. Additionally, increasing the prediction information of the vehicle speed by the agent further reduces the fuel consumption by about 5.12%. Compared with the deep reinforcement learning strategy alone, this combined strategy improves fuel economy by about 8.04%. Furthermore, compared with the A-ECMS strategy that uses SOC offset penalty, the fuel economy is improved by 5.81% to 16.18% under this proposed strategy.