Actor-Critic框架下的多智能体决策方法及其在兵棋上的应用

将人工智能应用于兵棋推演的智能战术兵棋正逐年发展, 基于Actor-Critic框架的决策方法可以实现智能战术兵棋的战术行动动态决策。但若Critic网络只对单算子进行评价, 多算子之间的网络没有协同, 本方算子之间各自行动决策会不够智能。针对上述方法的不足, 提出了一种基于强化学习并结合规则的多智能体决策方法, 以提升兵棋推演的智能水平。侧重采用强化学习对多算子的行动决策进行决策分析, 并结合产生式规则对战术决策进行规划。构建基于Actor-Critic框架的多算子分布执行集中训练的行动决策模型, 对比每个算子互不交流的封闭式行动决策学习方法, 提出的分布执行集中训练方法更具优势且有效。

Multi-agent decision-making method based on Actor-Critic framework and its application in wargame

The intelligent tactical wargame which applies artificial intelligence to wargame deduction is developed year by year.The decision-making method based on Actor-Critic framework can realize the dynamic decision-making of tactical action of intelligent tactical wargame.However,if the Critic network only evaluates the single agent,and there is no cooperation among multiple agents,the decision-making of each agent will not be intelligent enough.In order to improve the intelligence level of wargame deduction,a multi-agent decision-making method based on reinforcement learning and rules is proposed.The decision analysis of the multi-agent action decision by using reinforcement learning is focuses,and combining with the production rules to plan tactical decision.An action decision model based on Actor-Critic framework for multi-agent distributed execution training is constructed.Compared with the closed action decision-making learning method in which each operator does not communicate with each other,the proposed distributed execution and centralized training method is more advantageous and effective.