复杂任务下的多智能体协同进化算法

针对多智能体系统在处理复杂任务时存在的低效率、高冗积、多智能体系统内协同模型算法存在交互冲突、资源损耗过高等问题,提出一种基于复杂任务的多智能体系统优化算法.在差分进化算法与局部优化算法的基础上对二者进行优化,结合强化学习的训练框架,构建训练网络,通过对学习步长进行修订,改变种群迭代优化准则,使得种群在计算力充足的情况...     

MADDPG算法并行优先经验回放机制

多智能体深度确定性策略梯度(multi-agent deep deterministic policy gradient,MADDPG)算法是深度强化学习方法在多智能体系统(multi-agent system,MAS)领域的重要运用,为提升算法性能,提出基于并行优先经验回放机制的MADDPG算法.分析算法框架及训练方...     

Dynamic Decision Model in Evolutionary Games Based on Reinforcement Learning

In evolutionary games, it becomes more difficult to choose optimal strategies for players because of incomplete information and bounded rationality. For bounded rational players, how to maximize the expected sum of payoffs by learning and changing strategies is an important question in evolutionary game theory. Reinforcement learning does not need a model of its environment and can be used online, it is well-suited for problems with incomplete and uncertain information. Evolutionary game theory is the subject about the decision problems of multiagent with incomplete information. In this article, reinforcement learning is introduced in evolutionary games, multiagent reinforcement learning model is constructed, and the learning algorithm is presented based on Q-learning. The results of simulation experiments show that the multiagent reinforcement learning model can be applied successfully in evolutionary games for finding the optimal strategies.     

基于PSO神经网络的进化博弈研究

针对进化博弈中博弈人是有限理性的,提出了一种基于粒子群神经网络的进化博弈决策机制。该机制将神经网络技术引入到进化博弈中,并采用粒子群优化算法(PSO算法)来训练神经网络,因而可利用神经网络来模拟博弈人在进化过程中的学习和策略调整。利用该机制分别对有限理性条件下的鹰-鸽博弈和重复囚徒困境博弈进行了研究。实验表明:PSO神经网络可以准确地模拟进化博弈中博弈人的动态学习与决策过程,能有效地指导博弈人的策略选取,是进化博弈分析的有力工具。     

基于协作式深度强化学习的火灾应急疏散仿真研究

火灾是威胁公共安全的主要灾害之一,火灾产生的高温和有毒有害烟气严重影响了疏散路径的选择。将深度强化学习引入到应急疏散仿真研究,针对多智能体环境提出了协作式双深度Q网络算法。建立随时间动态变化的火灾场景模型,为人员疏散提供实时的危险区域分布信息;对各自独立的智能体神经网络进行整合,建立多智能体统一的深度神经网络,实现所有智能体之间的神经网络和经验共享,提高整体协作疏散效率。结果表明：所提方法具有良好的稳定性和适应性,训练和学习效率得到提升,具有良好的应用价值。     

基于自适应SGD-多智能体的防空资源部署优化

针对分布式环境下的战场指挥资源部署存在的效率低、速度慢、无法达到预期战略、数据集过大导致计算资源损耗过大等问题,提出了一种分布式环境下多智能体联盟的指挥控制资源部署优化算法。通过对深度学习中的梯度下降算法进行学习率的改进,将原本设定的学习率改为自适应的学习率,进而对指挥控制资源部署进行多智能体联盟的设计。仿真证明了该算法对此问题具有优越的适应性,可以高效地解决分布式环境下的多智能体联盟的指挥控制资源部署优化问题。     

针对无人潜航器的反潜策略研究

近年来无人潜航器对国家海洋国土安全带来的威胁逐渐增大,其低噪声特性和隐蔽入侵方式也给反潜行动带来极大困难。为此,提出了一种两阶段规划算法,用以学习优化反潜策略,在部署阶段,建立了基于不确定性马尔可夫决策过程的反潜资源分配模型,并设计了鲁棒性部署策略强化学习算法,用以求解不确定条件下分配模型的纳什均衡解。在搜索阶段,建立了基于部分可观察马尔可夫决策过程的搜潜模型,并设计了基于多智能体强化学习的搜潜策略学习算法。最后,通过仿真实验验证了本算法与比对算法相比具有更高的性能。     

基于量子多目标进化算法的多任务Agent联盟生成

多Agent联盟生成是多Agent系统的关键问题之一, 主要研究如何在多Agent系统中动态生成面向任务的最优联盟. 为使Agent能稳定的组织起来完成单Agent不能完成的任务并在成本、资源、利益等方面达到一个良好的平衡性能并达到全局最优, 提出了联盟多目标综合评价模型, 并将量子进化多目标算法应用于多目标多任务Agent联盟问题, 运用编码的映射, 将资源组合和任务分配合并为一个过程, 降低了问题的复杂性. 对比实验结果表明该算法求得的解的质量高, 平衡性好, 能有效避免了联盟死锁和资源浪费.     

基于EWA学习的共享资源捐赠习俗演化仿真研究

针对个体策略学习可能对共享资源捐赠习俗演化路径产生影响,提出用EWA学习模型刻画个体在现实场景下的策略调整过程,建立多个体EWA学习的演化博弈模型,并基于EWA学习给出算法流程,进行计算机仿真实验。仿真结果表明,在EWA学习支配下,随着迭代期次的增加,供给策略对个体的吸引度越来越大,个体的平均支付及联合支付也相对较高,能够克服共享资源供给上的囚徒困境,使共享资源捐赠习俗发育成为可能。     

Optimal substructure of set-valued solutions of normal-form games and coordination

A number of solution concepts of normal-form games have been proposed in the literature on subspaces of action profiles that have Nash type stability. While the literature mainly focuses on the minimal of such stable subspaces, this paper argues that non-minimal stable subspaces represent well the multi-agent situations to which neither Nash equilibrium nor rationalizability may be applied with satisfaction. As a theoretical support, the authors prove the optimal substructure of stable subspaces regarding the restriction of a game. It is further argued that the optimal substructure characterizes hierarchical diversity of coordination and interim phases in learning.     

基于分布式协商进化算法的多Agent目标冲突消解

针对多Agent系统研究中的目标冲突消解问题,建立了在多个Agent的局部目标和系统全局目标间进行协调优化的多目标优化模型.在多Agent分布式规划的框架下,提出了一种基于遗传算法(genetic algorithm,GA)的分布式协商进化算法,用于求解多目标规划模型.针对GA搜索中保持解的多样性、提高收敛速度等问题,对选择算子进行了设计.通过仿真实验,证明新的选择算子能有效提高解的质量.最后将该算法应用于部队机动协同路线规划的目标冲突消解问题,验证了其有效性.     

基于Swarm的股权拍卖机制设计与仿真研究

基于多主体的建模仿真方法，运用particle swarm optimization（PSO）群体智能算法模拟信息交互条件下外部投资者估价变化的学习机制和演化规律，在机制设计的基础上，建立了实现风险投资退出的股权拍卖模型.在Swarm平台上对股权拍卖模型的仿真分析表明，所设计的股权拍卖机制能够显著地提高风险投资家的收益，并能帮助风险投资家预测外部投资者的估价和拍卖参与度的变化.对股权拍卖模型的参数仿真发现，风险投资家可以通过引入更多的外部投资者参与股权拍卖来进一步提高自己的收益；即便外部投资者过度强化单一学习能力，最终也可以得到相对理想的股权拍卖结果.本文的研究可以为风险投资家的策略选择提供参考依据.     

Distributed Algorithm for Robust Resource Allocation with Polyhedral Uncertain Allocation Parameters

This paper studies a distributed robust resource allocation problem with nonsmooth objective functions under polyhedral uncertain allocation parameters. In the considered distributed robust resource allocation problem, the (nonsmooth) objective function is a sum of local convex objective functions assigned to agents in a multi-agent network. Each agent has a private feasible set and decides a local variable, and all the local variables are coupled with a global affine inequality constraint, which is subject to polyhedral uncertain parameters. With the duality theory of convex optimization, the authors derive a robust counterpart of the robust resource allocation problem. Based on the robust counterpart, the authors propose a novel distributed continuous-time algorithm, in which each agent only knows its local objective function, local uncertainty parameter, local constraint set, and its neighbors’ information. Using the stability theory of differential inclusions, the authors show that the algorithm is able to find the optimal solution under some mild conditions. Finally, the authors give an example to illustrate the efficacy of the proposed algorithm.     

基于多智能体强化学习的多机器人协作策略研究

研究了一种基于智能体动作预测的多智能体强化学习算法. 在多智能体系统中,学习智能体选择动作不可避免地要受到其他智能体执行动作的影响,因此强化学习系统需要考虑多智能体的联合状态和联合动作.基于此,提出使用概率神经网络来预测其他智能体动作的方法,从而构成多智能体联合动作,实现了多智能体强化学习算法. 此外,研究了该方法在足球机器人协作策略学习中的应用,使多机器人系统能够通过与环境的交互学习来掌握行为策略,从而实现多机器人的分工和协作.     

雷达海杂波反演大气波导的改进回溯搜索算法

了克服基本回溯搜索算法在大气波导反演问题中出现的收敛速度慢、容易陷入局部最优的缺点,提出一种基于反向学习机制和正交交叉机制的改进回溯搜索优化算法。该算法利用反向学习机制来选择较好的初始化种群,而正交交叉机制用来帮助算法加强全局搜索能力,避免算法陷入局部最优,从而提高算法的精度。通过常见测试函数的优化问题以及大气波导的反演问题来检验算法的性能。结果表明,所提算法具有较高的精度和较快收敛速度。     

基于多智能体强化学习的动态频谱分配方法综述

认知无线电和动态频谱分配技术是解决频谱资源短缺问题的有效手段。随着近年来深度学习和强化学习等机器学习技术迅速发展, 以多智能体强化学习为代表的群体智能技术不断取得突破, 使得分布式智能动态频谱分配成为可能。本文详细梳理了强化学习和多智能体强化学习领域关键研究成果, 以及基于多智能体强化学习的动态频谱分配过程建模方法与算法研究。并将现有算法归结为独立Q-学习、合作Q-学习、联合Q-学习和多智能体行动器-评判器算法4种, 分析了这些方法的优点与不足, 总结并给出了基于多智能体强化学习的动态频谱分配方法的关键问题与解决思路。     

基于分层SLAM的空地多智能体协同导航

针对单一智能体在导航过程中存在全球导航卫星系统（global navigation satellite system,GNSS）易受遮挡或干扰,惯性导航存在误差累积的问题,提出基于视觉的分层即时定位与地图构建（simultaneous localization and mapping,SLAM）空地多智能体协同算法。通过建立系统模型,采用基于扩展卡尔曼滤波融合欧氏点、逆深度点、锚定同质点3种不同特征点的分层SLAM算法,实现了对导航系统的辅助和增强。针对空地协同场景设计并开展了仿真实验。结果表明,空地多智能体协同算法可以将位置误差降低40%;而在使用锚定同质点以后,误识别率由49%降低至4%。实验验证该算法具有良好的定位精度、实用性和有效性。     

基于信息流改进的贝叶斯网络结构学习算法

基于信息流提出贝叶斯网络结构学习的改进型搜索评分算法。首先计算信息流进行全局因果分析,构造0/1优化问题,获得最优初始网络结构;在此初始结构的基础上产生搜索空间,采用贪婪算法搜索最优结构弧,同时由信息流确定弧方向,实现网络结构的一体化学习。首次将信息流引入贝叶斯网络的结构学习,优化了初始搜索空间,实现了弧和弧方向的同步确定,更能获得近似全局最优结构。实验表明,改进算法较其他算法的准确性和学习效率更高。     

Convergence of Distributed Gradient-Tracking-Based Optimization Algorithms with Random Graphs

This paper studies distributed convex optimization over a multi-agent system, where each agent owns only a local cost function with convexity and Lipschitz continuous gradients. The goal of the agents is to cooperatively minimize a sum of the local cost functions. The underlying communication networks are modelled by a sequence of random and balanced digraphs, which are not required to be spatially or temporally independent and have any special distributions. The authors use a distributed gradient-tracking-based optimization algorithm to solve the optimization problem. In the algorithm,each agent makes an estimate of the optimal solution and an estimate of the average of all the local gradients. The values of the estimates are updated based on a combination of a consensus method and a gradient tracking method. The authors prove that the algorithm can achieve convergence to the optimal solution at a geometric rate if the conditional graphs are uniformly strongly connected, the global cost function is strongly convex and the step-sizes don't exceed some upper bounds.     

Distributed Consensus-Based K-Means Algorithm in Switching Multi-Agent Networks

This paper discusses a distributed design for clustering based on the K-means algorithm in a switching multi-agent network, for the case when data are decentralized stored and unavailable to all agents. The authors propose a consensus-based algorithm in distributed case, that is, the doubleclock consensus-based K-means algorithm(DCKA). With mild connectivity conditions, the authors show convergence of DCKA to guarantee a distributed solution to the clustering problem, even though the network topology is time-varying. Moreover, the authors provide experimental results on various clustering datasets to illustrate the effectiveness of the fully distributed algorithm DCKA, whose performance may be better than that of the centralized K-means algorithm.