基于强化学习的三维游戏控制算法

基于强化学习,设计了一个面向三维第一人称射击游戏(DOOM)的智能体,该智能体可在游戏环境下移动、射击敌人、收集物品等.本文算法结合深度学习的目标识别算法Faster RCNN与Deep Q-Networks(DQN)算法,可将DQN算法的搜索空间大大减小,从而极大提升本文算法的训练效率.在虚拟游戏平台(ViZDoom)的两个场景下(Defend_the_center和Health_gathering)进行实验,将本文算法与最新的三维射击游戏智能体算法进行比较,结果表明本文算法可以用更少的迭代次数实现更优的训练结果.     

一种用于自主学习的虚拟仿真环境

赋予智能体通过与环境交互自主学习的能力是实现下一代人工智能的关键.本文,我们介绍了一种基于虚幻4的虚拟仿真环境,用于训练和测试自主智能体.该环境具有高逼真、可交互、灵活通用的特点,使得智能体能够在其中自由探索,自主学习场景感知、常识推理、决策控制等多项能力.为了验证该环境的可用性,我们用实验演示了如何在虚拟环境中构建自主智能,即利用强化学习方法训练端到端的神经网络实现基于视觉感知的目标搜索和目标追踪任务.     

基于群智能的NPC行为建模研究

人工智能技术的引入提高游戏中NPC的智能程度,但是NPC之间并没有通信能力,这一点极大地减弱NPC协同完成任务的能力,降低游戏的可玩性.针对NPC的协同处理能力问题,提出一种基于群智能的协同算法.该算法增加NPC之间的协同处理功能,有效地解决了NPC协同问题,提高游戏的可玩性,并从设计和时间上证明该算法的可行性.     

一种基于元学习的改进深度强化学习算法

传统的深度强化学习算法在解决任务时与环境交互量大且样本复杂度高,导致智能体的训练时间长,算法难以收敛,故在实际问题中的应用受限.针对该问题,在智能体采用梯度下降方法更新模型参数的过程中融入元学习思想,提出一种改进的深度强化学习算法,使得智能体利用在训练任务中学习到的先验知识快速地适应新任务.仿真结果表明:改进的深度强化学习算法可实现智能体在新任务上的快速适应,其收敛速度和稳定性等均优于传统算法.     

基于强化协作博弈方法的双车道混合交通流特性

对元胞自动机引入Gipps跟驰模型,并结合改进的Q强化学习方法分别建立普通车辆及智能网联车的微观行驶策略,提出了一种新型的混合交通流演化仿真方法.然后,利用数值模拟方式对双车道交通环境进行仿真,探索智能网联车对混合交通流的动态影响.结果表明,相比于元胞自动机构建的普通车辆智能体,改进的Q强化学习方法训练的智能网联车智能体具备更强的连续时空环境适应能力,双车道环境下道路通行能力随着智能网联车渗透率的提升而增大,最高可提升45.34%.此外,智能网联车渗透率的提高会降低车群低效的换道行为,拓宽高通行能力水平下的车辆密度范围,有利于改善交通拥堵.     

基于神经网络和模糊推理的移动机器人行为决策与控制

以足球机器人系统为实验平台 ,针对移动机器人智能决策中的实际问题 ,提出了一种基于径向基函数神经网络的机器人行为决策方法 ,通过神经元学习和训练以及自身的泛化能力 ,可以很好地利用多源信息进行机器人行为决策 ,以提高行为决策的有效性 .同时为了保证行为决策的实施效果 ,将模糊推理技术与传统的PID控制相结合 ,既保证了移动机器人系统运动控制的准确性和稳定性 ,又缩短了动态调整时间 ,取得了较好的控制效果 .     

景深约束下的深度强化学习机器人路径规划

为了提高未知环境下移动机器人的探索能力,基于深度强化学习训练提出一种基于最小深度信息有选择的训练模式,通过运动学方程约束,优化了状态空间的搜索与采集,提高了训练速率.在仿真未知环境中通过将RGB-D传感器的深度图像作为机器人的状态输入,学习模型将直接输出机器人的速度与角度并进行运动决策,验证了机器人路径规划控制策略.研究结果表明:在相同的训练时间下,所提出的训练模式对未知环境有更好的探索能力.     

针对不可微多阶段算法的环境升级式强化学习方法

多阶段算法的研究目前已取得很大进展,但仍存在2个重要问题。在推理阶段,信息不能从下游反馈到上游。在训练阶段,当整个模型涉及不可微函数时无法进行端到端的训练,因此不同阶段不能联合优化。提出一种新的环境升级式强化学习方法来解决反馈和联合优化问题,该方法的框架结构是通过一个强化学习智能体将下游阶段与上游阶段重新连接起来,利用优化上游阶段的输出来训练智能体,以提高最终性能,同时根据智能体的策略对下游阶段(环境)进行升级,实现智能体策略和环境的联合优化。针对智能体和环境的不同训练需求,还提出了一种基于该框架的训练算法,并在实例分割和人体姿态估计实验中证明了其有效性。     

基于近端策略优化的作战实体博弈对抗算法

针对一种大地图和稀疏奖励的兵棋推演对抗环境下,单纯的深度强化学习算法会导致训练无法快速收敛以及智能体对抗特定规则智能体胜率较低的问题,提出了一种基于监督学习和深度强化学习相结合以及设置额外奖励的方法,旨在提升智能博弈的训练效果。使用监督学习训练智能体;研究基于近端策略优化(Proximal policy optimization,PPO)的对抗算法;改进强化学习训练过程的额外奖励设置。以某在研兵棋推演环境为例的实验结果表明,该博弈对抗算法能使智能体在对抗其他智能体时的胜率稳步提升并在较短时间内达到收敛。     

基于改进强化学习的无人艇集群一致性控制

针对传统的建模研究方法在应用于无人水面艇集群时会遇到复杂的动态海洋环境问题,提出了一种新的多智能体马尔可夫决策过程控制框架,将一致性控制和势博弈理论结合起来.在强化学习过程中,通过映射每个智能体的动作-价值函数值(Q值)表到全局最大势函数表,从而得到最优联合决策矩阵用于协同控制.进行了仿真试验,根据平均回报值给出了分析结果,验证了控制器决策矩阵的自优化性,以及对于较大环境扰动的自适应性.     

Characterization of Ordovician carbonate reservoirs, southeastern Saskatchewan, Canada

The discovery of the prolific Ordovician Red River reservoirs in 1995 in southeastern Saskatchewan was the catalyst for extensive exploration activity which resulted in the discovery of more than 15 new Red River pools. The best yields of Red River production to date have been from dolomite reservoirs. Understanding the processes of dolomitization is, therefore, crucial for the prediction of the connectivity, spatial distribution and heterogeneity of dolomite reservoirs.The Red River reservoirs in the Midale area consist of 3～4 thin dolomitized zones, with a total thickness of about 20 m, which occur at the top of the Yeoman Formation. Two types of replacement dolomite were recognized in the Red River reservoir: dolomitized burrow infills and dolomitized host matrix. The spatial distribution of dolomite suggests that burrowing organisms played an important role in facilitating the fluid flow in the backfilled sediments. This resulted in penecontemporaneous dolomitization of burrow infills by normal seawater. The dolomite in the host matrix is interpreted as having occurred at shallow burial by evaporitic seawater during precipitation of Lake Almar anhydrite that immediately overlies the Yeoman Formation. However, the low δ18O values of dolomited burrow infills (-5.9‰～ -7.8‰, PDB) and matrix dolomites (-6.6‰～ -8.1‰, avg. -7.4‰ PDB) compared to the estimated values for the late Ordovician marine dolomite could be attributed to modification and alteration of dolomite at higher temperatures during deeper burial, which could also be responsible for its 87Sr/86Sr ratios (0.7084～0.7088) that are higher than suggested for the late Ordovician seawaters (0.7078～0.7080). The trace amounts of saddle dolomite cement in the Red River carbonates are probably related to "cannibalization" of earlier replacement dolomite during the chemical compaction.     

A computer generator for randomly layered structures

AcomputergeneratorforrandomlylayeredstructuresYUJia shun1,2,HEZhen hua2(1.TheInstituteofGeologicalandNuclearSciences,NewZealand;2.StateKeyLaboratoryofOilandGasReservoirGeologyandExploitation,ChengduUniversityofTechnology,China)Abstract:Analgorithmisintrod…     

海南岛地体及其毗邻陆缘晚中生代—新生代古地磁研究和构造演化

本文叙述了对海南岛及其毗邻大陆边缘白垩纪到第四纪地层岩石进行古地磁研究的全部工作过程。通过分析岩石中剩余磁矢量的磁偏角及磁倾角的变化,提出海南岛白垩纪以来经历的构造演化模式如下:早期伴随顺时针旋转而向南迁移,后期伴随逆时针转动并向北运移。联系该地区及邻区的地质、地球物理资料,对海南岛上述的构造地体运动提出以下认识:北部湾内早期有一拉张作用,主要是该作用使湾内地壳显著伸长减薄,形成北部湾盆地。从而导致了海南岛的早期构造运动,而海南岛后期的构造运动则主要是受南海海底扩张的影响。海南地体运动规律的阐明对于了解北部湾油气盆地的形成演化有重要的理论和实际意义。     

Exciton Seebeck in Molecular Systems

Various applications relevant to the exciton dynamics,such as the organic solar cell,the large-area organic light-emitting diodes and the thermoelectricity,are operating under temperature gradient.The potential abnormal behavior of the exicton dynamics driven by the temperature difference may affect the efficiency and performance of the corresponding devices.In the above situations,the exciton dynamics under temperature difference is mixed with     

Achieving Unusual States of Matter Under High Pressure

The elongation method,originally proposed by Imamura was further developed for many years in our group.As a method towards O(N)with high efficiency and high accuracy for any dimensional systems.This treatment designed for one-dimensional(ID)polymers is now available for three-dimensional(3D)systems,but geometry optimization is now possible only for 1D-systems.As an approach toward post-Hartree-Fock,it was also extended to     

《Chinese Science Bulletin》SUBJECT INDEX TO VOLUME 59(2014)

正~~     

An on-line scaling method for improving scalability of a database cluster

The explosive growth of the Internet and database applications has driven database to be more scalable and available, and able to support on-line scaling without interrupting service. To support more client's queries without downtime and degrading the response time, more nodes have to be scaled up while the database is running. This paper presents the overview of scalable and available database that satisfies the above characteristics. And we propose a novel on-line scaling method. Our method improves the existing on-line scaling method for fast response time and higher throughputs. Our proposed method reduces unnecessary network use, i.e. , we decrease the number of data copy by reusing the backup data. Also, our on-line scaling operation can be processed parallel by selecting adequate nodes as new node. Our performance study shows that our method results in significant reduction in data copy time.     

An improved R-tree based on childnode''s probability

R-Tree is a good structure for spatial searching. But in this indexing structure,either the sequence of nodes in the same level or sequence of traveling these nodes when queries are made is random. Since the possibility that the object appears in different MBR which have the same parents node is different, if we make the subnode who has the most possibility be traveled first, the time cost will be decreased in most of the cases. In some case, the possibility of a point belong to a rectangle will shows direct proportion with the size of the rectangle. But this conclusion is based on an assumption that the objects are symmetrically distributing in the area and this assumption is not always coming into existence. Now we found a more direct parameter to scale the possibility and made a little change on the structure of R-tree, to increase the possibility of founding the satisfying answer in the front sub trees. We names this structure probability based arranged R-tree (PBAR-tree).     

An Index mechanism for multi-scale view in spatial databases

There are numerous geometric objects stored in the spatial databases. An importance function in a spatial database is that users can browse the geometric objects as a map efficiently. Thus the spatial database should display the geometric objects users concern about swiftly onto the display window. This process includes two operations:retrieve data from database and then draw them onto screen. Accordingly, to improve the efficiency, we should try to reduce time of both retrieving object and displaying them. The former can be achieved with the aid of spatial index such as R-tree, the latter require to simplify the objects. Simplification means that objects are shown with sufficient but not with unnecessary detail which depend on the scale of browse. So the major problem is how to retrieve data at different detail level efficiently. This paper introduces the implementation of a multi-scale index in the spatial database SISP (Spatial Information Shared Platform) which is generalized from R-tree. The difference between the generalization and the R-tree lies on two facets: One is that every node and geometric object in the generalization is assigned with a importance value which denote the importance of them, and every vertex in the objects are assigned with a importance value,too. The importance value can be use to decide which data should be retrieve from disk in a query. The other difference is that geometric objects in the generalization are divided into one or more sub-blocks, and vertexes are total ordered by their importance value. With the help of the generalized R-tree, one can easily retrieve data at different detail levels.Some experiments are performed on real-life data to evaluate the performance of solutions that separately use normal spatial index and multi-scale spatial index. The results show that the solution using multi-scale index in SISP is satisfying.     

Integrating GIS Web services based on mediating architecture

The geographic information service is enabled by the advancements in general Web service technology and the focused efforts of the OGC in defining XML-based Web GIS service. Based on these models, this paper addresses the issue of services chaining,the process of combining or pipelining results from several interoperable GIS Web Services to create a customized solution. This paper presents a mediated chaining architecture in which a specific service takes responsibility for performing the process that describes a service chain. We designed the Spatial Information Process Language (SIPL) for dynamic modeling and describing the service chain, also a prototype of the Spatial Information Process Execution Engine (SIPEE) is implemented for executing processes written in SIPL. Discussion of measures to improve the functionality and performance of such system will be included.