Markov控制过程基于神经元动态规划的优化算法

论文在Markov性能势理论基础上，研究了Markov控制过程在神经元网络等逼近结构表示的随机平稳策略作用下的仿真优化算法，分析了它们在一个无限长的样本轨道上概率1的收敛性，并给出了一个三－状态受控Markov过程的数值实例。

Optimization Algorithms for Markov Control Processes Using Neuro-dynamic Programming

Motivated by the needs of on line optimization of real word engineering systems, single sample path based optimization algorithms were studied for Markov control processes controlled by randomized stationary policies. The concept of Markov performance potential is introduced, and the policies can be represented by some approximate architectures such as neural networks. Unlike traditional computation based approaches, the policy parameters can be iterated and an optimal (or suboptimal) randomized stationary policy can be found according to a sample path obtained by observing the operation of a real system.This optimization method is a form of neuro dynamic programming methodology. The algorithms provided here have good adaptability as they can be used in different real systems, with a suitable choice of the parameters in the algorithms. Finally, the convergence of the algorithms with probability one on an infinite sample path is considered, and a numerical example for a three state controlled Markov chain is provided.