基于策略迭代和遗传算法的 SMDP鲁棒控制策略求解

半马尔可夫决策过程(SMDP)描述的一类受控半Markov系统,其模型参数在实际中常常不确定或不可知,可能导致随机过程的性能函数和系统参数(即嵌入链转移概率和状态逗留时间分布)皆不确定。该文针对参数不相关的情况,给出求解鲁棒控制策略的迭代算法,并在迭代过程中引入遗传算法,以提高全局优化能力。数值例子表明,基于遗传算法的策略迭代应用于鲁棒决策问题中具有较好的优化效果。

Solution of the robust control policy for SMDPs based on the genetic algorithm and policy iteration

For a class of controlled semi-Markov systems,which are formulated as semi-Markov decision processes(SMDPs),some parameters are usually indeterminate or unknown,and the performance function or the system parameters,i.e.,the transition probabilities of the embedded chains and the sojourn time distribution of states,may be uncertain.For the case of independent parameters,a policy iteration is provided to derive the robust control policy,and the genetic algorithm is applied in order to improve the optimization result.The numerical example shows that the genetic algorithm-based policy iteration works well for robust decision problems.