宏微网络中基于后决策状态学习的基站关断节能策略

为了根据网络的业务状态动态地调整基站的开关状态以在保证用户服务质量的同时降低宏微网络的能量消耗,提出将时延限制下长期平均能耗的最小化问题映射为受限马尔科夫决策过程(constrained Markov decision process,CMDP).在该过程中把网络中每个基站的用户数目定义为系统状态,将每个小基站的开/关操作定义为网络行动.为了充分利用网络已知的先验知识加快学习速度,采用后决策状态学习算法在线更新网络的开/关策略.该算法可根据关断策略执行前已知的网络状态学习关断策略执行后未知的网络状态,从而快速制定出相应的关断策略.理论验证了后决策状态学习算法的收敛性.通过仿真对比可知,后决策状态学习算法不仅学习速度快,且可在保证网络服务质量的同时收敛到最优的基站关断节能策略.

Post-decision state learning based on/off strategies for energy saving in heterogeneous networks

To adjust the on/off states of base stations (BSs) dynamically based on the network load for reducing the energy consumption of the heterogeneous networks while ensuring the quality of service (QoS) of users, this paper maps the problem about the minimization of long-term average energy consumption under delay limit to a constrained Markov decision process (CMDP), where the user numbers of all BSs are defined as system states and the switching operations of all SBSs are defined as network actions. To take full advantage of the network prior knowledge to speed up the learning rate, this paper adopts the post-decision state learning algorithm to update on/off strategies online. This algorithm will learn the unknown network states after implementing on/off strategies based on the known network states, and then the proper actions can be determined. Besides, the convergence of the post-decision state learning algorithm is theoretically verified. The comparative simulation demonstrates that the post-decision state learning algorithm not only learns faster, but can converge to the best on/off strategies while guaranteeing the service quality of the networks.