基于强化学习的全电推进卫星变轨优化方法

采用电推力器实现自主轨道转移是全电推进卫星领域的关键技术之一。针对地球同步轨道(geostationary orbit, GEO)全电推进卫星的轨道提升问题, 将广义优势估计(generalized advantage estimator, GAE)和近端策略优化(proximal policy optimization, PPO)方法相结合, 在考虑多种轨道摄动影响以及地球阴影约束的情况下, 提出了基于强化学习的时间最优小推力变轨策略优化方法。针对状态空间过大、奖励稀疏导致训练困难这一关键问题, 提出了动作输出映射和分层奖励等训练加速方法, 有效提升了训练效率, 加快了收敛速度。数值仿真和结果对比表明, 所提方法更加简单、灵活、高效, 与传统的直接法、间接法以及反馈控制法相比，能够保证轨道转移时间的最优性。

Optimization method for orbit transfer of all-electric propulsion satellite based on reinforcement learning

Using electric thrusters for autonomous orbit transfer is one of the critical technologies in the field of all-electric propulsion satellites. In order to solve the orbit raising problem of all-electric propulsion geostationary orbit (GEO) satellites, a reinforcement learning-based optimization method for the time-optimal low-thrust orbit transfer strategy is formulated by combining generalized advantage estimator (GAE) and proximal policy optimization (PPO) methods, taking into account the influence of multiple orbital perturbations and the constraints of the earth's shadow. Aiming at the key problem of training difficulty caused by too large state space and sparse reward, training acceleration methods such as action output mapping and hierarchical reward are proposed, which effectively improve the training efficiency and accelerate the convergence speed. Through numerical simulation and comparison of the results with the direct method, the indirect method and the feedback control method, it shows that the optimization method based on reinforcement learning is more simple, flexible, efficient, and time-optimal in orbit transfer.