A Reinforcement Learning Driven Method for Agile Earth Observation Satellite Scheduling Problem

Agile earth observation satellites (AEOS) play a critical role in improving our understanding and real-time sensing of the Earth and its environment. In recent years, as the demand for satellite observation and the number of missions have increased dramatically, the agile earth observation satellite mission scheduling problem (AEOSSP) has become increasingly demanding in terms of computational efficiency and algorithmic performance for large-scale solutions. In this paper, we propose a scheduling method based on reinforcement learning to address the above challenges. First, we model the problem as a multidimensional multi-knapsack problem with conflicts. Then, we build a neural network and design the reward function to reflect the problem-specific objectives and the penalty function to reflect the constraints. We train this network using the actorcritic model within reinforcement learning, combining it with the policy gradient method to directly optimize the total task gain. Finally, the trained model outputs a probability vector, which the model generates, and thus the optimal task allocation results are derived. In order to verify the effectiveness of the proposed algorithm, multiple sets of test cases are generated by combining the Satellite Tool Kit (STK) simulation. The related test results show that the proposed algorithm is better than the classical genetic algorithm in terms of algorithmic efficiency, task gain and its stability index, which verifies its computational efficiency in solving this data set.