TriM-SOD: A Multi-Modal, Multi-Task, and Multi-Scale Spacecraft Optical Dataset

The acquisition and application of spacecraft optical data is an important part of space-based situational awareness (SSA). Spacecraft optical data processing techniques can assist in tasks such as on-orbit operation, space debris removal, and deep space exploration. However, the extreme lack of real spacecraft optical data is an insurmountable difficulty, which hinders the development of deep learning-based data processing techniques. Existing synthetic datasets usually only contain visible-light images, only support a specific task, and lack diversity in the scale of the spacecraft, which cannot adapt to actual application environments. Therefore, we propose a multi-modal, multi-task, and multi-scale spacecraft optical dataset (TriM-SOD), which has 3 superiorities: (a) multi-modal: it includes data in various modals, such as visible light and infrared; (b) multi-task: it includes labels for multiple tasks, such as spacecraft detection and spacecraft component segmentation; and (c) multi-scale: it features a variety of sizes for spacecraft in the images. To validate the effectiveness of our dataset and evaluate the performance of methods in the tasks, we use TriM-SOD to train and test several typical or recent methods for object detection and semantic segmentation. TriM-SOD has been made public and can be used as a benchmark to further promote the future development of SSA.