SAM-Based Fine-Tuning Strategy and Application of Amphibious Environment Perception

In view of the high false alarm rate and multi-sensory task integration challenges faced by amphibious unmanned platforms in uncertain environments, in this study a multi-model joint environment perception method based on the segment anything model (SAM) was proposed, which achieved unified processing of obstacle detection and amphibious domain segmentation. Specifically, U-Net and YOLOv8 were combined with SAM. U-Net and YOLOv8 were responsible for obtaining the rough outline of the target, while SAM achieved fine segmentation through its encoding-decoding structure. In addition, a special fine-tuning strategy was designed to achieve joint training, which further improved the performance of the model. In this study, a proprietary dataset USV-Dataset was also constructed and a data engine was developed to improve the annotation efficiency. In order to enhance the generalization ability of the model, four public datasets were used for mixed training with USV-Dataset, covering a variety of scenarios and obstacle categories. Experimental results show that this method achieves 96.8% mPA segmentation accuracy and 10 FPS inference speed, showing good generalization ability and meeting the real-time environment perception needs of medium and low-speed amphibious unmanned platforms.