TY - JOUR
T1 - MFS enhanced SAM
T2 - Achieving superior performance in bimodal few-shot segmentation
AU - Zhao, Ying
AU - Song, Kechen
AU - Cui, Wenqi
AU - Ren, Hang
AU - Yan, Yunhui
N1 - Publisher Copyright:
© 2023 Elsevier Inc.
PY - 2023/12
Y1 - 2023/12
N2 - Recently, Segment Anything Model (SAM) has become popular in computer vision field because of its powerful image segmentation ability and high interactivity of various prompts, which opens a new era of large vision foundation models. But is SAM really omnipotent? In this letter, we establish a comprehensive bimodal few-shot segmentation indoor dataset VT-840-5i, and compare SAM with eight state-of-the-art few-shot segmentation (FSS) methods on two benchmark datasets. Qualitative and quantitative experiment results show that although SAM is very effective in general object segmentation, it still has room for improvement in some challenging scenarios. Therefore, we introduce thermal infrared auxiliary information into the segmentation task and provide multiple fusion strategies (MFS) for readers to choose the most suitable approach for the specific task. Finally, we discuss several potential research trends about SAM in the future. Our test results are available at: https://github.com/VDT-2048/Bi-SAM.
AB - Recently, Segment Anything Model (SAM) has become popular in computer vision field because of its powerful image segmentation ability and high interactivity of various prompts, which opens a new era of large vision foundation models. But is SAM really omnipotent? In this letter, we establish a comprehensive bimodal few-shot segmentation indoor dataset VT-840-5i, and compare SAM with eight state-of-the-art few-shot segmentation (FSS) methods on two benchmark datasets. Qualitative and quantitative experiment results show that although SAM is very effective in general object segmentation, it still has room for improvement in some challenging scenarios. Therefore, we introduce thermal infrared auxiliary information into the segmentation task and provide multiple fusion strategies (MFS) for readers to choose the most suitable approach for the specific task. Finally, we discuss several potential research trends about SAM in the future. Our test results are available at: https://github.com/VDT-2048/Bi-SAM.
KW - Few-shot segmentation
KW - Gated prediction selection
KW - RGB-T SAM
KW - Segment anything
UR - http://www.scopus.com/inward/record.url?scp=85172861803&partnerID=8YFLogxK
U2 - 10.1016/j.jvcir.2023.103946
DO - 10.1016/j.jvcir.2023.103946
M3 - Article
AN - SCOPUS:85172861803
SN - 1047-3203
VL - 97
JO - Journal of Visual Communication and Image Representation
JF - Journal of Visual Communication and Image Representation
M1 - 103946
ER -