Lightweight multi-level feature difference fusion network for RGB-D-T salient object detection

Kechen Song, Han Wang, Ying Zhao, Liming Huang, Hongwen Dong, Yunhui Yan*

*此作品的通讯作者

科研成果: 期刊稿件文章同行评审

6 引用 (Scopus)

摘要

In recent years, bimodal salient object detection has developed rapidly. In view of the advanced performance of their robustness to extreme situations such as background similarity and illumination variation, researchers began to focus on RGB-Depth-Thermal salient object detection (RGB-D-T SOD). However, most existing bimodal methods usually need expensive computational costs to complete accurate prediction, and this situation is even more serious for three-modal methods, which undoubtedly limits their applicability. To solve this problem, we are the first to propose a lightweight multi-level feature difference fusion network (MFDF) for real-time RGB-D-T SOD. In view of the depth modality contains less useful information, we design an asymmetric three-stream encoder based on MobileNetV2. Due to the differences in semantics and details between high and low level features, using the same module without discrimination will lead to a large number of redundant parameters. On the contrary, in the coding stage, we introduce a cross-modal enhancement module (CME) and a cross-modal fusion module (CMF) to fuse low-level and high-level features respectively. In order to reduce redundant parameters, we design a low-level feature decoding module (LFD) and a multi-scale high-level feature fusion module (MHFF). A great deal of experiments proves that the proposed MFDF has more advantages than the 17 state-of-the-art methods. On the efficiency side, MFDF has a faster speed (124 FPS when the image size is 320 × 320) and much fewer parameters (8.9 M).

源语言英语
文章编号101702
期刊Journal of King Saud University - Computer and Information Sciences
35
8
DOI
出版状态已出版 - 9月 2023
已对外发布

指纹

探究 'Lightweight multi-level feature difference fusion network for RGB-D-T salient object detection' 的科研主题。它们共同构成独一无二的指纹。

引用此

Song, K., Wang, H., Zhao, Y., Huang, L., Dong, H., & Yan, Y. (2023). Lightweight multi-level feature difference fusion network for RGB-D-T salient object detection. Journal of King Saud University - Computer and Information Sciences, 35(8), 文章 101702. https://doi.org/10.1016/j.jksuci.2023.101702