CGMRF: CLIP-guide multimodal registration and fusion for a visible–infrared dual-modality imaging system

Wenqu Zhao; Lingxue Wang; Lian Zhang; Dezhi Zheng; Yi Cai

doi:10.1364/OL.558572

CGMRF: CLIP-guide multimodal registration and fusion for a visible–infrared dual-modality imaging system

Wenqu Zhao
, Lingxue Wang^*
, Lian Zhang
, Dezhi Zheng
, Yi Cai

^*Corresponding author for this work

MIIT Key Laboratory of Complex-Field Intelligent Exploration
Beijing Institute of Technology
Beijing Institute of Technology (Zhuhai)

Research output: Contribution to journal › Article › peer-review

1 Citation (Scopus)

Abstract

In this Letter, we propose CLIP-guided multimodal registration and fusion (CGMRF), a semantic understanding-based multimodal image fusion system, for visible and infrared (IR) dual-modality imaging. CGMRF leverages semantic similarity, better aligned with human visual interpretation, to address the challenges of multimodal image registration and fusion. Experimental results across multiple metrics demonstrate the advantages of the proposed CGMRF system.

Original language	English
Pages (from-to)	3907-3910
Number of pages	4
Journal	Optics Letters
Volume	50
Issue number	12
DOIs	https://doi.org/10.1364/OL.558572
Publication status	Published - 15 Jun 2025
Externally published	Yes

Access to Document

10.1364/OL.558572

Cite this

@article{51f4893e7c904365908924bc09fdf3f2,

title = "CGMRF: CLIP-guide multimodal registration and fusion for a visible{\textendash}infrared dual-modality imaging system",

abstract = "In this Letter, we propose CLIP-guided multimodal registration and fusion (CGMRF), a semantic understanding-based multimodal image fusion system, for visible and infrared (IR) dual-modality imaging. CGMRF leverages semantic similarity, better aligned with human visual interpretation, to address the challenges of multimodal image registration and fusion. Experimental results across multiple metrics demonstrate the advantages of the proposed CGMRF system.",

author = "Wenqu Zhao and Lingxue Wang and Lian Zhang and Dezhi Zheng and Yi Cai",

note = "Publisher Copyright: {\textcopyright} 2025 Optica Publishing Group. All rights, including for text and data mining (TDM), Artificial Intelligence (AI) training, and similar technologies, are reserved.",

year = "2025",

month = jun,

day = "15",

doi = "10.1364/OL.558572",

language = "English",

volume = "50",

pages = "3907--3910",

journal = "Optics Letters",

issn = "0146-9592",

publisher = "Optica Publishing Group (formerly OSA)",

number = "12",

}

TY - JOUR

T1 - CGMRF

T2 - CLIP-guide multimodal registration and fusion for a visible–infrared dual-modality imaging system

AU - Zhao, Wenqu

AU - Wang, Lingxue

AU - Zhang, Lian

AU - Zheng, Dezhi

AU - Cai, Yi

PY - 2025/6/15

Y1 - 2025/6/15

N2 - In this Letter, we propose CLIP-guided multimodal registration and fusion (CGMRF), a semantic understanding-based multimodal image fusion system, for visible and infrared (IR) dual-modality imaging. CGMRF leverages semantic similarity, better aligned with human visual interpretation, to address the challenges of multimodal image registration and fusion. Experimental results across multiple metrics demonstrate the advantages of the proposed CGMRF system.

AB - In this Letter, we propose CLIP-guided multimodal registration and fusion (CGMRF), a semantic understanding-based multimodal image fusion system, for visible and infrared (IR) dual-modality imaging. CGMRF leverages semantic similarity, better aligned with human visual interpretation, to address the challenges of multimodal image registration and fusion. Experimental results across multiple metrics demonstrate the advantages of the proposed CGMRF system.

UR - https://www.scopus.com/pages/publications/105007683326

U2 - 10.1364/OL.558572

DO - 10.1364/OL.558572

M3 - Article

C2 - 40512904

AN - SCOPUS:105007683326

SN - 0146-9592

VL - 50

SP - 3907

EP - 3910

JO - Optics Letters

JF - Optics Letters

IS - 12

ER -

CGMRF: CLIP-guide multimodal registration and fusion for a visible–infrared dual-modality imaging system

Abstract

Access to Document

Other files and links

Fingerprint

Cite this