跳到主要导航 跳到搜索 跳到主要内容

RET-CLIP: A Retinal Image Foundation Model Pre-trained with Clinical Diagnostic Reports

  • Jiawei Du
  • , Jia Guo
  • , Weihang Zhang
  • , Shengzhu Yang
  • , Hanruo Liu
  • , Huiqi Li*
  • , Ningli Wang
  • *此作品的通讯作者
  • Beijing Institute of Technology
  • Capital Medical University

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

The Vision-Language Foundation model is increasingly investigated in the fields of computer vision and natural language processing, yet its exploration in ophthalmology and broader medical applications remains limited. The challenge is the lack of labeled data for the training of foundation model. To handle this issue, a CLIP-style retinal image foundation model is developed in this paper. Our foundation model, RET-CLIP, is specifically trained on a dataset of 193,865 patients to extract general features of color fundus photographs (CFPs), employing a tripartite optimization strategy to focus on left eye, right eye, and patient level to reflect real-world clinical scenarios. Extensive experiments demonstrate that RET-CLIP outperforms existing benchmarks across eight diverse datasets spanning four critical diagnostic categories: diabetic retinopathy, glaucoma, multiple disease diagnosis, and multi-label classification of multiple diseases, which demonstrate the performance and generality of our foundation model. The sourse code and pre-trained model are available at https://github.com/sStonemason/RET-CLIP.

源语言英语
主期刊名Medical Image Computing and Computer Assisted Intervention – MICCAI 2024 - 27th International Conference, Proceedings
编辑Marius George Linguraru, Qi Dou, Aasa Feragen, Stamatia Giannarou, Ben Glocker, Karim Lekadir, Julia A. Schnabel
出版商Springer Science and Business Media Deutschland GmbH
页709-719
页数11
ISBN(印刷版)9783031723896
DOI
出版状态已出版 - 2024
活动27th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2024 - Marrakesh, 摩洛哥
期限: 6 10月 2024 → 10 10月 2024

丛书

姓名Lecture Notes in Computer Science
卷15012 LNCS
ISSN(印刷版)0302-9743
ISSN(电子版)1611-3349

会议

会议27th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2024
国家/地区摩洛哥
市Marrakesh
时期6/10/24 → 10/10/24

联合国可持续发展目标

此成果有助于实现下列可持续发展目标:

  1. 可持续发展目标 3 - 良好健康与福祉
    可持续发展目标 3 良好健康与福祉

学术指纹

探究 'RET-CLIP: A Retinal Image Foundation Model Pre-trained with Clinical Diagnostic Reports' 的科研主题。它们共同构成独一无二的学术指纹。

引用此