An Organ-Aware Diagnosis Framework for Radiology Report Generation

Shiyu Li, Pengchong Qiao, Lin Wang, Munan Ning, Li Yuan, Yefeng Zheng*, Jie Chen*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
Plum Print visual indicator of research metrics
  • Citations
    • Citation Indexes: 2
  • Captures
    • Readers: 2
see details

Abstract

Radiology report generation (RRG) is crucial to save the valuable time of radiologists in drafting the report, therefore increasing their work efficiency. Compared to typical methods that directly transfer image captioning technologies to RRG, our approach incorporates organ-wise priors into the report generation. Specifically, in this paper, we propose Organ-aware Diagnosis (OaD) to generate diagnostic reports containing descriptions of each physiological organ. During training, we first develop a task distillation (TD) module to extract organ-level descriptions from reports. We then introduce an organ-aware report generation module that, for one thing, provides a specific description for each organ, and for another, simulates clinical situations to provide short descriptions for normal cases. Furthermore, we design an auto-balance mask loss to ensure balanced training for normal/abnormal descriptions and various organs simultaneously. Being intuitively reasonable and practically simple, our OaD outperforms SOTA alternatives by large margins on commonly used IU-Xray and MIMIC-CXR datasets, as evidenced by a 3.4% BLEU-1 improvement on MIMIC-CXR and 2.0% BLEU-2 improvement on IU-Xray.

Original languageEnglish
Pages (from-to)4253-4265
Number of pages13
JournalIEEE Transactions on Medical Imaging
Volume43
Issue number12
DOIs
Publication statusPublished - 2024
Externally publishedYes

Keywords

  • Medical report generation
  • chest X-ray
  • data imbalance
  • organ-aware learning

Fingerprint

Dive into the research topics of 'An Organ-Aware Diagnosis Framework for Radiology Report Generation'. Together they form a unique fingerprint.

Cite this

Li, S., Qiao, P., Wang, L., Ning, M., Yuan, L., Zheng, Y., & Chen, J. (2024). An Organ-Aware Diagnosis Framework for Radiology Report Generation. IEEE Transactions on Medical Imaging, 43(12), 4253-4265. https://doi.org/10.1109/TMI.2024.3421599