Reliable Typical Case Diagnosis via Optimized Retrieval-Augmented Generation Techniques

Kaiyuan Zhang, Bo Wang, Changsen Yuan, Chong Feng*, Ge Shi

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Recent advances in Large Language Models (LLMs) have shown remarkable capabilities in various Natural Language Processing (NLP) tasks. However, their performance in specialized domains, particularly healthcare, is still limited by the lack of domain expertise and output reliability. In this paper, we present a retrieval-augmented framework for the CHIP 2024 Diagnostic Consistency Task in Typical Medical Case Records. Our approach addresses these limitations by combining the generative power of LLMs with the retrieval of relevant medical knowledge. Specifically, we introduce a hybrid retrieval mechanism that integrates traditional BM25 with dense retrieval methods, along with a context compression strategy and self-consistency verification module. This framework enables more reliable and accurate diagnostic predictions by leveraging both contextual similarities and domain-specific knowledge. Our method achieves an average F1 score of 0.9517 on the test set, demonstrating the effectiveness of our approach for specialized medical diagnosis tasks.

Original languageEnglish
Title of host publicationHealth Information Processing. Evaluation Track Papers - 10th China Health Information Processing Conference, CHIP 2024, Proceedings
EditorsYanchun Zhang, Qingcai Chen, Hongfei Lin, Lei Liu, Xiangwen Liao, Buzhou Tang, Tianyong Hao, Zhengxing Huang, Jianbo Lei, Zuofeng Li, Hui Zong
PublisherSpringer Science and Business Media Deutschland GmbH
Pages214-225
Number of pages12
ISBN (Print)9789819642977
DOIs
Publication statusPublished - 2025
Event10th China Health Information Processing Conference, CHIP 2024 - Fuzhou, China
Duration: 15 Nov 202417 Nov 2024

Publication series

NameCommunications in Computer and Information Science
Volume2458 CCIS
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937

Conference

Conference10th China Health Information Processing Conference, CHIP 2024
Country/TerritoryChina
CityFuzhou
Period15/11/2417/11/24

Keywords

  • Case Diagnosis
  • Large Language Model
  • Retrieval-Augmented Generation

Cite this