SynBERT: Chinese Synonym Discovery on Privacy-Constrain Medical Terms with Pre-trained BERT

Discovering medical synonym sets (i.e.,set of terms referring to a similar medical concept) is an important task in real-world, which can benefit many downstream applications such as medical information retrieval system and clinical decision support system. Recent synonym discovery methods take words as the input unit and leverage raw text as contextual information. However, they are ill-suited in Chinese participle as taking word as the input unit leads to serious Out-of-Vocabulary (OOV) problems. Additionally, it is hard to get large-scaled raw clinical texts in medical domain because of the privacy and security. Therefore, we define a new task discovering Chinese synonym from Privacy-Constrain terms (i.e., only terms without raw corpus) and propose a framework SynBERT to solve it. SynBERT consists of a binary classifier, inferring whether two term sets can form a synonym set, and two-phase clustering algorithm, applying classifier to cluster given terms into different synonym sets. In particular, SynBERT composes term’s embedding with character’s embedding to address the OOV problems. SynBERT introduces a BERT model pre-trained on public large-scaled corpus before to eliminate the need of raw context information. Ȧccording to our experiment, SynBERT outperforms better than baseline methods such as Kmeans, L2C, SynSetMine, etc.