TY - JOUR
T1 - Multi-view Test-time Adaptation for Semantic Segmentation in Clinical Cataract Surgery
AU - Li, Heng
AU - Ou, Mingyang
AU - Li, Haojin
AU - Qiu, Zhongxi
AU - Niu, Ke
AU - Fu, Huazhu
AU - Liu, Jiang
N1 - Publisher Copyright:
© 1982-2012 IEEE.
PY - 2025
Y1 - 2025
N2 - Cataract surgery, a widely performed operation worldwide, is incorporating semantic segmentation to advance computer-assisted intervention. However, the tissue appearance and illumination in cataract surgery often differ among clinical centers, intensifying the issue of domain shifts. While domain adaptation offers remedies to the shifts, the necessity for data centralization raises additional privacy concerns. To overcome these challenges, we propose a Multi-view Test-time Adaptation algorithm (MUTA) to segment cataract surgical scenes, which leverages multi-view learning to enhance model training within the source domain and model adaptation within the target domain. In the training phase, the segmentation model is equipped with multi-view decoders to boost its robustness against variations in cataract surgery. During the inference phase, test-time adaptation is implemented using multi-view knowledge distillation, enabling model updates in clinics without data centralization or privacy concerns. We conducted experiments in a simulated cross-center scenario using several cataract surgery datasets to evaluate the effectiveness of MUTA. Through comparisons and investigations, we have validated that MUTA effectively learns a robust source model and adapts the model to target data during the practical inference phase. Code and datasets are available at https://github.com/liamheng/CAI-algorithms.
AB - Cataract surgery, a widely performed operation worldwide, is incorporating semantic segmentation to advance computer-assisted intervention. However, the tissue appearance and illumination in cataract surgery often differ among clinical centers, intensifying the issue of domain shifts. While domain adaptation offers remedies to the shifts, the necessity for data centralization raises additional privacy concerns. To overcome these challenges, we propose a Multi-view Test-time Adaptation algorithm (MUTA) to segment cataract surgical scenes, which leverages multi-view learning to enhance model training within the source domain and model adaptation within the target domain. In the training phase, the segmentation model is equipped with multi-view decoders to boost its robustness against variations in cataract surgery. During the inference phase, test-time adaptation is implemented using multi-view knowledge distillation, enabling model updates in clinics without data centralization or privacy concerns. We conducted experiments in a simulated cross-center scenario using several cataract surgery datasets to evaluate the effectiveness of MUTA. Through comparisons and investigations, we have validated that MUTA effectively learns a robust source model and adapts the model to target data during the practical inference phase. Code and datasets are available at https://github.com/liamheng/CAI-algorithms.
KW - Cataract Surgery
KW - multi-view learning
KW - semantic segmentation
KW - test-time adaptation
UR - http://www.scopus.com/inward/record.url?scp=85215692916&partnerID=8YFLogxK
U2 - 10.1109/TMI.2025.3529875
DO - 10.1109/TMI.2025.3529875
M3 - Article
AN - SCOPUS:85215692916
SN - 0278-0062
JO - IEEE Transactions on Medical Imaging
JF - IEEE Transactions on Medical Imaging
ER -