Temporal Context Framework for Endoscopy Artefact Segmentation and Detection

Endoscopic video processing could facilitate pre-operative planning, intra-operative image guidance and generation of post-operative analysis of the surgical procedure. However, most of the current methods are still based on a single frame of image analysis, which makes the results of the previous frame images independent of each other and causes vibration. In this paper, we propose an temporal context framework for endoscopy artefact segmentation and detection. The framework extends the general segmentation and detection model to the form based on temporal input, and we add a Temporal Context Transformer(TCT) after the encoder of the model to improve the model’s ability to construct temporal context features. the experiments of the EndoCV 2022 challenge dataset that this framework can improve the robustness of the model.