Qualitative and Quantitative Assessment of Invasive Surgery Skill Based on a Custom Training Simulator

Purpose: Qualitative and quantitative assessment of interventional performance is a vital component in the evaluation of endovascular surgery skill training. We established a custom simulator with qualitative and quantitative metrics for endovascular performance training. Methods: The simulator included an in vitro silicone phantom, mock circulation loop, visual module, force-sensing module, and custom software for image and force data postprocessing. Two tasks to deliver the guidewire to the target location of the carotid artery were conducted by the expert (n=4), novice (n=6), and test (n=4) groups. Seven features with significant differences extracted from the expert and novice groups were applied for qualitative assessment using the support vector machine (SVM) and quantitative assessment using the Mahalanobis distance (MD). Results: Significant differences were observed in kinematic and force data between experts and novices during the intervention procedure. The median value of finished time for task 1 was 26.88 seconds for experts and 63.36 seconds for novices. The maximum speed for experts and novices was 32.79 and 7.43 cm/s, respectively. Moreover, the classified results depicted that the accuracy of qualitative assessment for task 1 and task 2 was 96.67% and 90%, respectively. As for the quantitative data, the residents had higher scores than individuals majored in biomedical engineering at 2 tasks (70.06±5.30 vs 41.81±6.58 for task 1, p=0.001). Conclusions: The proposed endovascular intervention skill training simulator provides qualitative and quantitative metrics on intervention performance skills and may be a useful tool in future interventional surgical training. Clinical Impact: This simulator comprised an in-vitro silicone phantom, mock circulation loop, visual module, force-sensing module, and custom software for image and force data post-processing. Seven interventional performance features were used for qualitative assessment using the support vector machine and quantitative assessment using the Mahalanobis Distance. From the observations, we conclude that this endovascular intervention skill training simulator provides qualitative and quantitative metrics on intervention performance and may be a useful tool in future surgical training.