Real-Time Obstacle Avoidance for Magnetic Microswimmers Based on Proximal Policy Optimization

Magnetic helical microswimmers represent a promising technology for biomedical applications such as minimally invasive surgery and targeted drug delivery. However, autonomous navigation in dynamic and unstructured environments characteristic of biological systems remains a critical challenge. Traditional path planning algorithms struggle with real-time adaptation to unpredictable environmental changes and dynamic obstacles. This paper proposes a novel Deep Reinforcement Learning approach tailored for autonomous obstacle avoidance in magnetic helical microswimmers. The navigation problem is framed as a Markov Decision Process, and a physics-informed virtual environment is developed to precisely simulate the dynamics of microswimmers and their interactions with obstacles. A Deep Reinforcement Learning agent is developed through Proximal Policy Optimization with domain randomization to enhance policy robustness and generalization across diverse conditions. Comprehensive evaluation through simulations and real-world experiments demonstrates superior performance across static, fluctuating, and flowing obstacle scenarios. Our Deep Reinforcement Learning-based controller achieves navigation success rates of 75% in environments with 30 static obstacles and 60% with 30 dynamic obstacles, significantly outperforming traditional navigation methods in both success rate and path efficiency. The results validate the approach's potential for practical biomedical applications requiring autonomous microrobot navigation in complex in vivo environments.