Automated contact assembly of vessel-mimetic microstructure through position and orientation estimation based on object detection

This paper presents an automated contact assembly method to fabricate vessel-mimetic microstructure through accurate position and orientation estimation of manipulators and micromodules. As an essential component for delivering sufficient nutrients to the regenerated composite tissues, a vessel-mimetic microtube can be precisely assembled through a microrobotic system with dual manipulators under an optical microscope. However, due to the complex situation during contact assembly, it is difficult to access the accurate position and orientation of micromanipulators and assembly units for full automation. To address this problem, we proposed a visual feedback method based on object detection to estimate the position and orientation. Firstly, we employ YOLO (You Only Look Once), a state-of-art object detection algorithm based on deep learning, to locate multi targets and optimize the algorithm for occlusion situations under optical microscopy. Then we combine traditional image-processing algorithms to access the accurate position and orientation of the micromanipulators and micromodules. The experimental results show that assembly units with different shapes can be precisely located with no more than 3μm (6 pixel at 4x magnification) error and the tip position error of micromanipulators remains within 7 μm. The error is acceptable and the algorithm is effective in real-time visual feedback for the automated assembly. As a result, the cell-embedded microtube is automatically assembled at six layers/min, which is efficient enough for automated contact assembly of vessel-mimetic microstructure potentially applied to vascular tissue engineering.