Extraction of Key Foreground Information from Visual Feedback Images for Contact Micromanipulation in Liquid Environment

Contact micromanipulation for cells is an important branch in the field of micromanipulation. Limited by the size of the sensor, it is difficult to integrate the sensor in the macroscopic scene into the micromanipulator. So in most operations, images are often the only reliable source of information. The first prerequisite for realizing automated micromanipulation is to be able to extract key information from images. The occlusion phenomenon will inevitably occur in the contact micromanipulation. Although it is possible to design a specific algorithm to identify the target edge information in the occlusion state according to the characteristics of the operating environment and the end-effector. But there is still no universal image processing method to solve this problem. In this paper, we propose an image processing function based on a composite deep learning network structure to solve this problem. Our algorithm is divided into two steps:In the first step, we input the original image into the target detection network to get the position information of the target and end-effector, and cut the region of interest from the original image according to this information. In the second step, we preprocess these candidate sub-images containing key foreground information, and then input them into the image segmentation network to obtain the contour information of the end-effector and target. We designed a cell aspiration experiment based on the digital holographic microscope imaging system to validate our algorithm. In future work, we will continue to improve the algorithm to have better robustness and generalization.