TY - JOUR
T1 - Correction of image distortion in large-field ssEM stitching by an unsupervised intermediate-space solving network
AU - He, Bintao
AU - Zhang, Yan
AU - Zhang, Fa
AU - Han, Renmin
N1 - Publisher Copyright:
© 2022 The Author(s).
PY - 2022/10/15
Y1 - 2022/10/15
N2 - Motivation: Serial-section electron microscopy (ssEM) is a powerful technique for cellular visualization, especially for large-scale specimens. Limited by the field of view, a megapixel image of whole-specimen is regularly captured by stitching several overlapping images. However, suffering from distortion by manual operations, lens distortion or electron impact, simple rigid transformations are not adequate for perfect mosaic generation. Non-linear deformation usually causes 'ghosting' phenomenon, especially with high magnification. To date, existing microscope image processing tools provide mature rigid stitching methods but have no idea with local distortion correction. Results: In this article, following the development of unsupervised deep learning, we present a multi-scale network to predict the dense deformation fields of image pairs in ssEM and blend these images into a clear and seamless montage. The model is composed of two pyramidal backbones, sharing parameters and interacting with a set of registration modules, in which the pyramidal architecture could effectively capture large deformation according to multi-scale decomposition. A novel 'intermediate-space solving' paradigm is adopted in our model to treat inputted images equally and ensure nearly perfect stitching of the overlapping regions. Combining with the existing rigid transformation method, our model further improves the accuracy of sequential image stitching. Extensive experimental results well demonstrate the superiority of our method over the other traditional methods.
AB - Motivation: Serial-section electron microscopy (ssEM) is a powerful technique for cellular visualization, especially for large-scale specimens. Limited by the field of view, a megapixel image of whole-specimen is regularly captured by stitching several overlapping images. However, suffering from distortion by manual operations, lens distortion or electron impact, simple rigid transformations are not adequate for perfect mosaic generation. Non-linear deformation usually causes 'ghosting' phenomenon, especially with high magnification. To date, existing microscope image processing tools provide mature rigid stitching methods but have no idea with local distortion correction. Results: In this article, following the development of unsupervised deep learning, we present a multi-scale network to predict the dense deformation fields of image pairs in ssEM and blend these images into a clear and seamless montage. The model is composed of two pyramidal backbones, sharing parameters and interacting with a set of registration modules, in which the pyramidal architecture could effectively capture large deformation according to multi-scale decomposition. A novel 'intermediate-space solving' paradigm is adopted in our model to treat inputted images equally and ensure nearly perfect stitching of the overlapping regions. Combining with the existing rigid transformation method, our model further improves the accuracy of sequential image stitching. Extensive experimental results well demonstrate the superiority of our method over the other traditional methods.
UR - http://www.scopus.com/inward/record.url?scp=85140144115&partnerID=8YFLogxK
U2 - 10.1093/bioinformatics/btac566
DO - 10.1093/bioinformatics/btac566
M3 - Article
C2 - 35977377
AN - SCOPUS:85140144115
SN - 1367-4803
VL - 38
SP - 4797
EP - 4805
JO - Bioinformatics
JF - Bioinformatics
IS - 20
ER -