GeneSegNet: a deep learning framework for cell segmentation by integrating gene expression and imaging

Yuxing Wang; Wenguan Wang; Dongfang Liu; Wenpin Hou; Tianfei Zhou; Zhicheng Ji

doi:10.1186/s13059-023-03054-0

GeneSegNet: a deep learning framework for cell segmentation by integrating gene expression and imaging

Yuxing Wang, Wenguan Wang, Dongfang Liu, Wenpin Hou, Tianfei Zhou^*, Zhicheng Ji^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

21 Citations (Scopus)

Abstract

When analyzing data from in situ RNA detection technologies, cell segmentation is an essential step in identifying cell boundaries, assigning RNA reads to cells, and studying the gene expression and morphological features of cells. We developed a deep-learning-based method, GeneSegNet, that integrates both gene expression and imaging information to perform cell segmentation. GeneSegNet also employs a recursive training strategy to deal with noisy training labels. We show that GeneSegNet significantly improves cell segmentation performances over existing methods that either ignore gene expression information or underutilize imaging information.

Original language	English
Article number	235
Journal	Genome Biology
Volume	24
Issue number	1
DOIs	https://doi.org/10.1186/s13059-023-03054-0
Publication status	Published - Dec 2023
Externally published	Yes

Keywords

Cell segmentation
Deep learning
In situ hybridization
Spatial transcriptomics

Access to Document

10.1186/s13059-023-03054-0

Cite this

Wang, Y., Wang, W., Liu, D., Hou, W., Zhou, T., & Ji, Z. (2023). GeneSegNet: a deep learning framework for cell segmentation by integrating gene expression and imaging. Genome Biology, 24(1), Article 235. https://doi.org/10.1186/s13059-023-03054-0

@article{333a059108c54eb78c4779fe1ecf9c3c,

title = "GeneSegNet: a deep learning framework for cell segmentation by integrating gene expression and imaging",

abstract = "When analyzing data from in situ RNA detection technologies, cell segmentation is an essential step in identifying cell boundaries, assigning RNA reads to cells, and studying the gene expression and morphological features of cells. We developed a deep-learning-based method, GeneSegNet, that integrates both gene expression and imaging information to perform cell segmentation. GeneSegNet also employs a recursive training strategy to deal with noisy training labels. We show that GeneSegNet significantly improves cell segmentation performances over existing methods that either ignore gene expression information or underutilize imaging information.",

keywords = "Cell segmentation, Deep learning, In situ hybridization, Spatial transcriptomics",

author = "Yuxing Wang and Wenguan Wang and Dongfang Liu and Wenpin Hou and Tianfei Zhou and Zhicheng Ji",

note = "Publisher Copyright: {\textcopyright} 2023, BioMed Central Ltd., part of Springer Nature.",

year = "2023",

month = dec,

doi = "10.1186/s13059-023-03054-0",

language = "English",

volume = "24",

journal = "Genome Biology",

issn = "1474-7596",

publisher = "BioMed Central Ltd.",

number = "1",

}

TY - JOUR

T1 - GeneSegNet

T2 - a deep learning framework for cell segmentation by integrating gene expression and imaging

AU - Wang, Yuxing

AU - Wang, Wenguan

AU - Liu, Dongfang

AU - Hou, Wenpin

AU - Zhou, Tianfei

AU - Ji, Zhicheng

PY - 2023/12

Y1 - 2023/12

N2 - When analyzing data from in situ RNA detection technologies, cell segmentation is an essential step in identifying cell boundaries, assigning RNA reads to cells, and studying the gene expression and morphological features of cells. We developed a deep-learning-based method, GeneSegNet, that integrates both gene expression and imaging information to perform cell segmentation. GeneSegNet also employs a recursive training strategy to deal with noisy training labels. We show that GeneSegNet significantly improves cell segmentation performances over existing methods that either ignore gene expression information or underutilize imaging information.

AB - When analyzing data from in situ RNA detection technologies, cell segmentation is an essential step in identifying cell boundaries, assigning RNA reads to cells, and studying the gene expression and morphological features of cells. We developed a deep-learning-based method, GeneSegNet, that integrates both gene expression and imaging information to perform cell segmentation. GeneSegNet also employs a recursive training strategy to deal with noisy training labels. We show that GeneSegNet significantly improves cell segmentation performances over existing methods that either ignore gene expression information or underutilize imaging information.

KW - Cell segmentation

KW - Deep learning

KW - In situ hybridization

KW - Spatial transcriptomics

UR - http://www.scopus.com/inward/record.url?scp=85174547881&partnerID=8YFLogxK

U2 - 10.1186/s13059-023-03054-0

DO - 10.1186/s13059-023-03054-0

M3 - Article

C2 - 37858204

AN - SCOPUS:85174547881

SN - 1474-7596

VL - 24

JO - Genome Biology

JF - Genome Biology

IS - 1

M1 - 235

ER -

GeneSegNet: a deep learning framework for cell segmentation by integrating gene expression and imaging

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this