CosMX cell segmentation

Hi all, I wonder if you have any pointers for cell segmentation on CosMX data. Nanostring uses Cellpose. However, the segmentation quality looks not great. Have you tested any other segmentation algorithms? can you run cellsegmation in Nanostring data using Squidpy?

1 Like

I’ve been working with some datasets that don’t have good membrane staining and found ClusterMap (GitHub - wanglab-broad/ClusterMap: ClusterMap for multi-scale clustering analysis of spatial gene expression) and Baysor (GitHub - kharchenkolab/Baysor: Bayesian Segmentation of Spatial Transcriptomics Data ) to be an okay start. It is able to output boundaries from transcription location alone.

I’m curious what your use case is. Do you care more about spatial accuracy or accuracy of assigning transcripts to cells?

1 Like

Thank you Clarence, this is super helpful. I’m working with hepatocyte, which is recognized for having large nuclei. I’m interested in getting the transcripts assigned to the cells, cellpose is missing a significant proportion of cells and splits some large nuclei.

1 Like

this might also be worth a try? Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation | Nature Methods

super interesting @ckmah , did you observe improvements when using those algos with segmentation masks as priors?

1 Like

Ah yes, excited to try omnipose!

the masks as priors do help, though if @afvallejo’s large nuclei are splitting you probably don’t want those priors.

1 Like