[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

GCLR: : A self-supervised representation learning pretext task for glomerular filtration barrier segmentation in TEM images

Published: 01 December 2023 Publication History

Abstract

Automatic segmentation of the three substructures of glomerular filtration barrier (GFB) in transmission electron microscopy (TEM) images holds immense potential for aiding pathologists in renal disease diagnosis. However, the labor-intensive nature of manual annotations limits the training data for a fully-supervised deep learning model. Addressing this, our study harnesses self-supervised representation learning (SSRL) to utilize vast unlabeled data and mitigate annotation scarcity. Our innovation, GCLR, is a hybrid pixel-level pretext task tailored for GFB segmentation, integrating two subtasks: global clustering (GC) and local restoration (LR). GC captures the overall GFB by learning global context representations, while LR refines three substructures by learning local detail representations. Experiments on 18,928 unlabeled glomerular TEM images for self-supervised pre-training and 311 labeled ones for fine-tuning demonstrate that our proposed GCLR obtains the state-of-the-art segmentation results for all three substructures of GFB with the Dice similarity coefficient of 86.56 ± 0.16%, 75.56 ± 0.36%, and 79.41 ± 0.16%, respectively, compared with other representative self-supervised pretext tasks. Our proposed GCLR also outperforms the fully-supervised pre-training methods based on the three large-scale public datasets – MitoEM, COCO, and ImageNet – with less training data and time.

Highlights

The first study of self-supervised representation learning for GFB ultrastructure segmentation.
A hybrid pretext task GCLR integrating two pixel-level subtasks: global clustering and local restoration.
No image-level subtask, avoiding the task alignment problem.
State-of-the-art performance and simple implementation.

References

[1]
Li A.S., Ingham J.F., Lennon R., Genetic disorders of the glomerular filtration barrier, Clin J Am Soc Nephrol 15 (12) (2020) 1818–1828,.
[2]
Weil E.J., Lemley K.V., Mason C.C., Yee B., Jones L.I., Blouch K., et al., Podocyte detachment and reduced glomerular capillary endothelial fenestration promote kidney disease in type 2 diabetic nephropathy, Kidney Int 82 (9) (2012) 1010–1017,.
[3]
Savige J., Rana K., Tonna S., Buzza M., Dagher H., Wang Y.Y., Thin basement membrane nephropathy, Kidney Int 64 (4) (2003) 1169–1178,.
[4]
Vivarelli M., Massella L., Ruggiero B., Emma F., Minimal change disease, Clin J Am Soc Nephrol 12 (2) (2017) 332–345,.
[5]
Roberts I.S.D., Mason P., Fogo A.B., Oxford textbook of clinical nephrology. Vol. 1, 4th ed., Oxford University Press, 2015, pp. 142–160,.
[6]
Asgari Taghanaki S., Abhishek K., Cohen J.P., Cohen-Adad J., Hamarneh G., Deep semantic segmentation of natural and medical images: A review, Artif Intell Rev 54 (1) (2021) 137–178,.
[7]
Wen J., Lin G., Zhang Y., Zhou Z., Cao L., Feng Q., Semantic segmentation of ultrastructural pathological images of glomerular filtration membrane using deep learning network, Chin J Med Phys 37 (2) (2019) 195–204,.
[8]
Qu C., Roth R., Puapatanakul P., Loitman C., Hammad D., Genin G.M., et al., Three-dimensional visualization of the podocyte actin network using integrated membrane extraction, electron microscopy, and machine learning, J Am Soc Nephrol 33 (1) (2022) 155–173,.
[9]
Jing L., Tian Y., Self-supervised visual feature learning with deep neural networks: A survey, IEEE Trans Pattern Anal Mach Intell 43 (11) (2020) 4037–4058,.
[10]
Zhuang X., Li Y., Hu Y., Ma K., Yang Y., Zheng Y., Self-supervised feature learning for 3D medical images by playing a rubik’s cube, in: International conference on medical image computing and computer-assisted intervention, Springer, 2019, pp. 420–428,.
[11]
Koohbanani N.A., Unnikrishnan B., Khurram S.A., Krishnaswamy P., Rajpoot N., Self-path: Self-supervision for classification of pathology images with limited annotations, IEEE Trans Med Imaging 40 (10) (2021) 2845–2856,.
[12]
Li X., Jia M., Islam M.T., Yu L., Xing L., Self-supervised feature learning via exploiting multi-modal data for retinal disease diagnosis, IEEE Trans Med Imaging 39 (12) (2020) 4023–4033,.
[13]
Haghighi F., Taher M.R.H., Zhou Z., Gotway M.B., Liang J., Transferable visual words: Exploiting the semantics of anatomical patterns for self-supervised learning, IEEE Trans Med Imaging 40 (10) (2021) 2857–2868,.
[14]
Li J., Lin T., Xu Y., SSLP: Spatial guided self-supervised learning on pathological images, in: International conference on medical image computing and computer-assisted intervention, Springer, 2021, pp. 3–12,.
[15]
Zhou Z., Sodha V., Pang J., Gotway M.B., Liang J., Models genesis, Med Image Anal 67 (2021),.
[16]
Zhang R., Liu S., Yu Y., Li G., Self-supervised correction learning for semi-supervised biomedical image segmentation, in: International conference on medical image computing and computer-assisted intervention, Springer, 2021, pp. 134–144,.
[17]
Kayal S., Chen S., Bruijne M.d., Region-of-interest guided supervoxel inpainting for self-supervision, in: International conference on medical image computing and computer-assisted intervention, Springer, 2020, pp. 500–509,.
[18]
Chen L., Bentley P., Mori K., Misawa K., Fujiwara M., Rueckert D., Self-supervised learning for medical image analysis using image context restoration, Med Image Anal 58 (2019),.
[19]
Tang Y., Yang D., Li W., Roth H.R., Landman B., Xu D., et al., Self-supervised pre-training of swin transformers for 3d medical image analysis, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 20730–20740,.
[20]
Xie Z., Lin Y., Zhang Z., Cao Y., Lin S., Hu H., Propagate yourself: Exploring pixel-level consistency for unsupervised visual representation learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16684–16693,.
[21]
Haraldsson B., Nyström J., Deen W.M., Properties of the glomerular barrier and mechanisms of proteinuria, Physiol Rev (2008),.
[22]
Daehn I.S., Duffield J.S., The glomerular filtration barrier: A structural target for novel kidney therapies, Nat Rev Drug Discov 20 (10) (2021) 770–788,.
[23]
Naylor R.W., Morais M.R., Lennon R., Complexities of the glomerular basement membrane, Nat Rev Nephrol 17 (2) (2021) 112–127,.
[24]
Kriz W., Elger M., Mundel P., Lemley K.V., Structure-stabilizing forces in the glomerular tuft, J Am Soc Nephrol 5 (10) (1995) 1731–1739,.
[25]
Grahammer F., Schell C., Huber T.B., The podocyte slit diaphragm—from a thin grey line to a complex signalling hub, Nat Rev Nephrol 9 (10) (2013) 587–598,.
[26]
Cattran D.C., Reich H.N., Oxford textbook of clinical nephrology. Vol. 1, 4th ed., Oxford University Press, 2015, pp. 539–543,.
[27]
Ong S., Giam S., Sinniah R., et al., Adaptive window-based tracking for the detection of membrane structures in kidney electron micrographs, Mach Vis Appl 6 (4) (1993) 215–223,.
[28]
Rangayyan R.M., Kamenetsky I., Benediktsson H., Segmentation and analysis of the glomerular basement membrane in renal biopsy samples using active contours: A pilot study, J Digit Imaging 23 (3) (2010) 323–331,.
[29]
Kamenetsky I., Rangayyan R.M., Benediktsson H., Segmentation and analysis of the glomerular basement membrane using the split and merge method, in: 2008 30th annual international conference of the IEEE engineering in medicine and biology society, IEEE, 2008, pp. 3064–3067,.
[30]
Wu H.-S., Dikman S., Gil J., A semi-automatic algorithm for measurement of basement membrane thickness in kidneys in electron microscopy images, Comput Methods Programs Biomed 97 (3) (2010) 223–231,.
[31]
Wu H.-S., Dikman S., Segmentation and thickness measurement of glomerular basement membranes from electron microscopy images, J Electron Microsc 59 (5) (2010) 409–418,.
[32]
Cao L., Lu Y., Li C., Yang W., Automatic segmentation of pathological glomerular basement membrane in transmission electron microscopy images with random forest stacks, Comput Math Methods Med 2019 (2019),.
[33]
Noroozi M., Favaro P., Unsupervised learning of visual representations by solving Jigsaw puzzles, in: European conference on computer vision, Springer, 2016, pp. 69–84,.
[34]
Chen X., Fan H., Girshick R., He K., Improved baselines with momentum contrastive learning, 2020,. arXiv preprint arXiv:2003.04297.
[35]
Caron M., Bojanowski P., Joulin A., Douze M., Deep clustering for unsupervised learning of visual features, in: Proceedings of the european conference on computer vision, 2018, pp. 132–149,.
[36]
Asano Y.M., Rupprecht C., Vedaldi A., Self-labelling via simultaneous clustering and representation learning, 2019,. arXiv preprint arXiv:1911.05371.
[37]
Caron M., Misra I., Mairal J., Goyal P., Bojanowski P., Joulin A., Unsupervised learning of visual features by contrasting cluster assignments, Adv Neural Inf Process Syst 33 (2020) 9912–9924,.
[38]
Cao B., Zhang H., Wang N., Gao X., Shen D., Auto-GAN: Self-supervised collaborative learning for medical image synthesis, in: Proceedings of the AAAI conference on artificial intelligence. Vol. 34, no. 07, 2020, pp. 10486–10493,.
[39]
Liu X., Sinha A., Ishii M., Hager G.D., Reiter A., Taylor R.H., et al., Dense depth estimation in monocular endoscopy with self-supervised learning methods, IEEE Trans Med Imaging 39 (5) (2019) 1438–1447,.
[40]
Shao S., Pei Z., Chen W., Zhu W., Wu X., Sun D., et al., Self-supervised monocular depth and ego-motion estimation in endoscopy: Appearance flow to the rescue, Med Image Anal 77 (2022),.
[41]
Ross T., Zimmerer D., Vemuri A., Isensee F., Wiesenfarth M., Bodenstedt S., et al., Exploiting the potential of unlabeled endoscopic video data with self-supervised learning, Int J Comput Assist Radiol Surg 13 (6) (2018) 925–933,.
[42]
Yang P., Yin X., Lu H., Hu Z., Zhang X., Jiang R., et al., CS-CO: A hybrid self-supervised visual representation learning method for H&E-stained histopathological images, Med Image Anal 81 (2022),.
[43]
Chaitanya K., Erdil E., Karani N., Konukoglu E., Contrastive learning of global and local features for medical image segmentation with limited annotations, Adv Neural Inf Process Syst 33 (2020) 12546–12558,.
[44]
Zheng R., Zhong Y., Yan S., Sun H., Shen H., Huang K., MsVRL: Self-supervised multiscale visual representation learning via cross-level consistency for medical image segmentation, IEEE Trans Med Imaging 42 (1) (2022) 91–102,.
[45]
Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA. Context encoders: Feature learning by inpainting. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, p. 2536–44.
[46]
Oord A.v.d., Li Y., Vinyals O., Representation learning with contrastive predictive coding, 2018,. arXiv preprint arXiv:1807.03748.
[47]
Gidaris S., Singh P., Komodakis N., Unsupervised representation learning by predicting image rotations, 2018,. arXiv preprint arXiv:1803.07728.
[48]
Xu R., Wunsch D., Survey of clustering algorithms, IEEE Trans Neural Netw 16 (3) (2005) 645–678,.
[49]
Wei D., Lin Z., Franco-Barranco D., Wendt N., Liu X., Yin W., et al., MitoEM dataset: Large-scale 3D mitochondria instance segmentation from EM images, in: International conference on medical image computing and computer-assisted intervention, Springer, 2020, pp. 66–76,.
[50]
Lin T.-Y., Maire M., Belongie S., Hays J., Perona P., Ramanan D., et al., Microsoft coco: Common objects in context, in: European conference on computer vision, Springer, 2014, pp. 740–755,.
[51]
Russakovsky O., Deng J., Su H., Krause J., Satheesh S., Ma S., et al., Imagenet large scale visual recognition challenge, Int J Comput Vis 115 (3) (2015) 211–252,.
[52]
Smith L.N., Topin N., Super-convergence: Very fast training of neural networks using large learning rates, in: Artificial intelligence and machine learning for multi-domain operations applications. Vol. 11006, SPIE, 2019, pp. 369–386,.
[53]
Bengio Y., Lamblin P., Popovici D., Larochelle H., Greedy layer-wise training of deep networks, in: Advances in neural information processing systems. Vol. 19, 2006.
[54]
Liu D., Yu J., Otsu method and K-means, in: 2009 ninth international conference on hybrid intelligent systems. Vol. 1, IEEE, 2009, pp. 344–349,.
[55]
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision. 2017, p. 618–26.
[56]
Guo X., Liu X., Zhu E., Zhu X., Li M., Xu X., et al., Adaptive self-paced deep clustering with data augmentation, IEEE Trans Knowl Data Eng 32 (9) (2020) 1680–1693,.
[57]
Wang X., Ma H., You S., Deep clustering for weakly-supervised semantic segmentation in autonomous driving scenes, Neurocomputing 381 (2020) 20–28,.
[58]
Von Luxburg U., A tutorial on spectral clustering, Statist Comput 17 (4) (2007) 395–416,.
[59]
Murtagh F., Contreras P., Algorithms for hierarchical clustering: An overview, Wiley Interdisc Rev: Data Min Knowl Discov 2 (1) (2012) 86–97,.
[60]
Kim W., Kanezaki A., Tanaka M., Unsupervised learning of image segmentation based on differentiable feature clustering, IEEE Trans Image Process 29 (2020) 8055–8068,.
[61]
Ayache J., Beaunier L., Boumendil J., Ehret G., Laub D., Sample preparation handbook for transmission electron microscopy: techniques. vol. 2, Springer Science & Business Media, 2010, pp. 142–160,.
[62]
Kirillov A., Mintun E., Ravi N., Mao H., Rolland C., Gustafson L., Xiao T., et al., Segment anything, 2023,. arXiv preprint arXiv:2304.02643.
[63]
Yang J., Hu X., Pan H., Chen P., Xia S., Multi-scale attention network for segmentation of electron dense deposits in glomerular microscopic images, Microsc Res Tech 85 (9) (2022) 3256–3264,.

Index Terms

  1. GCLR: A self-supervised representation learning pretext task for glomerular filtration barrier segmentation in TEM images
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image Artificial Intelligence in Medicine
        Artificial Intelligence in Medicine  Volume 146, Issue C
        Dec 2023
        294 pages

        Publisher

        Elsevier Science Publishers Ltd.

        United Kingdom

        Publication History

        Published: 01 December 2023

        Author Tags

        1. Glomerular filtration barrier
        2. Transmission electron microscopy
        3. Image segmentation
        4. Self-supervised representation learning
        5. Pretext task

        Qualifiers

        • Research-article

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 0
          Total Downloads
        • Downloads (Last 12 months)0
        • Downloads (Last 6 weeks)0
        Reflects downloads up to 23 Feb 2025

        Other Metrics

        Citations

        View Options

        View options

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media