[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

RTNet: a residual t-shaped network for medical image segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Accurate segmentation of lesion areas in medical image analysis can assist clinicians develop more personalized treatment tools, improving treatment efficacy and survival rates. Furthermore, the single encoding structure within the U-shaped architecture limits the network’s capacity to aggregate semantic information of various scales. In this paper, we propose a residual T-shaped network (RTNet) for medical image segmentation. The RTNet incorporates Multi-Level Feature Enhanced Residual Block (MFER Block) and attention gates into a T-shaped structure, making our network capture richer contextual information at multiple scales. First, by utilizing two sets of encoders with different structures, our network can extract image features at different scales. Secondly, the MFER Block is employed to combine and learn the feature representations from the encoders, facilitating more effective feature fusion. Finally, the attention gates fuse those feature maps obtained from each stage via skip connections, then they are concatenated by the decoder to generate the final feature map for segmentation. Experimental results show that, on the BUSI and ISIC2017 datasets, the IoU of our RTNet reached 67.39% and 84.39%, respectively; and the DSC score achieved 79.95% and 91.44%, respectively. Our network outperforms state-of-the-art models for medical image segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and materials

The datasets generated during or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Kumari P, Saxena P (2023) Disease localization and its prediction from retinal fundus images using explicitly designed deep learning architecture. Multimed Tools Appl. https://doi.org/10.1007/s11042-023-16585-2

    Article  Google Scholar 

  2. Chakraborty S, Mali K (2023) An overview of biomedical image analysis from the deep learning perspective. Research Anthology on Improving Medical Imaging Techniques for Analysis and Intervention 43–59. https://doi.org/10.4018/978-1-6684-7544-7.ch003

  3. Wan Y, Shao M, Cheng Y, Ding W (2023) Fuzzy-based cross-image pixel contrastive learning for compact medical image segmentation. Multimed Tools Appl. https://doi.org/10.1007/s11042-023-16611-3

    Article  Google Scholar 

  4. Xiao H, Li L, Liu Q, Zhu X, Zhang Q (2023) Transformers in medical image segmentation: A review. Biomed Signal Proc Control 84:104791. https://doi.org/10.1016/j.bspc.2023.104791

    Article  Google Scholar 

  5. Shamshad F, Khan S, Zamir SW, Khan MH, Hayat M, Khan FS, Fu H (2023) Transformers in medical imaging: A survey. Med Image Anal 102802. https://doi.org/10.1016/j.media.2023.102802

  6. D’Angelo T, Caudo D, Blandino A, Albrecht MH, Vogl TJ, Gruenewald LD, Gaeta M, Yel I, Koch V, Martin SS et al (2022) Artificial intelligence, machine learning and deep learning in musculoskeletal imaging: Current applications. J Clinical Ultrasound 50(9):1414–1431. https://doi.org/10.1002/jcu.23321

    Article  Google Scholar 

  7. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28. Springer

  8. Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J (2018) Unet++: a nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, pp 3–11. https://doi.org/10.1007/978-3-030-00889-5_1. Springer

  9. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B et al (2018) Attention u-net: learning where to look for the pancreas. https://doi.org/10.48550/arXiv.1804.03999

  10. Huang H, Lin L, Tong R, Hu H, Zhang Q, Iwamoto Y, Han X, Chen Y-W, Wu J (2020) Unet 3+: a full-scale connected unet for medical image segmentation. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1055–1059. https://doi.org/10.1109/ICASSP40776.2020.9053405. IEEE

  11. Nasrin S, Alom MZ, Burada R, Taha TM, Asari VK (2019) Medical image denoising with recurrent residual u-net (r2u-net) base auto-encoder. In: 2019 IEEE national aerospace and electronics conference (NAECON), pp 345–350. https://doi.org/10.1109/NAECON46414.2019.9057834. IEEE

  12. Valanarasu JMJ, Patel VM (2022) Unext: Mlp-based rapid medical image segmentation network. In: International conference on medical image computing and computer-assisted intervention, pp 23–33. https://doi.org/10.1007/978-3-031-16443-9_3. Springer

  13. Zhang Z, Liu Q, Wang Y (2018) Road extraction by deep residual u-net. IEEE Geosci Remote Sens Lett 15(5):749–753. https://doi.org/10.1109/LGRS.2018.2802944

    Article  Google Scholar 

  14. Tang F, Wang L, Ning C, Xian M, Ding J (2023) Cmu-net: a strong convmixer-based medical ultrasound image segmentation network. In: 2023 IEEE 20th international symposium on biomedical imaging (ISBI), pp 1–5. https://doi.org/10.1109/ISBI53787.2023.10230609

  15. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems 30. https://doi.org/10.48550/arXiv.1706.03762

  16. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. https://doi.org/10.48550/arXiv.2010.11929

  17. Valanarasu JMJ, Oza P, Hacihaliloglu I, Patel VM (2021) Medical transformer: gated axial-attention for medical image segmentation. In: Medical image computing and computer assisted intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24, pp 36–46. https://doi.org/10.1007/978-3-030-87193-2_4. Springer

  18. Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Lu L, Yuille AL, Zhou Y (2021) Transunet: transformers make strong encoders for medical image segmentation. https://doi.org/10.48550/arXiv.2102.04306

  19. Gao Y, Zhou M, Metaxas DN (2021) Utnet: a hybrid transformer architecture for medical image segmentation. In: Medical image computing and computer assisted intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part III 24, pp 61–71. https://doi.org/10.1007/978-3-030-87199-4_6. Springer

  20. Punn NS, Agarwal S (2022) Modality specific u-net variants for biomedical image segmentation: a survey. Artif Intell Rev 55(7):5845–5889. https://doi.org/10.1007/s10462-022-10152-1

    Article  Google Scholar 

  21. He C, Li K, Zhang Y, Tang L, Zhang Y, Guo Z, Li X (2023) Camouflaged object detection with feature decomposition and edge reconstruction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 22046–22055. https://doi.org/10.1109/CVPR52729.2023.02111

  22. He C, Li K, Xu G, Yan J, Tang L, Zhang Y, Wang Y, Li X (2023) Hqg-net: unpaired medical image enhancement with high-quality guidance. IEEE Trans Neural Netw LearnSyst. https://doi.org/10.1109/TNNLS.2023.3315307

  23. He C, Li K, Zhang Y, Zhang Y, Guo Z, Li X, Danelljan M, Yu F (2023) Strategic preys make acute predators: enhancing camouflaged object detectors by generating camouflaged objects. https://doi.org/10.48550/arXiv.2308.03166

  24. He C, Li K, Zhang Y, Xu G, Tang L, Zhang Y, Guo Z, Li X (2023) Weakly-supervised concealed object segmentation with sam-based pseudo labeling and multi-scale feature grouping. https://doi.org/10.48550/arXiv.2305.11003

  25. Di Biasi L, De Marco F, Auriemma Citarella A, Castrillón-Santana M, Barra P, Tortora G (2023) Refactoring and performance analysis of the main cnn architectures: using false negative rate minimization to solve the clinical images melanoma detection problem. BMC bioinformatics 24(1):386. https://doi.org/10.1186/s12859-023-05516-5

    Article  Google Scholar 

  26. Sharma V, Gupta SK, Shukla KK et al (2023) Deep learning models for tuberculosis detection and infected region visualization in chest x-ray images. Intell Med. https://doi.org/10.1016/j.imed.2023.06.001

    Article  Google Scholar 

  27. Kumar S, Nagar R, Bhatnagar S, Vaddi R, Gupta SK, Rashid M, Bashir AK, Alkhalifah T (2022) Chest x ray and cough sample based deep learning framework for accurate diagnosis of covid-19. Comput Electr Eng 103:108391. https://doi.org/10.1016/j.compeleceng.2022.108391

    Article  Google Scholar 

  28. Kumar V, Pathak V, Badal N, Pandey PS, Mishra R, Gupta SK (2022) Complex entropy based encryption and decryption technique for securing medical images. Multimed Tools Appl 81(26):37441–37459. https://doi.org/10.1007/s11042-022-13546-z

    Article  Google Scholar 

  29. Kumar S, Gupta SK, Kumar V, Kumar M, Chaube MK, Naik NS (2022) Ensemble multimodal deep learning for early diagnosis and accurate classification of covid-19. Comput Electr Eng 103:108396. https://doi.org/10.1016/j.compeleceng.2022.108396

    Article  Google Scholar 

  30. Kumar S, Chaube MK, Alsamhi SH, Gupta SK, Guizani M, Gravina R, Fortino G (2022) A novel multimodal fusion framework for early diagnosis and accurate classification of covid-19 patients using x-ray images and speech signal processing techniques. Comput Methods Programs Biomed 226:107109. https://doi.org/10.1016/j.cmpb.2022.107109

    Article  Google Scholar 

  31. Huang L, Ruan S, Denœux T (2023) Application of belief functions to medical image segmentation: A review. Information fusion 91:737–756. https://doi.org/10.1016/j.inffus.2022.11.008

    Article  Google Scholar 

  32. Yu Y, Wang C, Fu Q, Kou R, Huang F, Yang B, Yang T, Gao M (2023) Techniques and challenges of image segmentation: A review. Electronics 12(5):1199. https://doi.org/10.3390/electronics12051199

    Article  Google Scholar 

  33. Parasar D, Rathod VR (2017) Particle swarm optimisation k-means clustering segmentation of foetus ultrasound image. Int J Signal Imaging Syst Eng 10(1-2):95–103. https://doi.org/10.1504/IJSISE.2017.084569

  34. Asanambigai V, Sasikala J (2018) Adaptive chemical reaction based spatial fuzzy clustering for level set segmentation of medical images. Ain Shams Eng J 9(4):1251–1262. https://doi.org/10.1016/j.asej.2016.08.003

    Article  Google Scholar 

  35. Aljawawdeh A, Imraiziq E, Aljawawdeh A (2017) Enhanced k-mean using evolutionary algorithms for melanoma detection and segmentation in skin images. Int J Adv Comput Sci Appl 8(12). https://doi.org/10.14569/IJACSA.2017.081263

  36. Azad R, Aghdam EK, Rauland A, Jia Y, Avval AH, Bozorgpour A, Karimijafarbigloo S, Cohen JP, Adeli E, Merhof D (2022) Medical image segmentation review: the success of u-net. https://doi.org/10.48550/arXiv.2211.14830

  37. Li J, Chen J, Tang Y, Wang C, Landman BA, Zhou SK (2023) Transforming medical imaging with transformers? a comparative review of key properties, current progresses, and future perspectives. Medical image analysis 102762. https://doi.org/10.1016/j.media.2023.102762

  38. Yuan F, Zhang Z, Fang Z (2023) An effective cnn and transformer complementary network for medical image segmentation. Pattern Recogn 136:109228. https://doi.org/10.1016/j.patcog.2022.109228

    Article  Google Scholar 

  39. Liu Z, Lv Q, Yang Z, Li Y, Lee CH, Shen L (2023) Recent progress in transformer-based medical image analysis. Comput Biol Med 107268. https://doi.org/10.1016/j.compbiomed.2023.107268

  40. Cao H, Wang Y, Chen J, Jiang D, Zhang X, Tian Q, Wang M (2022) Swin-unet: unet-like pure transformer for medical image segmentation. In: European conference on computer vision, pp 205–218. https://doi.org/10.1007/978-3-031-25066-8_9. Springer

  41. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022. https://doi.org/10.1109/ICCV48922.2021.00986

  42. Fu Z, Li J, Hua Z (2022) Deau-net: attention networks based on dual encoder for medical image segmentation. Comput Biol Med 150:106197. https://doi.org/10.1016/j.compbiomed.2022.106197

  43. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258. https://doi.org/10.1109/CVPR.2017.195

  44. Al-Dhabyani W, Gomaa M, Khaled H, Fahmy A (2020) Dataset of breast ultrasound images. Data in brief 28:104863. https://doi.org/10.1016/j.dib.2019.104863

    Article  Google Scholar 

  45. Codella NC, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H et al (2018) Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp 168–172. https://doi.org/10.1109/ISBI.2018.8363547. IEEE

  46. Zhang K, Liu D (2023) Customized segment anything model for medical image segmentation. https://doi.org/10.48550/arXiv.2304.13785

Download references

Funding

This work was supported in part by the Key Scientific Research Project of Higher School of Henan Province under Grant 21A520022.

Author information

Authors and Affiliations

  1. The School of Computer and Information Engineering, Henan Normal University, Xinxiang, 453007, China

    Shangwang Liu, Yinghai Lin, Danyang Liu, Guoqi Liu & Hualei Shen

  2. Engineering Lab of Intelligence Business and Internet of Things, Henan Normal University, Xinxiang, 453007, China

    Shangwang Liu, Yinghai Lin, Danyang Liu, Guoqi Liu & Hualei Shen

Authors
  1. Shangwang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinghai Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Danyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guoqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hualei Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors participate in the writing of each chapter and discuss changes.

Corresponding author

Correspondence to Shangwang Liu.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships.

Ethical Approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Lin, Y., Liu, D. et al. RTNet: a residual t-shaped network for medical image segmentation. Multimed Tools Appl 83, 74939–74954 (2024). https://doi.org/10.1007/s11042-024-18544-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-024-18544-x

Keywords

Search

Navigation