CT-Net: : Asymmetric compound branch Transformer for medical image segmentation
Authors: 
Ning Zhang, Long Yu, Dezhi Zhang, Weidong Wu, Shengwei Tian, Xiaojing Kang, Min LiAuthors Info & Claims
References
[1]
Bernal J., Sánchez F.J., Fernández-Esparrach G., Gil D., Miguel C.R., Vilariño F., WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs saliency maps from physicians, Computerized Medical Imaging and Graphics: the Official Journal of the Computerized Medical Imaging Society 43 (2015) 99–111,.
[2]
Cao H., Wang Y., Chen J., Jiang D., Zhang X., Tian Q., et al., Swin-unet: Unet-like pure transformer for medical image segmentation, 2021,. arXiv, abs/2105.05537.
[3]
Chen J.N., Lu Y.Y., Yu Q.H., Luo X.L., Adeli E., Wang W., et al., TransUNet: Transformers make strong encoders for medical image segmentation, 2021,. arXiv abs/2102.04306.
[4]
Cheng J., Tian S., Yu L., Gao C., Kang X., Ma X., et al., ResGANet: Residual group attention network for medical image classification and segmentation, Medical Image Analysis 76 (2022),.
[5]
Chu X., Tian Z., Wang Y., Zhang B., Ren H., Wei X., et al., Twins: Revisiting the design of spatial attention in vision transformers, in: NeurIPS, 2021,.
[6]
Chu X., Zhang B., Tian Z., Wei X., Xia H., Do we really need explicit position encodings for vision transformers?, 2021,. arXiv, abs/2102.10882.
[7]
Çiçek Ö, Abdulkadir A., Lienkamp S.S., Brox T., Ronneberger O., 3D U-Net: Learning dense volumetric segmentation from sparse annotation, in: Ourselin S., Joskowicz L., Sabuncu M.R., Unal G., Wells W. (Eds.), Medical image computing and computer-assisted intervention – MICCAI 2016, Springer International Publishing, Cham, 2016, pp. 424–432,.
[8]
d’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., & Sagun, L. (2021). ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases. In ICML. https://doi.org/10.48550/arXiv.2103.10697.
[9]
Dong B., Wang W., Fan D., Li J., Fu H., Shao L., Polyp-PVT: Polyp segmentation with pyramid vision transformers, 2021,. arXiv, abs/2108.06932.
[10]
Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., et al., An image is worth 16x16 words: transformers for image recognition at scale, 2021,. arXiv, abs/2010.11929.
[11]
Dutta A. Dubey, Detection of liver cancer using image processing techniques, in: 2019 Int. conf. commun. signal process, IEEE, 2019, pp. 0315–0318,.
[12]
Fan D.P., Ji G.P., Zhou T., Chen G., Fu H., Shen J., et al., Pranet: Parallel reverse attention network for polyp segmentation, in: International conference on medical image computing and computer-assisted intervention, Springer, Cham, 2020, pp. 263–273,.
[13]
Fang X., Shi Y., Guo Q., Wang L., Liu Z., Sub-band based attention for robust polyp segmentation, in: International joint conference on artificial intelligence, 2023,.
[14]
Feng S., Zhao H., Shi F., Cheng X., Wang M., Ma Y., et al., CPFNet: Context pyramid fusion network for medical image segmentation, IEEE Transactions on Medical Imaging 39 (2020) 3008–3018,.
[15]
Fu J., Liu J., Tian H., et al., Dual attention network for scene segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2019, pp. 3146–3154,.
[16]
Fu S., Lu Y., Wang Y., Zhou Y., Shen W., Fishman E.K., et al., Domain adaptive relational reasoning for 3D multi-organ segmentation, 2020,. arXiv, abs/2005.09120.
[17]
Gu Z., Cheng J., Fu H., Zhou K., Hao H., Zhao Y., et al., CE-Net: Context encoder network for 2D medical image segmentation, IEEE Transactions on Medical Imaging 38 (2019) 2281–2292,.
[18]
Gutman D.A., Codella N.C., Celebi M.E., Helba B., Marchetti M.A., Mishra N.K., et al., Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC), in: 2018 IEEE 15th international symposium on biomedical imaging, 2018, pp. 168–172,.
[19]
Heidari M., Kazerouni A., Soltany M., Azad R., Aghdam E.K., Cohen-Adad J., et al., Hiformer: Hierarchical multi-scale representations using Transformers for medical image segmentation, in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2023, pp. 6202–6212,.
[20]
Hu J., Shen L., Sun G., Squeeze-and-excitation networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141,.
[21]
Huang X., Deng Z., Li D., Yuan X., MISSFormer: An effective medical image segmentation transformer, 2021,. arXiv, abs/2109.07162.
[22]
Huang H., Meng F., Zhou S., Brain image segmentation based on FCM clustering algorithm and rough set, IEEE Access 7 (2019) 12386–12396,.
[23]
Huang C., Wu H., Lin Y.S., HarDNet-MSEG: A simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 FPS, 2021,. arXiv, abs/2101.07172.
[24]
Ibtehaz N., Rahman M.S., MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation, Neural Networks: the Official Journal of the International Neural Network Society 121 (2020) 74–87,.
[25]
Isensee F., Jaeger P.F., Kohl S.A., Petersen J., Maier-Hein K.H., nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation, Nature Methods 18 (2) (2021) 203–211,.
[26]
Jha D., Riegler M., Johansen D., Halvorsen P., Johansen H.D., DoubleU-Net: A deep convolutional neural network for medical image segmentation, in: 2020 IEEE 33rd international symposium on computer-based medical systems, 2020, pp. 558–564,.
[27]
Jha D., Smedsrud P.H., Riegler M.A., Halvorsen P., Lange T.D., Johansen D., et al., Kvasir-SEG: A segmented polyp dataset, in: International conference on multimedia modeling, Springer, Cham, 2020, pp. 451–462,.
[28]
Jha D., Smedsrud P.H., Riegler M., Johansen D., Lange T.D., Halvorsen P., et al., ResUNet++: An advanced architecture for medical image segmentation, in: 2019 IEEE international symposium on multimedia, 2019, pp. 225–2255,.
[29]
Ji Y., Zhang R., Wang H., Li Z., Wu L., Zhang S., et al., Multi-compound transformer for accurate biomedical image segmentation, in: International conference on medical image computing and computer-assisted intervention, Springer, Cham, 2021, pp. 326–336,.
[30]
Khaniabadi S.M., Ibrahim H., Huqqani I.A., Khaniabadi F.M., Sakim H.A.M., Teoh S.S., Comparative review on traditional and deep learning methods for medical image segmentation, in: 2023 IEEE 14th control and system graduate research colloquium, IEEE, 2023, pp. 45–50,.
[31]
Kim T., Lee H., Kim D., UACANet: Uncertainty augmented context attention for polyp segmentation, in: Proceedings of the 29th ACM international conference on multimedia, 2021,.
[32]
Kumar N., Verma R., Sharma S., Bhargava S.K., Vahadane A., Sethi A., A dataset and a technique for generalized nuclear segmentation for computational pathology, IEEE Transactions on Medical Imaging 36 (2017) 1550–1560,.
[33]
Landman, B., Xu, Z., Igelsias, J. E., Styner, M., Langerak, T., & Klein, A. (2015). Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge. In Proceed of MICCAI: Multi-atlas labeling beyond cranial vault-workshop challenge.
[34]
Lei T., Sun R., Wang X., Wang Y., He X., Nandi A., CiT-Net: Convolutional neural networks hand in hand with vision transformers for medical image segmentation, 2023,. arXiv preprint arXiv:2306.03373.
[35]
Li X., Pang S., Zhang R., Zhu J., Fu X., Tian Y., et al., AtTransUNet: An enhanced hybrid transformer architecture for ultrasound and histopathology image segmentation, Computers in Biology and Medicine 152 (2023),.
[36]
Lin A., Chen B., Xu J., Zhang Z., Lu G., DS-TransUNet: Dual swin transformer u-net for medical image segmentation, 2022,. arXiv, abs/2106.06716.
[37]
Lin W., Wu Z., Chen J., Huang J., Jin L., Scale-aware modulation meet transformer, 2023,. arXiv preprint arXiv:2307.08579.
[38]
Liu Z., Lin Y., Cao Y., Hu H., Wei Y., Zhang Z., et al., Swin transformer: Hierarchical vision transformer using shifted windows, in: 2021 IEEE/CVF international conference on computer vision, 2021, pp. 9992–10002,.
[39]
Long J., Shelhamer E., Darrell T., Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE conf. comput. vis. pattern recognition, 2015, pp. 3431–3440,.
[40]
Luo G., Zhou Y., Sun X., Wang Y., Cao L., Wu Y., et al., Towards lightweight transformer via group-wise transformation for vision-and-language tasks, IEEE Transactions on Image Processing 31 (2022) 3386–3398,.
[41]
Milletari F., Navab N., Ahmadi S., V-Net: Fully convolutional neural networks for volumetric medical image segmentation, in: 2016 Fourth international conference on 3D vision, 2016, pp. 565–571,.
[42]
Oktay J. Schlemper, Folgoc L.L., Lee M.C.H., Heinrich M.P., Misawa K., Mori K., et al., Attention U-Net: learning where to look for the pancreas, in: Comput. vis. pattern recognit., 2018,.
[43]
Pan X., Ye T., Xia Z., Song S., Huang G., Slide-transformer: Hierarchical vision transformer with local self-attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 2082–2091,.
[44]
Peng Z., Huang W., Gu S., Xie L., Wang Y., Jiao J., et al., Conformer: Local features coupling global representations for visual recognition, in: 2021 IEEE/CVF international conference on computer vision, 2021, pp. 357–366,.
[45]
Qiu S., Li C., Feng Y., Zuo S., Liang H., Xu A., GFANet: Gated Fusion Attention Network for skin lesion segmentation, Computers in Biology and Medicine 155 (2023),.
[46]
Ronneberger P. Fischer, Brox T., U-net: Convolutional networks for biomedical image segmentation, in: International conference on medical image computing and computer-assisted intervention, Springer, Cham, 2015, pp. 234–241,.
[47]
Silva J., Histace A., Romain O., Dray X., Granado B., Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer, International Journal of Computer Assisted Radiology and Surgery 9 (2013) 283–293,.
[48]
Sirinukunwattana K., Pluim J.P., Chen H., Qi X., Heng P., Guo Y.B., et al., Gland segmentation in colon histology images: The glas challenge contest, Medical Image Analysis 35 (2017) 489–502,.
[49]
Tajbakhsh N., Gurudu S.R., Liang J., Automated polyp detection in colonoscopy videos using shape and context information, IEEE Transactions on Medical Imaging 35 (2016) 630–644,.
[50]
Valanarasu J.M., Oza P., Hacihaliloglu I., Patel V.M., Medical transformer: Gated axial-attention for medical image segmentation, in: MICCAI, 2021,.
[51]
Vaswani A., Shazeer N.M., Parmar N., Uszkoreit J., Jones L., Gomez A.N., et al., Attention is all you need, 2017,. arXiv, abs/1706.03762.
[52]
Vázquez D., Bernal J., Sánchez F.J., Fernández-Esparrach G., López A.M., Romero A., et al., A benchmark for endoluminal scene segmentation of colonoscopy images, Journal of Healthcare Engineering 2017 (2017),.
[53]
Wang H., Cao P., Wang J., Zaiane O.R., UCTransNet: Rethinking the skip connections in U-Net from a channel-wise perspective with transformer, in: Proceedings of the AAAI conference on artificial intelligence. Vol. 36. No. 3, 2022, pp. 2441–2449,.
[54]
Wang W., Xie E., Li X., Fan D., Song K., Liang D., et al., Pyramid vision transformer: A versatile backbone for dense prediction without convolutions, in: 2021 IEEE/CVF international conference on computer vision, 2021, pp. 548–558,.
[55]
Wang H., Xie S., Lin L., Iwamoto Y., Han X., Chen Y., et al., Mixed transformer u-net for medical image segmentation, in: ICASSP, 2022,.
[56]
Woo S., Park J., Lee J.Y., et al., Cbam: Convolutional block attention module, in: Proceedings of the European conference on computer vision, 2018, pp. 3–19,.
[57]
Wu H., Chen S., Chen G., Wang W., Lei B., Wen Z., FAT-Net: Feature adaptive Transformers for automated skin lesion segmentation, Medical Image Analysis 76 (2022),.
[58]
Wu Y., Liu Y., Zhan X., Cheng M., P2T: Pyramid pooling transformer for scene understanding, IEEE Transactions on Pattern Analysis and Machine Intelligence (2021) 1–12,.
[59]
Wu H., Xiao B., Codella N.C., Liu M., Dai X., Yuan L., et al., CvT: Introducing convolutions to vision transformers, in: 2021 IEEE/CVF international conference on computer vision, 2021, pp. 22–31,.
[60]
Xie E., Wang W., Yu Z., Anandkumar A., Álvarez J.M., Luo P., SegFormer: Simple and efficient design for semantic segmentation with transformers, in: NeurIPS, 2021,.
[61]
Yu Y., Li Y., Wang J., Guan H., Li F., Xiao S., et al., C2-CapsViT: Cross-context and cross-scale capsule vision transformers for remote sensing image scene classification, IEEE Geoscience and Remote Sensing Letters 19 (2022) 1–5,.
[62]
Yu Z., Yu L., Zheng W., Wang S., EIU-Net: Enhanced feature extraction and improved skip connections in U-Net for skin lesion segmentation, Computers in Biology and Medicine (2023),.
[63]
Zhang Y., Liu H., Hu Q., TransFuse: Fusing transformers and CNNs for medical image segmentation, in: MICCAI, 2021,.
[64]
Zhang Z., Liu Q., Wang Y., Road extraction by deep residual U-net, IEEE Geoscience and Remote Sensing Letters 15 (2018) 749–753,.
[65]
Zheng S., Lu J., Zhao H., Zhu X., Luo Z., Wang Y., et al., Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers, in: 2021 IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 6877–6886,.
[66]
Zhou Z., Rahman Siddiquee M.M., Tajbakhsh N., Liang J., UNet++: a nested U-Net architecture for medical image segmentation, in: Stoyanov D., Taylor Z., Carneiro G., Syeda-Mahmood T., Martel A., Maier-Hein L., Tavares J.M.R., Bradley A., Papa J.P., Belagiannis V., Nascimento J.C., Lu Z., Conjeti S., Moradi M., Greenspan H., Madabhushi A. (Eds.), Deep learning in medical image analysis and multimodal learning for clinical decision support, Springer International Publishing, Cham, 2018, pp. 3–11,.
Recommendations
FAFuse: A Four-Axis Fusion framework of CNN and Transformer for medical image segmentation
AbstractMedical image segmentation is crucial for accurate diagnosis and treatment in the medical field. In recent years, convolutional neural networks (CNNs) and Transformers have been frequently adopted as network architectures in medical image ...
Highlights- Based on CNN and Transformer, the FAFuse framework can effectively integrate fine-grained local features and high-level semantic context information through FAF.
- Four-Axis attention can effectively capture non-local and multi-...
TSCA-Net: Transformer based spatial-channel attention segmentation network for medical images
AbstractDeep learning architectures based on convolutional neural network (CNN) and Transformer have achieved great success in medical image segmentation. Models based on the encoder–decoder framework like U-Net have been successfully employed in many ...
Highlights- A novel encoder–decoder architecture with spatial and channel attention based on Transformer is designed for medical image segmentation.
- Modules based on Transformer are designed to extract simultaneously valuable spatial and channel ...
HmsU-Net: A hybrid multi-scale U-net based on a CNN and transformer for medical image segmentation
AbstractAccurate medical image segmentation is of great significance for subsequent diagnosis and analysis. The acquisition of multi-scale information plays an important role in segmenting regions of interest of different sizes. With the emergence of ...
Highlights- We proposed a multi-scale network HmsU-Net based on the CNN and Transformer.
- HmsU-Net reduced inconsistencies in the fusion of features at different scales.
- To alleviate the computational cost, we designed a lightweight attention ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Elsevier Ltd.
Publisher
Elsevier Science Ltd.
United Kingdom
Publication History
Published: 12 April 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 16 Jan 2025