Abstract
Retinal images play an essential role in the early diagnosis of ophthalmic diseases. Automatic segmentation of retinal vessels in color fundus images is challenging due to the morphological differences between the retinal vessels and the low-contrast background. At the same time, automated models struggle to capture representative and discriminative retinal vascular features. To fully utilize the structural information of the retinal blood vessels, we propose a novel deep learning network called Pre-Activated Convolution Residual and Triple Attention Mechanism Network (PCRTAM-Net). PCRTAM-Net uses the pre-activated dropout convolution residual method to improve the feature learning ability of the network. In addition, the residual atrous convolution spatial pyramid is integrated into both ends of the network encoder to extract multiscale information and improve blood vessel information flow. A triple attention mechanism is proposed to extract the structural information between vessel contexts and to learn long-range feature dependencies. We evaluate the proposed PCRTAM-Net on four publicly available datasets, DRIVE, CHASE_DB1, STARE, and HRF. Our model achieves state-of-the-art performance of 97.10%, 97.70%, 97.68%, and 97.14% for ACC and 83.05%, 82.26%, 84.64%, and 81.16% for F1, respectively.
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Li L Z, Verma M, Nakashima Y, Nagahara H, Kawasaki R. IterNet: Retinal image segmentation utilizing structural redundancy in vessel networks. In Proc. the 2020 IEEE Winter Conference on Applications of Computer Vision, Mar. 2020, pp.3645–3654. https://doi.org/10.1109/WACV45572.2020.9093621.
Li Z Y, Zhang X F, Muller H, Zhang S T. Large-scale retrieval for medical image analytics: A comprehensive review. Medical Image Analysis, 2018, 43: 66–84. https://doi.org/10.1016/j.media.2017.09.007.
Huang K, Yan M. A region based algorithm for vessel detection in retinal images. In Proc. the 9th International Conference on Medical Image Computing and Computer-Assisted Intervention, Oct. 2006, pp.645–653. https://doi.org/10.1007/11866565_79.
Hoover A D, Kouznetsova V, Goldbaum M. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Medical Imaging, 2002, 19(3): 203–210. https://doi.org/10.1109/42.845178.
Liskowski P, Krawiec K. Segmenting retinal blood vessels with deep neural networks. IEEE Trans. Medical Imaging, 2016, 35(11): 2369–2380. https://doi.org/10.1109/TMI.2016.2546227.
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Oct. 2015, pp.234–241. https://doi.org/10.1007/978-3-319-24574-4_28.
Staal J, Abramoff M D, Niemeijer M, Viergever M A, Van Ginneken B. Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Medical Imaging, 2004, 23(4): 501–509. https://doi.org/10.1109/TMI.2004.825627.
Orlando J I, Prokofyeva E, Blaschko M B. A discriminatively trained fully connected conditional random field model for blood vessel segmentation in fundus images. IEEE Trans. Biomedical Engineering, 2017, 64(1): 16–27. https://doi.org/10.1109/TBME.2016.2535311.
Sheng B, Li P, Mo S J, Li H T, Hou X H, Wu Q, Qin J, Fang R G, Feng D D. Retinal vessel segmentation using minimum spanning superpixel tree detector. IEEE Trans. Cybernetics, 2018, 49(7): 2707–2719. https://doi.org/10.1109/TCYB.2018.2833963.
Yin B J, Li H T, Sheng B, Hou X H, Chen Y, Wu W, Li P, Shen R M, Bao Y Q, Jia W P. Vessel extraction from non-fluorescein fundus images using orientation-aware detector. Medical Image Analysis, 2015, 26(1): 232–242. https://doi.org/10.1016/j.media.2015.09.002.
Dai L, Wu L, Li H T, Cai C, Wu Q, Kong H Y, Liu R H, Wang X N, Hou X H, Liu Y X, Long X X, Wen Y, Lu L N, Shen Y X, Chen Y, Shen D G, Yang X K, Zou H D, Sheng B, Jia W P. A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nature Communications, 2021, 12: Article No. 3242. https://doi.org/10.1038/s41467-021-23458-5.
Wang D Y, Haytham A, Pottenburgh J, Saeedi O, Tao Y. Hard attention net for automatic retinal vessel segmentation. IEEE Journal of Biomedical and Health Informatics, 2020, 24(12): 3384–3396. https://doi.org/10.1109/JBHI.2020.3002985.
Sun M Y, Li K Q, Qi X Q, Dang H, Zhang G H. Contextual information enhanced convolutional neural networks for retinal vessel segmentation in color fundus images. Journal of Visual Communication and Image Representation, 2021, 77: 103134. https://doi.org/10.1016/j.jvcir.2021.103134.
Jin Q G, Meng Z P, Pham T D, Chen Q, Wei L Y, Su R. DUNet: A deformable network for retinal vessel segmentation. Knowledge-Based Systems, 2019, 178: 149–162. https://doi.org/10.1016/j.knosys.2019.04.025.
Mou L, Chen L, Cheng J, Gu Z W, Zhao Y T, Liu J. Dense dilated network with probability regularized walk for vessel detection. IEEE Trans. Medical Imaging, 2020, 39(5): 1392–1403. https://doi.org/10.1109/TMI.2019.2950051.
Wei J H, Zhu G J, Fan Z, Liu J C, Rong Y B, Mo J J, Li W J, Chen X J. Genetic U-Net: Automatically designed deep networks for retinal vessel segmentation using a genetic algorithm. IEEE Trans. Medical Imaging, 2022, 41(2): 292–307. https://doi.org/10.1109/TMI.2021.3111679.
Fu J, Liu J, Tian H J, Li Y, Bao Y J, Fang Z W, Lu H Q. Dual attention network for scene segmentation. In Proc. the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2019, pp.3141–3149. https://doi.org/10.1109/CVPR.2019.00326.
Yang Q, Ma B Q, Cui H, Ma J Q. AMF-NET: Attention-aware multi-scale fusion network for retinal vessel segmentation. In Proc. the 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Nov. 2021, pp.3277–3280. https://doi.org/10.1109/EMBC46164.2021.9630756.
Wu T F, Li L L, Li J B. MSCAN: Multi-scale channel attention for fundus retinal vessel segmentation. In Proc. the 2nd IEEE International Conference on Power Data Science, Dec. 2020, pp.18–27. https://doi.org/10.1109/ICPDS51559.2020.9332494.
Wu H S, Wang W, Zhong J F, Lei B Y, Wen Z K, Qin J. SCS-Net: A scale and context sensitive network for retinal vessel segmentation. Medical Image Analysis, 2021, 70: 102025. https://doi.org/10.1016/j.media.2021.102025.
Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. In Proc. the 14th International Conference on Artificial Intelligence and Statistics, Apr. 2011, pp.315–323.
Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp.2261–2269. https://doi.org/10.1109/CVPR.2017.243.
He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp.770–778. https://doi.org/10.1109/CVPR.2016.90.
Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proc. the 32nd International Conference on Machine Learning, Jul. 2015, pp.448–456.
Mou L, Zhao Y T, Fu H Z, Liu Y H, Cheng J, Zheng Y L, Su P, Yang J L, Chen L, Frangi A F, Akiba M, Liu J. CS2-Net: Deep learning segmentation of curvilinear structures in medical imaging. Medical Image Analysis, 2021, 67: 101874. https://doi.org/10.1016/j.media.2020.101874.
Shi Z J, Wang T Y, Huang Z, Xie F, Liu Z H, Wang B L, Xu J. MD-Net: A multi-scale dense network for retinal vessel segmentation. Biomedical Signal Processing and Control, 2021, 70: 102977. https://doi.org/10.1016/j.bspc.2021.102977.
Owen C G, Rudnicka A R, Mullen R, Barman S A, Monekosso D, Whincup P H, Ng J, Paterson C. Measuring retinal vessel tortuosity in 10-year-old children: Validation of the computer-assisted image analysis of the retina (CAIAR) program. Investigative Ophthalmology & Visual Science, 2009, 50(5): 2004–2010. https://doi.org/10.1167/iovs.083018.
Köhler T, Budai A, Kraus M F, Odstrčilik J, Michelson G, Hornegger J. Automatic no-reference quality assessment for retinal fundus images using vessel segmentation. In Proc. the 26th IEEE International Symposium on Computer-Based Medical Systems, Jun. 2013, pp.95–100. https://doi.org/10.1109/CBMS.2013.6627771.
Woo S, Park J, Lee J Y, Kweon I S. CBAM: Convolutional block attention module. In Proc. the 15th European Conference on Computer Vision, Sept. 2018, pp.3–19. https://doi.org/10.1007/978-3-030-01234-2_1.
Zhuo Z S, Huang J P, Lu K, Pan D R, Feng S T. A size-invariant convolutional network with dense connectivity applied to retinal vessel segmentation measured by a unique index. Computer Methods and Programs in Biomedicine, 2020, 196: 105508. https://doi.org/10.1016/j.cmpb.2020.105508.
Wu Y C, Xia Y, Song Y, Zhang Y N, Cai W D. NFN+: A novel network followed network for retinal vessel segmentation. Neural Networks, 2020, 126: 153–162. https://doi.org/10.1016/j.neunet.2020.02.018.
Khan T M, Khan M A U, Rehman N U, Naveed K, Afridi I U, Naqvi S S, Raazak I. Width-wise vessel bifurcation for improved retinal vessel segmentation. Biomedical Signal Processing and Control, 2022, 71: 103169. https://doi.org/10.1016/j.bspc.2021.103169.
Li Q L, Feng B W, Xie L P, Liang P, Zhang H S, Wang T F. A cross-modality learning approach for vessel segmentation in retinal images. IEEE Trans. Medical Imaging, 2016, 35(1): 109–118. https://doi.org/10.1109/TMI.2015.2457891.
Wu Y C, Xia Y, Song Y, Zhang Y N, Cai W D. Multiscale network followed network model for retinal vessel segmentation. In Proc. the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, Sept. 2018, pp.119–126.
Tang P, Liang Q K, Yan X T, Zhang D, Coppola G, Sun W. Multi-proportion channel ensemble model for retinal vessel segmentation. Computers in Biology and Medicine, 2019, 111: 103352. https://doi.org/10.1016/j.compbiomed.2019.103352.
Dong F F, Wu D Y, Guo C Y, Zhang S T, Yang B L, Gong X Y. CRAUNet: A cascaded residual attention U-Net for retinal vessel segmentation. Computers in Biology and Medicine, 2022, 147: 105651. https://doi.org/10.1016/j.compbiomed.2022.105651.
Soomro T A, Afifi A J, Gao J B, Hellwich O, Zheng L H, Paul M. Strided fully convolutional neural network for boosting the sensitivity of retinal blood vessels segmentation. Expert Systems with Applications, 2019, 134: 36–52. https://doi.org/10.1016/j.eswa.2019.05.029.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Idowu Paul Okuwobi worked on the investigation and writing and provided computing power, and Bing-Bing Li was responsible for the data curation and methodology.
Supplementary Information
ESM 1
(PDF 2083 kb)
Rights and permissions
About this article
Cite this article
Wang, HD., Li, ZZ., Okuwobi, I.P. et al. PCRTAM-Net: A Novel Pre-Activated Convolution Residual and Triple Attention Mechanism Network for Retinal Vessel Segmentation. J. Comput. Sci. Technol. 38, 567–581 (2023). https://doi.org/10.1007/s11390-023-3066-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11390-023-3066-4