Incorporating Adaptive Sparse Graph Convolutional Neural Networks for Segmentation of Organs at Risk in Radiotherapy
Authors: 
Junjie Hu, Chengrong Yu, Shengqian Zhu, Haixian Zhang Academic Editor: Alexander HošovskýAuthors Info & Claims
Abstract
Precisely segmenting the organs at risk (OARs) in computed tomography (CT) plays an important role in radiotherapy’s treatment planning, aiding in the protection of critical tissues during irradiation. Renowned deep convolutional neural networks (DCNNs) and prevailing transformer-based architectures are widely utilized to accomplish the segmentation task, showcasing advantages in capturing local and contextual characteristics. Graph convolutional networks (GCNs) are another specialized model designed for processing the nongrid dataset, e.g., citation relationship. The DCNNs and GCNs are considered as two distinct models applicable to the grid and nongrid datasets, respectively. Motivated by the recently developed dynamic-channel GCN (DCGCN) that attempts to leverage the graph structure to enhance the feature extracted by the DCNNs, this paper proposes a novel architecture termed adaptive sparse GCN (ASGCN) to mitigate the inherent limitations in DCGCN from the aspect of node’s representation and adjacency matrix’s construction. For the node’s representation, the global average pooling used in the DCGCN is replaced by the learning mechanism to accommodate the segmentation task. For the adjacency matrix, an adaptive regularization strategy is leveraged to penalize the coefficient in the adjacency matrix, resulting in a sparse one that can better exploit the relationships between nodes. Rigorous experiments on multiple OARs’ segmentation tasks of the head and neck demonstrate that the proposed ASGCN can effectively improve the segmentation accuracy. Comparison between the proposed method and other prevalent architectures further confirms the superiority of the ASGCN.
References
[1]
H. Tang, X. Chen, Y. Liu, Z. Lu, J. You, M. Yang, S. Yao, G. Zhao, Y. Xu, T. Chen, Y. Liu, and X. Xie, “Clinically applicable deep learning framework for organs at risk delineation in ct images,” Nature Machine Intelligence, vol. 1, no. 10, pp. 480–491, 2019.
[2]
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105, 2012.
[3]
Z. Yi, “nmode: neural memory ordinary differential equation,” Artificial Intelligence Review, vol. 56, no. 12, pp. 14403–14438, 2023.
[4]
O. Ronneberger, P. Fischer, and T. Brox, “U-net: convolutional networks for biomedical image segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241, Springer, Berlin, Germany, 2015.
[5]
F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,” Nature Methods, vol. 18, no. 2, pp. 203–211, 2021.
[6]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” 2017, https://arxiv.org/abs/1706.03762.
[7]
A. Hatamizadeh, Y. Tang, V. Nath, D. Yang, A. Myronenko, B. Landman, H. R. Roth, and D. Xu, “Unetr: transformers for 3d medical image segmentation,” 2022, https://arxiv.org/abs/2103.10504.
[8]
P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Gallagher, and T. Eliassi-Rad, “Collective classification in network data,” AI Magazine, vol. 29, no. 3, pp. 93–106, 2008.
[9]
A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. R. Hruschka, and T. M. Mitchell, “Toward an architecture for never-ending language learning,” in Proceedings of the Twenty-Fourth AAAI conference on artificial intelligence, Atlanta, Georgia, July 2010.
[10]
M. Zitnik and J. Leskovec, “Predicting multicellular function through multi-layer tissue networks,” Bioinformatics, vol. 33, no. 14, pp. i190–i198, 2017.
[11]
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” 2017, https://arxiv.org/abs/1609.02907.
[12]
Y. Li, Y. Zhang, W. Cui, B. Lei, X. Kuang, and T. Zhang, “Dual encoder-based dynamic-channel graph convolutional network with edge enhancement for retinal vessel segmentation,” IEEE Transactions on Medical Imaging, vol. 41, no. 8, pp. 1975–1989, 2022.
[13]
M. Lin, Q. Chen, and S. Yan, “Network in network,” 2013, https://arxiv.org/abs/1312.4400.
[14]
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014, https://arxiv.org/abs/1409.1556.
[15]
S. Ioffe and C. Szegedy, “Batch normalization: accelerating deep network training by reducing internal covariate shift,” in Proceedings of the International conference on machine learning, PMLR, pp. 448–456, Lille, France, July 2015.
[16]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, Las Vegas, NV, USA, June 2016.
[17]
M. Tan and Q. Le, “Efficientnet: rethinking model scaling for convolutional neural networks,” in Proceedings of the International conference on machine learning, PMLR, pp. 6105–6114, Lille, France, December 2019.
[18]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: a large-scale hierarchical image database,” in Proceedings of the 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255, Miami, FL, USA, June 2009.
[19]
Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3d u-net: learning dense volumetric segmentation from sparse annotation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 424–432, Springer, Berlin, Germany, 2016.
[20]
F. Milletari, N. Navab, and S.-A. Ahmadi, “V-net: fully convolutional neural networks for volumetric medical image segmentation,” in Proceedings of the 2016 fourth international conference on 3D vision (3DV), pp. 565–571, IEEE, Stanford, CA, USA, October 2016.
[21]
S. F. Qadri, H. Lin, L. Shen, M. Ahmad, S. Qadri, S. Khan, M. Khan, S. S. Zareen, M. A. Akbar, M. B. Bin Heyat, and S. Qamar, “Ct-based automatic spine segmentation using patch-based deep learning,” International Journal of Intelligent Systems, vol. 2023, 14 pages, 2023.
[22]
D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” 2015, https://arxiv.org/abs/1409.0473.
[23]
O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y. Hammerla, and B. Kainz, “Attention u-net: learning where to look for the pancreas,” 2018, https://arxiv.org/abs/1804.03999.
[24]
R. K. Srivastava, K. Greff, and J. Schmidhuber, “Highway networks,” 2015, https://arxiv.org/abs/1505.00387.
[25]
S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: convolutional block attention module,” in Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19, Springer, Berlin, Germany, 2018.
[26]
J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu, “Dual attention network for scene segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3146–3154, Long Beach, CA, USA, June 2019.
[27]
L. Mou, H. Fu, Y. Zhao, Y. Liu, J. Cheng, Y. Zheng, P. Su, J. Yang, L. Chen, A. F. Frangi et al., “Cs2-net: curvilinear structure segmentation network for medical images,” Medical Image Analysis, vol. 15, 2020.
[28]
S. Chen, X. Tan, B. Wang, and X. Hu, “Reverse attention for salient object detection,” 2018, https://arxiv.org/abs/1807.09940.
[29]
D.-P. Fan, G.-P. Ji, T. Zhou, G. Chen, H. Fu, J. Shen, and L. Shao, “Pranet: parallel reverse attention network for polyp segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 263–273, Springer, Berlin, Germany, 2020.
[30]
Y. Wang, L. Zhong, W. Huang, X. He et al., “An edge-assisted computing and mask attention based network for lung region segmentation,” International Journal of Intelligent Systems, vol. 2023, 13 pages, 2023.
[31]
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, and S. Gelly, “An image is worth 16x16 words: transformers for image recognition at scale,” 2020, https://arxiv.org/abs/2010.11929.
[32]
A. Hatamizadeh, V. Nath, Y. Tang, D. Yang, H. Roth, and D. Xu, “Swin unetr: swin transformers for semantic segmentation of brain tumors in mri images,” 2022, https://arxiv.org/abs/2201.01266.
[33]
Y. Zhang, X. Ji, W. Liu, Z. Li, J. Zhang, S. Liu, W. Zhong, L. Hu, and W. Li, “A spine segmentation method under an arbitrary field of view based on 3d swin transformer,” International Journal of Intelligent Systems, vol. 2023, 16 pages, 2023, 8686471.
[34]
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022, Montreal, Canada, September 2021.
[35]
D. Hong, B. Zhang, H. Li, Y. Li, J. Yao, C. Li, M. Werner, J. Chanussot, A. Zipf, and X. X. Zhu, “Cross-city matters: a multimodal remote sensing benchmark dataset for cross-city semantic segmentation using high-resolution domain adaptation networks,” Remote Sensing of Environment, vol. 299, 2023.
[36]
J. Yao, B. Zhang, C. Li, D. Hong, and J. Chanussot, “Extended vision transformer (exvit) for land use and land cover classification: a multimodal deep learning framework,” IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–15, 2023.
[37]
J. Yao, D. Hong, J. Chanussot, D. Meng, X. Zhu, and Z. Xu, “Cross-attention in coupled unmixing nets for unsupervised hyperspectral super-resolution,” in Computer Vision–ECCV 2020: 16th European Conference, pp. 208–224, Springer, Glasgow, UK, 2020.
[38]
J. Yao, X. Cao, D. Hong, X. Wu, D. Meng, J. Chanussot, and Z. Xu, “Semi-active convolutional neural networks for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–15, 2022.
[39]
D. Hong, B. Zhang, X. Li, Y. Li, C. Li, J. Yao, N. Yokoya, H. Li, P. Ghamisi, and X. Jia, “Spectralgpt: spectral remote sensing foundation model,” 2023, https://arxiv.org/abs/2311.07113.
[40]
J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, “Spectral networks and deep locally connected networks on graphs,” 2014, https://arxiv.org/abs/1312.6203.
[41]
M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” 2017, https://arxiv.org/abs/1606.09375.
[42]
W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” 2018, https://arxiv.org/abs/1706.02216.
[43]
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” 2018, https://arxiv.org/abs/1710.10903.
[44]
S. Zheng, Z. Zhu, Z. Liu, Z. Guo, Y. Liu, Y. Yang, and Y. Zhao, “Multi-modal graph learning for disease prediction,” IEEE Transactions on Medical Imaging, vol. 41, no. 9, pp. 2207–2216, 2022.
[45]
C. Mao, L. Yao, and Y. Luo, “Imagegcn: multi-relational image graph convolutional networks for disease identification with chest x-rays,” 2019, https://arxiv.org/abs/1904.00325.
[46]
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014.
[47]
J. Hu, Y. Chen, L. Zhang, and Z. Yi, “Surrogate dropout: learning optimal drop rate through proxy,” Knowledge-Based Systems, vol. 206, 2020.
[48]
D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” 2015, https://arxiv.org/abs/1412.6980.
[49]
A. Hatamizadeh, V. Nath, Y. Tang, D. Yang, H. R. Roth, and D. Xu, “Swin unetr: swin transformers for semantic segmentation of brain tumors in mri images,” International MICCAI Brainlesion Workshop, pp. 272–284, Springer, Berlin, Germany, 2021.
[50]
J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132–7141, Salt Lake City, UT, USA, June 2018.
Recommendations
Automatic Tumor Segmentation with Deep Convolutional Neural Networks for Radiotherapy Applications
Accurate tumor delineation in medical images is of great importance in guiding radiotherapy. In nasopharyngeal carcinoma (NPC), due to its high variability, low contrast and discontinuous boundaries in magnetic resonance images (MRI), the margin of the ...
Segmentation of Head and Neck Organs-At-Risk in Longitudinal CT Scans Combining Deformable Registrations and Convolutional Neural Networks
Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision SupportAbstractAutomated segmentation of organs-at-risk (OAR) in follow-up images of the patient acquired during the course of treatment could greatly facilitate adaptive treatment planning in radiotherapy. Instead of segmenting each image separately, the ...
Pancreas Segmentation in MRI Using Graph-Based Decision Fusion on Convolutional Neural Networks
Medical Image Computing and Computer Assisted Intervention − MICCAI 2017
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024 Junjie Hu et al.
This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Publisher
John Wiley and Sons Ltd.
United Kingdom
Publication History
Published: 26 April 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Jan 2025