Abstract
3D features are desired in nature for segmenting CT volumes. It is, however, computationally expensive to employ a 3D convolutional neural network (CNN) to learn 3D features. Existing methods hence learn 3D features by still relying on 2D CNNs while attempting to consider more 2D slices, but up until now it is difficulty for them to consider the whole volumetric data, resulting in information loss and performance degradation. In this paper, we propose a simple and effective technique that allows a 2D CNN to learn 3D features for segmenting CT volumes. Our key insight is that all boundary voxels of a 3D object form a surface that can be represented by using a 2D matrix, and therefore they can be perfectly recognized by a 2D CNN in theory. We hence learn 3D features for recognizing these boundary voxels by learning the projection distance between a set of prescribed spherical surfaces and the object’s surface, which can be readily performed by a 2D CNN. By doing so, we can consider the whole volumetric data when spherical surfaces are sampled sufficiently dense, without any information loss. We assessed the proposed method on a publicly available dataset. The experimental evidence shows that the proposed method is effective, outperforming existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Available on https://zenodo.org/record/1169361#.XSFOm-gzYuU.
References
Cerrolaza, J., Picazo, M., Humbert, L., et al.: Computational anatomy for multi-organ analysis in medical imaging: a review. Med. Image Anal. 56, 44–67 (2019)
Roth, H., Shen, Ch., Oda, H., et al.: Deep learning and its application to medical image segmentation. Med. Imaging Technol. 36(2), 63–71 (2018)
Roth, H.R., et al.: A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 417–425. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_48
Kakeya, H., Okada, T., Oshiro, Y.: 3D U-JAPA-Net: mixture of convolutional networks for abdominal multi-organ CT segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 426–433. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_49
Chen, J., Yang, L., Zhang, Y., et al.: Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation. In: Advances in Neural Information Processing Systems, pp. 3036–3044 (2016)
Christ, P., et al.: Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 415–423. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_48
Cai, J., Lu, L., Xie, Y., et al.: Improving deep pancreas segmentation in CT and MRI images via recurrent neural contextual learning and direct loss function. arXiv preprint arXiv:1707.04912 (2017)
Novikov, A., Major, D., Wimmer, M., et al.: Deep sequential segmentation of organs in volumetric medical scans. IEEE Trans. Med. Imaging 38(5), 1207–1215 (2018)
Li, X., Chen, H., Qi, X., et al.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018)
Xia, Y., Xie, L., Liu, F., Zhu, Z., Fishman, E.K., Yuille, A.L.: Bridging the gap between 2D and 3D organ segmentation with volumetric fusion net. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 445–453. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_51
Yu, Q., Xia, Y., Xie, L., et al.: Thickened 2D networks for 3D medical image segmentation. arXiv preprint arXiv:1904.01150 (2019)
Ambellan, F., Tack, A., Ehlke, M., et al.: Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: data from the osteoarthritis initiative. Med. Image Anal. 52, 109–118 (2019)
Wang, Y., Zhou, Y., Shen, W., et al.: Abdominal multi-organ segmentation with organ-attention networks and statistical fusion. Med. Image Anal. 55, 88–102 (2019)
Wang, Z., Wang, G.: Triplanar convolutional neural network for automatic liver and tumor image segmentation. International Journal of Performability Engineering 14(12), 3151–3158 (2019)
Li, Y., et al.: Volumetric medical image segmentation: a 3D deep coarse-to-fine framework and its adversarial examples. In: Lu, L., Wang, X., Carneiro, G., Yang, L. (eds.) Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics. ACVPR, pp. 69–91. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-13969-8_4
Quattoni, A., Collins, M. and Darrell, T.: Conditional random fields for object recognition. In. Advances in Neural Information Processing Systems, pp. 1097–1104 (2005)
Zheng, S., Jayasumana, S., Romera-Paredes B., et al.: Conditional random fields as recurrent neural networks. In. IEEE International Conference on Computer Vision, pp. 1529–1537 (2015)
Tianwei, N., Lingxi, X., Huangjie, Zh, et al.: Elastic boundary projection for 3D medical image segmentation. In: IEEE International Conference on Computer Vision, pp. 2109–2118 (2019)
Lipton, Z., Berkowitz, J. and Elkan, C.: A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019 (2015)
Mandic, D. and Chambers, J.: Recurrent neural networks for prediction: learning algorithms, architectures and stability. (2001)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Zhang, C., Bengio, S., Hardt, M., et al.: Understanding deep learning requires rethinking generalization. In: International Conference on Learning Representations, pp. 1–15 (2017)
Arpit, D., Jastrzebski, S., Ballas, N., et al.: A closer look at memorization in deep networks. In: International Conference on Machine Learning, pp. 233–242 (2017)
Ma, X., Wang, Y., Houle, M., et al.: Dimensionality-driven learning with noisy labels. In: International Conference on Machine Learning, pp. 3361–3370 (2018)
Warfield, S., Zou, K., Wells, W.: Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans. Med. Imaging 23(7), 903–921 (2004)
Soler, L., Hostettler, A., Agnus, V., et al.: 3D image reconstruction for comparison of algorithm database: a patient specific anatomical and medical image database. Technical report IRCAD, Strasbourg, France (2010)
Gibson, E., Giganti, F., Hu, Y., et al.: Automatic multi-organ segmentation on abdominal CT with dense v-networks. IEEE Trans. Med. Imaging 37(8), 1822–1834 (2018)
Clark, K., Vendt, B., Smith, K., et al.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26(6), 1045–1057 (2013)
Roth, H.R., et al.: DeepOrgan: multi-level deep convolutional networks for automated pancreas segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 556–564. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_68
Roth, H., Farag, A., Turkbey, E., et al.: Data from pancreas-CT. The cancer imaging archive. (2015)
Landman, B., Xu, Z., Eugenio, I., et al.: MICCAI multi-atlas labeling beyond the cranial vault-workshop and challenge (2015)
Xu, Z., Lee, C., Heinrich, M., et al.: Evaluation of six registration methods for the human abdomen on clinically acquired CT. IEEE Trans. Biomed. Eng. 63(8), 1563–1572 (2016)
Isensee, F., Petersen, J., Klein, A., et al.: NNU-Net: self-adapting framework for u-net-based medical image segmentation. arXiv preprint arXiv:1809.10486 (2018)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Acknowledgement
The work described in this paper is supported by a grant from the Hong Kong Research Grants Council (Project No. PolyU 152035/17E), a grant from the Natural Foundation of China (Grant No. 61902232), a grant from the Li Ka Shing Foundation Cross-Disciplinary Research (Grant no. 2020LKSFG05D), a grant from the Innovative Technology Fund (Grant No. MRP/015/18), and a grant from the General Research Fund (Grant No. PolyU 152006/19E).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Song, Y. et al. (2020). Learning 3D Features with 2D CNNs via Surface Projection for CT Volume Segmentation. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12264. Springer, Cham. https://doi.org/10.1007/978-3-030-59719-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-59719-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59718-4
Online ISBN: 978-3-030-59719-1
eBook Packages: Computer ScienceComputer Science (R0)