Abstract
Segmentation of focal liver lesions serves as an essential preprocessing step for initial diagnosis, stage differentiation, and post-treatment efficacy evaluation. Multimodal MRI scans (e.g., T1WI, T2WI) provide complementary information on liver lesions and is widely used for diagnosis. However, some modalities (e.g., T1WI) have high resolution but lack of important visual information (e.g., edge) belonged to other modalities (T2WI), it is significant to enhance tissue lesion quality in T1WI using other modality priors (T2WI) and improve segmentation performance. In this paper, we propose a graph learning based approach with the motivation of extracting modality-specific features efficiently and establishing the regional correspondence effectively between T1WI and T2WI. We first project deep features into a graph domain and employ graph convolution to propagate information across all regions for extraction of modality-specific features. Then we propose a mutual information based graph co-attention module to learn weight coefficients of one bipartite graph, which is constructed by the fully-connection of graphs with different modalities in the graph domain. At last, we get the final refined features for segmentation by re-projection and residual connection. We validate our method on a multimodal MRI liver lesion dataset. Experimental results show that the proposed approach achieves improvement of liver lesion segmentation in T1WI by learning guided features from multimodal priors (T2WI) compared to existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Forner, A., Reig, M., Bruix, J.: Hepatocellular carcinoma. Lancet (London, England) 391, 1301–1314 (2018). https://doi.org/10.1016/s0140-6736(18)30010-2
Chen, C., Dou, Q., Jin, Y., Chen, H., Qin, J., Heng, P.-A.: Robust multimodal brain tumor segmentation via feature disentanglement and gated fusion. In: Shen, D., et al. (eds.) MICCAI 2019. Robust multimodal brain tumor segmentation via feature disentanglement and gated fusion, vol. 11766, pp. 447–456. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_50
Chen, Y., Rohrbach, M., Yan, Z., Shuicheng, Y., Feng, J., Kalantidis, Y.: Graph-based global reasoning networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 433–442 (2019)
Dolz, J., Desrosiers, C., Ayed, I.B.: IVD-Net: intervertebral disc localization and segmentation in MRI with a multi-modal UNet. In: Zheng, G., Belavy, D., Cai, Y., Li, S. (eds.) CSI 2018. LNCS, vol. 11397, pp. 130–143. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-13736-6_11
Dolz, J., Gopinath, K., Yuan, J., Lombaert, H., Desrosiers, C., Ayed, I.B.: HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation. IEEE Trans. Med. Imaging 38(5), 1116–1126 (2018)
El-Serag, H.B.: Epidemiology of hepatocellular carcinoma. In: The Liver: Biology and Pathobiology, pp. 758–772 (2020)
Havaei, M., Guizard, N., Chapados, N., Bengio, Y.: HeMIS: hetero-modal image segmentation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 469–477. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_54
Jansen, M.J., et al.: Liver segmentation and metastases detection in MR images using convolutional neural networks. J. Med. Imaging 6(4), 044003 (2019)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24–26, 2017, Conference Track Proceedings. OpenReview.net (2017). https://openreview.net/forum?id=SJU4ayYgl
Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Liang, X., Hu, Z., Zhang, H., Lin, L., Xing, E.P.: Symbolic graph reasoning meets convolutions. In: Advances in Neural Information Processing Systems, pp. 1853–1863 (2018)
Maes, F., Vandermeulen, D., Suetens, P.: Medical image registration using mutual information. Proc. IEEE 91(10), 1699–1722 (2003)
Marstal, K., Berendsen, F., Staring, M., Klein, S.: SimpleElastix: a user-friendly, multi-lingual library for medical image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 134–142 (2016)
Mazurowski, M.A., Buda, M., Saha, A., Bashir, M.R.: Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI. J. Magn. Reson. Imaging 49(4), 939–954 (2019)
Sedghi, A., et al.: Semi-supervised image registration using deep learning. In: Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 10951, p. 109511G. International Society for Optics and Photonics (2019)
Thekumparampil, K.K., Wang, C., Oh, S., Li, L.J.: Attention-based graph neural network for semi-supervised learning. arXiv preprint arXiv:1803.03735 (2018)
Jansen, M.J., et al.: Liver segmentation and metastases detection in MR images using convolutional neural networks. J. Med. Imaging 6(4), 044003 (2019)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30–May 3, 2018, Conference Track Proceedings. OpenReview.net (2018). https://openreview.net/forum?id=rJXMpikCZ
Xiao, X., et al.: Radiomics-guided GAN for segmentation of liver tumor without contrast agents. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 237–245. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_27
Zeng, Q., et al.: Liver segmentation in magnetic resonance imaging via mean shape fitting with fully convolutional neural networks. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 246–254. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_28
Zhou, T., Ruan, S., Canu, S.: A review: deep learning for medical image segmentation using multi-modality fusion. Array, p. 100004 (2019)
Acknowledgements
This work was supported in part by Major Scientific Research Project of Zhejiang Lab under the Grant No. 2018DG0ZX01, and in part by the Grant-in Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under the Grant No. 18H03267 and No.17H00754.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Mo, S. et al. (2020). Multimodal Priors Guided Segmentation of Liver Lesions in MRI Using Mutual Information Based Graph Co-Attention Networks. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12264. Springer, Cham. https://doi.org/10.1007/978-3-030-59719-1_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-59719-1_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59718-4
Online ISBN: 978-3-030-59719-1
eBook Packages: Computer ScienceComputer Science (R0)