Abstract
The boundary of tumors (hepatocellular carcinoma, or HCC) contains rich semantics: capsular invasion, visibility, smoothness, folding and protuberance, etc. Capsular invasion on tumor boundary has proven to be clinically correlated with the prognostic indicator, microvascular invasion (MVI). Investigating tumor boundary semantics has tremendous clinical values. In this paper, we propose the first and novel computational framework that disentangles the task into two components: spatial vertex localization and sequential semantic classification. (1) A HCC tumor segmentor is built for tumor mask boundary extraction, followed by polar transform representing the boundary with radius and angle. Vertex generator is used to produce fixed-length boundary vertices where vertex features are sampled on the corresponding spatial locations. (2) The sampled deep vertex features with positional embedding are mapped into a sequential space and decoded by a multilayer perceptron (MLP) for semantic classification. Extensive experiments on tumor capsule semantics demonstrate the effectiveness of our framework. Mining the correlation between the boundary semantics and MVI status proves the feasibility to integrate this boundary semantics as a valid HCC prognostic biomarker.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
An, C., Kim, M.J.: Imaging features related with prognosis of hepatocellular carcinoma. Abdominal Radiol. 44(2), 509–516 (2019)
Bertasius, G., Shi, J., Torresani, L.: High-for-low and low-for-high: efficient boundary detection from deep object features and its applications to high-level vision. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 504–512 (2015)
Chan, A.W., et al.: Development of pre and post-operative models to predict early recurrence of hepatocellular carcinoma after surgical resection. J. Hepatol. 69(6), 1284–1293 (2018)
Chen, G., Chen, J., Lienen, M., Conradt, J., Röhrbein, F., Knoll, A.C.: Flgr: fixed length gists representation learning for rnn-hmm hybrid-based neuromorphic continuous gesture recognition. Front. Neuroscience 13, 73 (2019)
Chen, J., et al.: Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
Choi, J.Y., Lee, J.M., Sirlin, C.B.: Ct and mr imaging diagnosis and staging of hepatocellular carcinoma: part i. development, growth, and spread: key pathologic and imaging aspects. Radiology 272(3), 635–654 (2014)
Dosovitskiy, A., et al.: An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale. In: ICLR (2021)
Ehman, E.C., et al.: Rate of observation and inter-observer agreement for li-rads major features at ct and mri in 184 pathology proven hepatocellular carcinomas. Abdominal Radiol. 41(5), 963–969 (2016)
Hariharan, B., Arbeláez, P., Bourdev, L., Maji, S., Malik, J.: Semantic contours from inverse detectors. In: 2011 International Conference on Computer Vision, pp. 991–998. IEEE (2011)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hou, X., Yuille, A., Koch, C.: Boundary detection benchmarking: beyond f-measures. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2123–2130 (2013)
Kojiro, M.: Histopathology of liver cancers. Best Pract. Res. Clin. Gastroenterol. 19(1), 39–62 (2005)
Li, X., Sun, X., Meng, Y., Liang, J., Wu, F., Li, J.: Dice loss for data-imbalanced nlp tasks. arXiv preprint arXiv:1911.02855 (2019)
Liang, J., Homayounfar, N., Ma, W.-C., Xiong, Y., Hu, R., Urtasun, R.: Polytransform: deep polygon transformer for instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9131–9140 (2020)
Liu, R., et al.: An intriguing failing of convolutional neural networks and the coordconv solution. arXiv preprint arXiv:1807.03247 (2018)
Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., Yuille, A.: Deep captioning with multimodal recurrent neural networks (m-rnn). arXiv preprint arXiv:1412.6632 (2014)
Qi, H., Collins, S., Alison Noble, J.: Upi-net: semantic contour detection in placental ultrasound. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, p. 0 (2019)
Schmidt, U., Weigert, M., Broaddus, C., Myers, G.: Cell detection with star-convex polygons. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 265–273. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_30
Shah, S.A., et al.: Recurrence after liver resection for hepatocellular carcinoma: risk factors, treatment, and outcomes. Surgery 141(3), 330–339 (2007)
Shen, W., Wang, X., Wang, Y., Bai, X., Zhang, Z.: Deepcontour: a deep convolutional feature learned by positive-sharing loss for contour detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3982–3991 (2015)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. arXiv preprint arXiv:1409.3215 (2014)
Tang, Y., Tang, Y., Zhu, Y., Xiao, J., Summers, R.M.: E2net: an edge enhanced network for accurate liver and tumor segmentation on ct scans. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 512–522. Springer (2020)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Xie, E., et al.: Polarmask: single shot instance segmentation with polar representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12193–12202 (2020)
Xu, W., Wang, H., Qi, F., Lu, C.: Explicit shape encoding for real-time instance segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5168–5177 (2019)
Yao, J., Shi, Y., Lu, L., Xiao, J., Zhang, L.: DeepPrognosis: preoperative prediction of pancreatic cancer survival and surgical margin via contrast-enhanced CT imaging. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 272–282. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_27
Zhu, F., Yang, F., Li, J., Chen, W., Yang, W.: Incomplete tumor capsule on preoperative imaging reveals microvascular invasion in hepatocellular carcinoma: a systematic review and meta-analysis. Abdominal Radiol. 44(9), 3049–3057 (2019)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, JN. et al. (2021). Sequential Learning on Liver Tumor Boundary Semantics and Prognostic Biomarker Mining. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12907. Springer, Cham. https://doi.org/10.1007/978-3-030-87234-2_72
Download citation
DOI: https://doi.org/10.1007/978-3-030-87234-2_72
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87233-5
Online ISBN: 978-3-030-87234-2
eBook Packages: Computer ScienceComputer Science (R0)