Abstract
Pulmonary airway labeling identifies anatomical names for branches in bronchial trees. These fine-grained labels are critical for disease diagnosis and intra-operative navigation. Recently, various methods have been proposed for this task. However, accurate labeling of each bronchus is challenging due to the fine-grained categories and inter-individual variations. On the one hand, training a network with limited data to recognize multitudinous classes sets an obstacle to the design of algorithms. We propose to maximize the use of latent relationships by a transformer-based network. Neighborhood information is properly integrated to capture the priors in the tree structure, while a U-shape layout is introduced to exploit the correspondence between different nomenclature levels. On the other hand, individual variations cause the distribution overlapping of adjacent classes in feature space. To resolve the confusion between sibling categories, we present a novel generator that predicts the weight matrix of the classifier to produce dynamic decision boundaries between subsegmental classes. Extensive experiments performed on publicly available datasets demonstrate that our method can perform better than state-of-the-art methods. The code is publicly available at https://github.com/EndoluminalSurgicalVision-IMR/AirwayFormer.
This work was partly supported by National Key R &D Program of China (2019YFB1311503, 2017YFC0112700), Committee of Science and Technology, Shanghai, China (19510711200), Shanghai Sailing Program (20YF1420800), NSFC (61661010, 61977046, 62003208), SJTU Trans-med Awards Research (20210101), Science and Technology Commission of Shanghai Municipality (20DZ2220400).
W. Yu and H. Zheng—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Dwivedi, V.P., Bresson, X.: A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699 (2020)
Feragen, A., et al.: Geodesic atlas-based labeling of anatomical trees: application and evaluation on airways extracted from CT. IEEE Trans. Med. Imaging 34(6), 1212–1226 (2014)
Hendrycks, D., Gimpel, K.: Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415 (2016)
Huang, J., Yang, J.: Unignn: a unified framework for graph and hypergraph neural networks. arXiv preprint arXiv:2105.00956 (2021)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Kitaoka, H., et al.: Automated nomenclature labeling of the bronchial tree in 3D-CT lung images. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2489, pp. 1–11. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45787-9_1
Lo, P., van Rikxoort, E.M., Goldin, J., Abtin, F., de Bruijne, M., Brown, M.: A bottom-up approach for labeling of human airway trees. MICCAI Int. WS. Pulm. Im. Anal. (2011)
Min, E., et al.: Transformer for graphs: an overview from architecture perspective. arXiv preprint arXiv:2202.08455 (2022)
Mori, K., Hasegawa, J.I., Suenaga, Y., Toriwaki, J.I.: Automated anatomical labeling of the bronchial branch and its application to the virtual bronchoscopy system. IEEE Trans. Med. Imaging 19(2), 103–114 (2000)
Mori, K., et al.: Automated anatomical labeling of bronchial branches extracted from CT datasets based on machine learning and combination optimization and its application to bronchoscope guidance. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009. LNCS, vol. 5762, pp. 707–714. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04271-3_86
Netter, F.H.: Atlas of human anatomy, Professional Edition E-Book: including NetterReference. com Access with full downloadable image Bank. Elsevier health sciences (2014)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Tan, Z., Feng, J., Zhou, J.: SGNet: structure-aware graph-based network for airway semantic segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 153–163. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_15
Thost, V., Chen, J.: Directed acyclic graph neural networks. arXiv preprint arXiv:2101.07965 (2021)
Tschirren, J., McLennan, G., Palágyi, K., Hoffman, E.A., Sonka, M.: Matching and anatomical labeling of human airway tree. IEEE Trans. Med. Imaging 24(12), 1540–1547 (2005)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Yadati, N., Nimishakavi, M., Yadav, P., Nitin, V., Louis, A., Talukdar, P.: Hypergcn: a new method for training graph convolutional networks on hypergraphs. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Ying, C., et al.: Do transformers really perform badly for graph representation? Adv. Neural. Inf. Process. Syst. 34, 28877–28888 (2021)
Yu, W., et al.: TNN: tree neural network for airway anatomical labeling. IEEE Trans. Med. Imaging 42(1), 103–118 (2022)
Zhao, T., Yin, Z., Wang, J., Gao, D., Chen, Y., Mao, Y.: Bronchus segmentation and classification by neural networks and linear programming. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 230–239. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_26
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, W., Zheng, H., Gu, Y., Xie, F., Sun, J., Yang, J. (2023). AirwayFormer: Structure-Aware Boundary-Adaptive Transformers for Airway Anatomical Labeling. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14226. Springer, Cham. https://doi.org/10.1007/978-3-031-43990-2_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-43990-2_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43989-6
Online ISBN: 978-3-031-43990-2
eBook Packages: Computer ScienceComputer Science (R0)