Abstract
Purpose
In recent years, Non-Local-based methods have been successfully applied to lung nodule classification. However, these methods offer 2D attention or limited 3D attention to low-resolution feature maps. Moreover, they still depend on a convenient local filter such as convolution as full 3D attention is expensive to compute and requires a big dataset, which might not be available.
Methods
We propose to use 3D Axial-Attention, which requires a fraction of the computing power of a regular Non-Local network (i.e., self-attention). Unlike a regular Non-Local network, the 3D Axial-Attention network applies the attention operation to each axis separately. Additionally, we solve the invariant position problem of the Non-Local network by proposing to add 3D positional encoding to shared embeddings.
Results
We validated the proposed method on 442 benign nodules and 406 malignant nodules, extracted from the public LIDC-IDRI dataset by following a rigorous experimental setup using only nodules annotated by at least three radiologists. Our results show that the 3D Axial-Attention model achieves state-of-the-art performance on all evaluation metrics, including AUC and Accuracy.
Conclusions
The proposed model provides full 3D attention, whereby every element (i.e., pixel) in the 3D volume space attends to every other element in the nodule effectively. Thus, the 3D Axial-Attention network can be used in all layers without the need for local filters. The experimental results show the importance of full 3D attention for classifying lung nodules.
This is a preview of subscription content, log in via an institution to check access.
Data availability
This article used the LIDC-IDRI public dataset.
References
American Cancer Society (2020) Cancer Facts & Figures
Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, Gareen IF, Gatsonis C, Marcus PM, Sicks JRD (2011) Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 365:395–409. https://doi.org/10.1056/NEJMoa1102873
Shen W, Zhou M, Yang F, Yu D, Dong D, Yang C, Zang Y, Tian J (2017) Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognit 61:663–673. https://doi.org/10.1016/j.patcog.2016.05.029
Ren Y, Tsai MY, Chen L, Wang J, Li S, Liu Y, Jia X, Shen C (2020) A manifold learning regularization approach to enhance 3D CT image-based lung nodule classification. Int J Comput Assist Radiol Surg 15:287–295. https://doi.org/10.1007/s11548-019-02097-8
Shen C, Tsai M-Y, Chen L, Li S, Nguyen D, Wang J, Jiang SB, Jia X (2020) On the robustness of deep learning based lung nodule classification for CT images with respect to image noise. Phys Med Biol. https://doi.org/10.1088/1361-6560/abc812
Al-Shabi M, Lee HK, Tan M (2019) Gated-dilated networks for lung nodule classification in CT Scans. IEEE Access 7:178827–178838. https://doi.org/10.1109/ACCESS.2019.2958663
Onishi Y, Teramoto A, Tsujimoto M, Tsukamoto T, Saito K, Toyama H, Imaizumi K, Fujita H (2020) Multiplanar analysis for pulmonary nodule classification in CT images using deep convolutional neural network and generative adversarial networks. Int J Comput Assist Radiol Surg 15:173–178. https://doi.org/10.1007/s11548-019-02092-z
Al-Shabi M, Lan BL, Chan WY, Ng K-H, Tan M (2019) Lung nodule classification using deep Local-Global networks. Int J Comput Assist Radiol Surg 14:1815–1819. https://doi.org/10.1007/s11548-019-01981-7
Jiang H, Gao F, Xu X, Huang F, Zhu S (2020) Attentive and ensemble 3D dual path networks for pulmonary nodules classification. Neurocomputing 398:422–430. https://doi.org/10.1016/j.neucom.2019.03.103
Wang X, Girshick R, Gupta A, He K (2018) Non-local Neural Networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, pp 7794–7803
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł ukasz, Polosukhin I (2017) Attention is All you Need. In: Guyon I, Luxburg U V, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in Neural Information Processing Systems 30. Curran Associates, Inc., pp 5998–6008
Al-Shabi M, Shak K, Tan M (2020) ProCAN: Progressive Growing Channel Attentive Non-Local Network for Lung Nodule Classification. ArXiv:2010.15417
Ho J, Kalchbrenner N, Weissenborn D, Salimans T (2019) Axial Attention in Multidimensional Transformers. ArXiv:1912.12180
Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA, Kazerooni EA, MacMahon H, Van Beek EJR, Yankelevitz D, Biancardi AM, Bland PH, Brown MS, Engelmann RM, Laderach GE, Max D, Pais RC, Qing DPY, Roberts RY, Smith AR, Starkey A, Batra P, Caligiuri P, Farooqi A, Gladish GW, Jude CM, Munden RF, Petkovska I, Quint LE, Schwartz LH, Sundaram B, Dodd LE, Fenimore C, Gur D, Petrick N, Freymann J, Kirby J, Hughes B, Vande Casteele A, Gupte S, Sallam M, Heath MD, Kuhn MH, Dharaiya E, Burns R, Fryd DS, Salganicoff M, Anand V, Shreter U, Vastagh S, Croft BY, Clarke LP (2011) The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 38:915–931. https://doi.org/10.1118/1.3528204
Ba J, Kiros J, Hinton GE (2016) Layer normalization. ArXiv:1607.06450
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958. https://doi.org/10.1214/12-AOS1000
Kingma DP, Ba J (2015) Adam: A Method for Stochastic Optimization. In: Bengio Y, LeCun Y (eds) 3rd International Conference on Learning Representations, ICLR 2015,San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings
Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, pp 249–256
Shen S, Han SX, Aberle DR, Bui AA, Hsu W (2019) An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification. Expert Syst Appl 128:84–95. https://doi.org/10.1016/j.eswa.2019.01.048
Acknowledgements
This work was supported by the Fundamental Research Grant Scheme (FRGS), Ministry of Education Malaysia (MOE), under grant FRGS/1/2018/ICT02/MUSM/03/1, the Electrical and Computer Systems Engineering and Advanced Engineering Platform, School of Engineering, Monash University Malaysia and the TWAS-COMSTECH Joint Research Grant, UNESCO.
Funding
This study was funded by Fundamental Research Grant Scheme (FRGS), Ministry of Education Malaysia (MOE), under grant FRGS/1/2018/ICT02/MUSM/03/1, the Electrical and Computer Systems Engineering and Advanced Engineering Platform, School of Engineering, Monash University Malaysia and the TWAS-COMSTECH Joint Research Grant, UNESCO.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Al-Shabi, M., Shak, K. & Tan, M. 3D axial-attention for lung nodule classification. Int J CARS 16, 1319–1324 (2021). https://doi.org/10.1007/s11548-021-02415-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-021-02415-z