More Web Proxy on the site http://driver.im/

research-article

A deep neural architecture for harmonizing 3-D input data analysis and decision making in medical imaging

Authors:

Dimitrios Kollias,

Anastasios Arsenos,

Stefanos KolliasAuthors Info & Claims

Volume 542, Issue C

https://doi.org/10.1016/j.neucom.2023.126244

Published: 14 July 2023 Publication History

Abstract

Harmonizing the analysis of data, especially of 3-D image volumes, consisting of different number of slices and annotated per volume, is a significant problem in training and using deep neural networks in various applications, including medical imaging. Moreover, unifying the decision making of the networks over different input datasets is crucial for the generation of rich data-driven knowledge and for trusted usage in the applications. This paper presents a new deep neural architecture, named RACNet, which includes routing and feature alignment steps and effectively handles different input lengths and single annotations of the 3-D image inputs, whilst providing highly accurate decisions. In addition, through latent variable extraction from the trained RACNet, a set of anchors are generated providing further insight on the network’s decision making. These can be used to enrich and unify data-driven knowledge extracted from different datasets. An extensive experimental study illustrates the above developments, focusing on COVID-19 diagnosis through analysis of 3-D chest CT scans from databases generated in different countries and medical centers.

References

[1]

A. Amyar, R. Modzelewski, H. Li, S. Ruan, Multi-task deep learning based ct imaging analysis for covid-19 pneumonia: Classification and segmentation, Comput. Biol. Med. 126 (2020).

Digital Library

[2]

X. Wang, X. Deng, Q. Fu, Q. Zhou, J. Feng, H. Ma, W. Liu, C. Zheng, A weakly-supervised framework for covid-19 classification and lesion localization from chest ct, IEEE Trans. Med. Imaging 39 (8) (2020) 2615–2625.

[3]

L. Zhang, M. Wang, M. Liu, D. Zhang, A survey on deep learning for neuroimaging-based brain disorder analysis, Front. Neurosci. (2020) 779.

[4]

A. Tagaris, D. Kollias, A. Stafylopatis, G. Tagaris, S. Kollias, Machine learning for neurodegenerative disorder diagnosis–survey of practices and launch of benchmark dataset, Int. J. Artif. Intell. Tools 27 (03) (2018) 1850011.

[5]

F. Gu, M.-H. Chung, M. Chignell, S. Valaee, B. Zhou, X. Liu, A survey on deep learning for human activity recognition, ACM Comput. Surveys 54 (8) (2021) 1–34.

[6]

T. Zhang, Z. Tan, Deep emotion recognition using facial, speech and textual cues: A survey.

[7]

F. Caliva, F.S. De Ribeiro, A. Mylonakis, C. Demazi’ere, P. Vinai, G. Leontidis, S. Kollias, A deep learning approach to anomaly detection in nuclear reactors, in: 2018 International joint conference on neural networks (IJCNN), IEEE, 2018, pp. 1–8.

[8]

J. Wang, C. Lan, C. Liu, Y. Ouyang, W. Zeng, T. Qin, Generalizing to unseen domains: A survey on domain generalization, arXiv preprint arXiv:2103.03097.

[9]

S. Wang, B. Kang, J. Ma, X. Zeng, M. Xiao, J. Guo, M. Cai, J. Yang, Y. Li, X. Meng, et al., A deep learning algorithm using ct images to screen for corona virus disease (covid-19), Eur. Radiol. (2021) 1–9.

[10]

K. Zhang, X. Liu, J. Shen, Z. Li, Y. Sang, X. Wu, Y. Zha, W. Liang, C. Wang, K. Wang, L. Ye, M. Gao, Z. Zhou, L. Li, J. Wang, Z. Yang, H. Cai, J. Xu, L. Yang, W. Cai, W. Xu, S. Wu, W. Zhang, S. Jiang, L. Zheng, X. Zhang, L. Wang, L. Lu, J. Li, H. Yin, W. Wang, O. Li, C. Zhang, L. Liang, T. Wu, R. Deng, K. Wei, Y. Zhou, T. Chen, J. Yiu-Nam Lau, M. Fok, J. He, T. Lin, W. Li, G. Wang, Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography, Cell 182 (5) (2020) 1360.

[11]

S.P. Morozov, A.E. Andreychenko, I.A. Blokhin, P.B. Gelezhe, A.P. Gonchar, A.E. Nikolaev, N.A. Pavlov, V.Y. Chernina, V.A. Gombolevskiy, Mosmeddata: data set of 1110 chest ct scans performed during the covid-19 epidemic, Digital Diagnostics 1 (1) (2020) 49–59.

[12]

X. Yang, X. He, J. Zhao, Y. Zhang, S. Zhang, P. Xie, Covid-ct-dataset: a ct image dataset about covid-19, arXiv preprint arXiv:2003.13865.

[13]

A. Jaiswal, N. Gianchandani, D. Singh, V. Kumar, M. Kaur, Classification of the covid-19 infected patients using densenet201 based deep transfer learning, J. Biomol. Struct. Dyn. (2020) 1–8.

[14]

X. He, S. Wang, S. Shi, X. Chu, J. Tang, X. Liu, C. Yan, J. Zhang, G. Ding, Benchmarking deep learning models and automated model design for covid-19 detection with chest ct scans, medRxiv arXiv:https://www.medrxiv.org/content/early/2021/11/04/2020.06.08.20125963.full.pdf. URL: https://www.medrxiv.org/content/early/2021/11/04/2020.06.08.20125963.

[15]

X. Wu, C. Chen, M. Zhong, J. Wang, J. Shi, COVID-AL: The diagnosis of COVID-19 with deep active learning, Med. Image Anal. 68 (2020).

[16]

X. He, S. Wang, X. Chu, S. Shi, J. Tang, X. Liu, C. Yan, J. Zhang, G. Ding, Automated model design and benchmarking of deep learning models for covid-19 detection with chest ct scans, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021, pp. 4821–4829.

[17]

X. He, G. Ying, J. Zhang, X. Chu, Evolutionary multi-objective architecture search framework: Application to covid-19 3d ct classification, in: L. Wang, Q. Dou, P.T. Fletcher, S. Speidel, S. Li (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, Springer Nature Switzerland, Cham, 2022, pp. 560–570.

[18]

D. Tran, H. Wang, L. Torresani, J. Ray, Y. LeCun, M. Paluri, A closer look at spatiotemporal convolutions for action recognition (2017). URL: https://arxiv.org/abs/1711.11248.

[19]

A. Khadidos, A.O. Khadidos, S. Kannan, Y. Natarajan, S.N. Mohanty, G. Tsaramirsis, Analysis of covid-19 infections on a ct image using deepsense model, Front. Public Health 8.

[20]

S.A. Ali Ahmed, M.C. Yavuz, M.U. Şen, F. Gülşen, O. Tutar, B. Korkmazer, C. Samancaı, S. Şirolu, R. Hamid, A.E. Eryürekli, T. Mammadov, B. Yanikoglu, Comparison and ensemble of 2d and 3d approaches for covid-19 detection in ct images, Neurocomputing 488 (2022) 457–469,.

Digital Library

[21]

X. Li, W. Tan, P. Liu, Q. Zhou, J. Yang, Classification of covid-19 chest ct images based on ensemble deep learning, J. Healthcare Eng. 2021 (2021) 5528441,.

[22]

D. Arthur, S. Vassilvitskii, k-means++: The advantages of careful seeding, (Tech. rep.) Stanford, 2006.

[23]

D. Kollias, A. Arsenos, L. Soukissian, S. Kollias, Mia-cov19d: Covid-19 detection through 3-d chest ct image analysis, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 537–544.

[24]

D. Kollias, N. Bouas, Y. Vlaxos, V. Brillakis, M. Seferis, I. Kollia, L. Sukissian, J. Wingate, S. Kollias, Deep transparent prediction through latent representation analysis, arXiv preprint arXiv:2009.07044.

[25]

D. Kollias, Y. Vlaxos, M. Seferis, I. Kollia, L. Sukissian, J. Wingate, S.D. Kollias, Transparent adaptation in deep medical image diagnosis., in: TAILOR, 2020, pp. 251–267.

[26]

D. Kollias, A. Tagaris, A. Stafylopatis, S. Kollias, G. Tagaris, Deep neural architectures for prediction in healthcare, Complex Intell. Syst. 4 (2) (2018) 119–131.

[27]

D. Kollias, A. Arsenos, S. Kollias, Ai-mia: Covid-19 detection and severity analysis through medical imaging, in: Computer Vision–ECCV 2022 Workshops: Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VII, Springer, 2023, pp. 677–690.

[28]

A. Arsenos, D. Kollias, S. Kollias, A large imaging database and novel deep neural architecture for covid-19 diagnosis, in: 2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP), IEEE, 2022, p. 1–5.

[29]

J. Zhao, Y. Zhang, X. He, P. Xie, Covid-ct-dataset: a ct scan dataset about covid-19, arXiv preprint arXiv:2003.13865.

[30]

K. Hara, H. Kataoka, Y. Satoh, Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 6546–6555.

[31]

Z. Han, B. Wei, Y. Hong, T. Li, J. Cong, X. Zhu, H. Wei, W. Zhang, Accurate screening of covid-19 using attention-based deep 3d multiple instance learning, IEEE Trans. Med. Imaging 39 (8) (2020) 2584–2594.

[32]

S. Chen, K. Ma, Y. Zheng, Med3d: Transfer learning for 3d medical image analysis, arXiv preprint arXiv:1904.00625.

[33]

C.-C. Hsu, G.-L. Chen, M.-H. Wu, Visual transformer with statistical test for covid-19 classification (2021). URL: https://arxiv.org/abs/2107.05334.

[34]

R. Miron, C. Moisii, S. Dinu, M. Breaban, Covid detection in chest cts: Improving the baseline on cov19-ct-db (2021). URL: https://arxiv.org/abs/2107.04808.

[35]

J. Hou, J. Xu, R. Feng, Y. Zhang, F. Shan, W. Shi, Cmc-cov19d: Contrastive mixup classification for covid-19 diagnosis, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 454–461.

[36]

R. Turnbull, Cov3d: Detection of the presence and severity of COVID-19 from CT scans using 3D ResNets [Preliminary Preprint] URL: https://melbourne.figshare.com/articles/preprint/Cov3d_Detection_of_the_presence_and_severity_of_COVID-19_from_CT_scans_using_3D_ResNets_Preliminary_Preprint_/20226087.

[37]

J. Hou, J. Xu, R. Feng, Y. Zhang, Fdvts’s solution for 2nd cov19d competition on covid-19 detection and severity analysis (2022). URL: https://arxiv.org/abs/2207.01758.

[38]

C.-C. Hsu, C.-H. Tsai, G.-L. Chen, S.-D. Ma, S.-C. Tai, Spatiotemporal feature learning based on two-step lstm and transformer for ct scans (2022). URL: https://arxiv.org/abs/2207.01579.

[39]

A. Diba, M. Fayyaz, V. Sharma, A.H. Karami, M.M. Arzani, R. Yousefzadeh, L. Van Gool, Temporal 3d convnets: New architecture and transfer learning for video classification (2017). URL: https://arxiv.org/abs/1711.08200.

[40]

W. Zouch, D. Sagga, A. Echtioui, R. Khemakhem, M. Ghorbel, C. Mhiri, A.B. Hamida, Detection of covid-19 from ct and chest x-ray images using deep learning models, Ann. Biomed. Eng. 50 (7) (2022) 825–835,.

Recommendations

Decision making process: typology, intelligence, and optimization

Decision making is concerned with evaluating and/or ranking possible alternatives of action. In this paper, we develop a model for the process of decision making. Understanding the decision process can provide insights into how humans make decisions, ...
Sustainable decision making: the role of decision support systems

Sustainable decision making stands for decision making which contributes to the transition to a sustainable society. It raises a number of challenging problems for which existing decision support systems (DSS) may not be equipped. The role of DSS in ...
Medical Image Analysis of MRI Brain Images by Deep RBF GMDH-type Neural Network Using Principal Component-Regression Analysis
IIAI-AAI '15: Proceedings of the 2015 IIAI 4th International Congress on Advanced Applied Informatics

The deep radial basis function (RBF) Group Method of Data Handling (GMDH)-type neural network is applied to the medical image analysis of magnetic resonance imaging (MRI) brain images. In this deep RBF GMDH-type neural network algorithm, many hidden ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 542, Issue C

Jul 2023

144 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 14 July 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents