Abstract
In this study, we proposed a novel explainable artificial intelligence (XAI) technique to explain massive-training artificial neural networks (MTANNs). Firstly, we optimized the structure of an MTANN to find a compact model that performs equivalently well to the original one. This enables to “condense” functions in a smaller number of hidden units in the network by removing “redundant” units. Then, we applied an unsupervised hierarchical clustering algorithm to the function maps in the hidden layers with the single-linkage method. From the clustering and visualization results, we were able to group the hidden units into those with similar functions together and reveal the behaviors and functions of the trained MTANN models. We applied this XAI technique to explain the MTANN model trained to segment liver tumors in CT. The original MTANN model with 80 hidden units (F1 = 0.6894, Dice = 0.7142) was optimized to the one with nine hidden units (F1 = 0.6918, Dice = 0.7005) with almost equivalent performance. The nine hidden units were clustered into three groups, and we found the following three functions: 1) enhancing liver area, 2) suppressing non-tumor area, and 3) suppressing the liver boundary and false enhancement. The results shed light on the “black-box” problem with deep learning (DL) models; and we demonstrated that our proposed XAI technique was able to make MTANN models “transparent”.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
Castelvecchi, D.: Can we open the black box of AI? Nat. News 538(7623), 20 (2016)
Gunning, D., Aha, D.: DARPA’s explainable artificial intelligence (XAI) program. AI Mag. 40(2), 44–58 (2019)
Adabi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10, 1–46 (2015)
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: Proceedings International Conference Machine Learning, pp. 3145–3153 (2017)
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144 (2016)
Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Suzuki, K., Horiba, I., Sugie, N.: Neural edge enhancer for supervised edge enhancement from noisy images. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1582–1596 (2003)
Suzuki, K., Horiba, I., Sugie, N., et al.: Neural filter with selection of input features and its application to image quality improvement of medical image sequences. IEICE Trans. Inf. Syst. 85(10), 1710–1718 (2002)
Suzuki, K., et al.: Extraction of left ventricular contours from left ventriculograms by means of a neural edge detector. IEEE Trans. Med. Imaging 23(3), 330–339 (2004)
Suzuki, K., Li, F., Sone, S., Doi, K.: Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose CT by use of massive training artificial neural network. IEEE Trans. Med. Imaging 24(9), 1138–1150 (2009)
Suzuki, K., Rockey, D.C., Dachman, A.H.: CT colonography: advanced computer-aided detection scheme utilizing MTANNs for detection of ‘missed’ polyps in a multicenter clinical trial. Med. Phys 37(1), 12–21 (2010)
Weigend, A.: On overfitting and the effective number of hidden units. In: Proceedings of the 1993 Connectionist Models Summer School, vol. 1 (1994)
Suzuki, K., Horiba, I., Sugie, N.: A simple neural network pruning algorithm with application to filter synthesis. Neural Process. Lett 13(1), 43–53 (2001). https://doi.org/10.1023/A:1009639214138
Bar-Joseph, Z., Gifford, D.K., Jaakkola, T.S.: Fast optimal leaf ordering for hierarchical clustering. Bioinformatics 17(1), 22–29 (2001)
Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Mach. Learn. 36, 105–139 (1999). https://doi.org/10.1023/A:1007515423169
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 13(4), 600–612 (2004)
Sato, M., Jin, Z., Suzuki, K.: Semantic segmentation of liver tumor in contrast-enhanced hepatic CT by using deep learning with hessian-based enhancer with small training dataset size. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 34–37 (2021)
Simpson, A.L., Antonelli, M., Bakas, S., et al.: A large annotated medical image dataset for the development and evaluation of segmentation algorithms. ArXiv Prepr. ArXiv190209063 (2019)
Huynh, H.T., Le-Trong, N., Bao, P.T., Oto, A., Suzuki, K.: Fully automated MR liver volumetry using watershed segmentation coupled with active contouring. Int. J. Comput. Assist. Radiol. Surg. 12(2), 235–243 (2017). https://doi.org/10.1007/s11548-016-1498-9
Sato, Y., et al.: Tissue classification based on 3D local intensity structures for volume rendering. IEEE Trans. Vis. Comput. Graph. 6(2), 160–180 (2000)
Jin, Z., Arimura, H., Kakeda, S., Yamashita, F., Sasaki, M., Korogi, Y.: An ellipsoid convex enhancement filter for detection of asymptomatic intracranial aneurysm candidates in CAD frameworks. Med. Phys. 43(2), 951–960 (2016)
Acknowledgment
This paper is based on results obtained from a project commissioned by the New Energy and Industrial Technology Development Organization (NEDO).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jin, Z. et al. (2023). Explaining Massive-Training Artificial Neural Networks in Medical Image Analysis Task Through Visualizing Functions Within the Models. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14221. Springer, Cham. https://doi.org/10.1007/978-3-031-43895-0_67
Download citation
DOI: https://doi.org/10.1007/978-3-031-43895-0_67
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43894-3
Online ISBN: 978-3-031-43895-0
eBook Packages: Computer ScienceComputer Science (R0)