Abstract
Predicting the evolution of Alzheimer’s disease (AD) is important for accurate diagnosis and the development of personalized treatments. However, learning a predictive model is challenging since it is difficult to obtain a large amount of data that includes changes over a long period of time. Conditional Generative Adversarial Networks (cGAN) may be an effective way to generate images that match specific conditions, but they are impractical to generate 3D images due to memory resource limitations. To address this issue, we propose a novel cGAN that is capable of synthesizing MR images at different stages of AD (i.e., normal, mild cognitive impairment, and AD). The proposed method consists of a 2D generator that synthesizes an image according to a condition with the help of 2D and 3D discriminators that evaluate how realistic the synthetic image is. We optimize both the 2D GAN loss and the 3D GAN loss to determine whether multiple consecutive 2D images generated in a mini-batch have real or fake appearance in 3D space. The proposed method can generate smooth and natural 3D images at different conditions by using a single network without large memory requirements. Experimental results show that the proposed method can generate better quality 3D MR images than 2D or 3D cGAN and can also boost the classification performance when the synthesized images are used to train a classification model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ben-Cohen, A., Klang, E., Raskin, S.P., Amitai, M.M., Greenspan, H.: Virtual PET images from CT data using deep convolutional networks: initial results. In: Tsaftaris, S.A., Gooya, A., Frangi, A.F., Prince, J.L. (eds.) SASHIMI 2017. LNCS, vol. 10557, pp. 49–57. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68127-6_6
Choi, H., Kang, H., Lee, D.S., The Alzheimer’s Disease Neuroimaging Initiative: Predicting aging of brain metabolic topography using variational autoencoder. Front. Aging Neurosci. 10, 212 (2018). https://doi.org/10.3389/fnagi.2018.00212
Choi, Y., Choi, M.J., Kim, M., Ha, J.W., Kim, S., Choo, J.: StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. CoRR abs/1711.09020 (2017). http://dblp.uni-trier.de/db/journals/corr/corr1711.html
Goodfellow, I., et al.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27, 2672–2680 (2014)
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of Wasserstein GANs. In: NIPS, pp. 5767–5777 (2017). http://dblp.uni-trier.de/db/conf/nips/nips2017.html
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: NIPS, December 2017
Jack, C., et al.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imaging 27(4), 685–691 (2008). https://doi.org/10.1002/jmri.21049
Jung, E., Chikontwe, P., Zong, X., Lin, W., Shen, D., Park, S.: Enhancement of perivascular spaces using densely connected deep convolutional neural network. IEEE Access 7(8), 18382–18391 (2019). https://doi.org/10.1109/ACCESS.2019.2896911
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference for Learning Representations (2014)
Lei, Y., et al.: MRi-only based synthetic CT generation using dense cycle consistent generative adversarial networks. Med. Phys. 46(8), 3565–3581 (2019). https://doi.org/10.1002/mp.13617. https://aapm.onlinelibrary.wiley.com/doi/abs/10.1002/mp.13617
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
Sohail, M., Riaz, M.N., Wu, J., Long, C., Li, S.: Unpaired multi-contrast MR image synthesis using generative adversarial networks. In: Burgos, N., Gooya, A., Svoboda, D. (eds.) SASHIMI 2019. LNCS, vol. 11827, pp. 22–31. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32778-1_3
Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier GANs. In: International Conference on Machine Learning (2017). https://arxiv.org/abs/1610.09585
Pan, Y., Liu, M., Lian, C., Zhou, T., Xia, Y., Shen, D.: Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 455–463. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_52
Prokopenko, D., Stadelmann, J., Schulz, H., Renisch, S., Dylov, D.: Synthetic CT generation from MRI using improved DualGAN. arXiv:1909.08942, September 2019
Pumarola, A., Agudo, A., Martinez, A.M., Sanfeliu, A., Moreno-Noguer, F.: GANimation: one-shot anatomically consistent facial animation. Int. J. Comput. Vis. 128(3), 698–713 (2019). https://doi.org/10.1007/s11263-019-01210-3
Romero, A., Arbelaez, P., Van Gool, L., Timofte, R.: SMIT: stochastic multi-label image-to-image translation. In: 2019 IEEE International Conference on Computer Vision (ICCV), December 2018
Dar, S.U.H., Yurt, M., Karacan, L., Erdem, A., Erdem, E., Çukur, T.: Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Trans. Med. Imaging 38(10), 2375–2388 (2019)
Shmelkov, K., Schmid, C., Alahari, K.: How good is my GAN? In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11206, pp. 218–234. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01216-8_14
Wegmayr, V., Horold, M., Buhmann, J.: Generative aging of brain MRI for early prediction of MCI-AD conversion. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1042–1046, April 2019. https://doi.org/10.1109/ISBI.2019.8759394
Welander, P., Karlsson, S., Eklund, A.: Generative adversarial networks for image-to-image translation on multi-contrast MR images - a comparison of CycleGAN and UNIT. arXiv preprint arXiv:1806.07777, June 2018
Wolterink, J.M., Dinkla, A.M., Savenije, M.H.F., Seevinck, P.R., van den Berg, C.A.T., Išgum, I.: Deep MR to CT synthesis using unpaired data. In: Tsaftaris, S.A., Gooya, A., Frangi, A.F., Prince, J.L. (eds.) SASHIMI 2017. LNCS, vol. 10557, pp. 14–23. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68127-6_2
Zhao, Q., Adeli, E., Honnorat, N., Leng, T., Pohl, K.M.: Variational AutoEncoder for regression: application to brain aging analysis. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 823–831. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_91
Zhu, J., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017). https://doi.org/10.1109/ICCV.2017.244
Acknowledgement
This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIT) (No. 2019R1C1C1008727) and the MSIT (Ministry of Science, ICT), Korea, under the High-Potential Individuals Global Training Program (2019-0-01557) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Jung, E., Luna, M., Park, S.H. (2020). Conditional Generative Adversarial Network for Predicting 3D Medical Images Affected by Alzheimer’s Diseases. In: Rekik, I., Adeli, E., Park, S.H., Valdés Hernández, M.d.C. (eds) Predictive Intelligence in Medicine. PRIME 2020. Lecture Notes in Computer Science(), vol 12329. Springer, Cham. https://doi.org/10.1007/978-3-030-59354-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-59354-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59353-7
Online ISBN: 978-3-030-59354-4
eBook Packages: Computer ScienceComputer Science (R0)