Abstract
Functional MRI has attracted increasing attention in cognitive neuroscience and clinical mental health research. Towards understanding how brain give rises to mental phenomena, deep learning has been applied to functional MRI (fMRI) dataset to discover the physiological basis of cognitive process. Considering the unsupervised nature of fMRI due to the complex intrinsic brain activities, an encoder-decoder structure is promising to model hidden structure of latent signal sources. Inspired by the success of deep residual learning, we propose a 68-layer 3D residual autoencoder (3D ResAE) to model deep representations of fMRI in this paper. The proposed model is evaluated on the fMRI data under 3 cognitive tasks in Human Connectome Project (HCP). The experimental results have shown that the temporal representations learned by the encoder matches the task design and the spatial representations can be interpreted to be meaningful functional brain networks (FBNs), which not only include tasks based FBNs, but also intrinsic FBNs. The proposed model also outperforms a 3-layer autoencoder, showing the key factor for the performance improvement is depth. Our work demonstrates the feasibility and success of adopting 2D advanced deep residual networks in computer vision into 3D fMRI volume modeling.
Q. Dong and N. Qiang—Equally contribution to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Huettel, S.A., et al.: Functional Magnetic Resonance Imaging, vol. 1. Sinauer Associates, Sunderland (2004)
Smith, S.M., et al.: Correspondence of the brain’s functional architecture during activation and rest. Proc. Natl. Acad. Sci. 106(31), 13040–13045 (2009)
Pessoa, L.: Understanding brain networks and brain organization. Phys. Life Rev. 11(3), 400–435 (2014)
Lv, J., et al.: Task fMRI data analysis based on supervised stochastic coordinate coding. Med. Image Anal. 38, 1–16 (2017)
Archbold, K.H., et al.: Neural activation patterns during working memory tasks and OSA disease severity: preliminary findings. J. Clin. Sleep Med. 5(01), 21–27 (2009)
Barch, D.M., et al.: Function in the human connectome: task-fMRI and individual differences in behavior. Neuroimage 80, 169–189 (2013)
Binder, J.R., et al.: Mapping anterior temporal lobe language areas with fMRI: a multicenter normative study. Neuroimage 54(2), 1465–1475 (2011)
Dosenbach, N.U., et al.: A core system for the implementation of task sets. Neuron 50(5), 799–812 (2006)
Kanwisher, N.: Functional specificity in the human brain: a window into the functional architecture of the mind. Proc. Natl. Acad. Sci. 107(25), 11163–11170 (2010)
McKeown, M.J.: Detection of consistently task-related activations in fMRI data with hybrid independent component analysis. NeuroImage 11(1), 24–35 (2000)
Calhoun, V.D., et al.: A method for making group inferences from functional MRI data using independent component analysis. Hum. Brain Mapp. 14(3), 140–151 (2001)
Beckmann, C.F., et al.: Investigations into resting-state connectivity using independent component analysis. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360(1457), 1001–1013 (2005)
Calhoun, V.D., et al.: Multisubject independent component analysis of fMRI: a decade of intrinsic networks, default mode, and neurodiagnostic discovery. IEEE Rev. Biomed. Eng. 5, 60–73 (2012)
Beckmann, C.F., et al.: General multilevel linear modeling for group analysis in FMRI. Neuroimage 20(2), 1052–1063 (2003)
Jiang, X., et al.: Sparse representation of HCP grayordinate data reveals novel functional architecture of cerebral cortex. Hum. Brain Mapp. 36(12), 5301–5319 (2015)
Lv, J., et al.: Holistic atlases of functional networks and interactions reveal reciprocal organizational architecture of cortical function. IEEE Trans. Biomed. Eng. 62(4), 1120–1131 (2015)
Li, X., et al.: Multple-demand system identification and characterization via sparse representations of fMRI data. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE (2016)
Bengio, Y.: Learning deep architectures for AI. Found. Trends® Mach. Learn. 2(1), 1–127 (2009)
Bengio, Y., et al.: Deep learning. Nature 521(7553), 436–444 (2015)
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)
Yamins, D.L., et al.: Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19(3), 356 (2016)
Hjelm, R.D., et al.: Restricted Boltzmann machines for neuroimaging: an application in identifying intrinsic networks. NeuroImage 96, 245–260 (2014)
Jang, H., et al.: Task-specific feature extraction and classification of fMRI volumes using a deep neural network initialized with a deep belief network: evaluation using sensorimotor tasks. NeuroImage 145, 314–328 (2017)
Dong, Q., et al.: Modeling hierarchical brain networks via volumetric sparse deep belief network (VS-DBN). IEEE Trans. Biomed. Eng. (2019)
Huang, H., et al.: Modeling task fMRI data via mixture of deep expert networks. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE (2018)
Huang, H., et al.: Modeling task fMRI data via deep convolutional autoencoder. IEEE Trans. Med. Imaging 37(7), 1551–1561 (2018)
Zhao, Y., et al.: 4D modeling of fMRI data via spatio-temporal convolutional neural networks (ST-CNN). IEEE Trans. Cogn. Dev. Syst. (2019)
Wang, H., et al.: Recognizing brain states using deep sparse recurrent neural network. IEEE Trans. Med. Imaging 38, 1058–1068 (2018)
Li, Q., et al.: Simultaneous spatial-temporal decomposition of connectome-scale brain networks by deep sparse recurrent auto-encoders. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 579–591. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_45
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Glasser, M.F., et al.: The minimal preprocessing pipelines for the human Connectome project. Neuroimage 80, 105–124 (2013)
Jenkinson, M., et al.: Fsl. Neuroimage 62(2), 782–790 (2012)
Ioffe, S., et al.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015)
Kingma, D.P., et al.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Abraham, A., et al.: Machine learning for neuroimaging with scikit-learn. Front. Neuroinform. 8, 14 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Dong, Q., Qiang, N., Lv, J., Li, X., Liu, T., Li, Q. (2020). Discovering Functional Brain Networks with 3D Residual Autoencoder (ResAE). In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12267. Springer, Cham. https://doi.org/10.1007/978-3-030-59728-3_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-59728-3_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59727-6
Online ISBN: 978-3-030-59728-3
eBook Packages: Computer ScienceComputer Science (R0)