Multimodal manifold-regularized transfer learning for MCI conversion prediction

ADNI database

The data used in the preparation of this paper were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (http://adni.loni.usc.edu/). ADNI researchers collect, validate and utilize data such as MRI and PET images, genetics, cognitive tests, CSF, and blood biomarkers as predictors for Alzheimer's disease. Data from the North American ADNI's study participants, including Alzheimer's disease patients, mild cognitive impairment subjects and elderly controls, are available in this database. The ADNI was launched in 2003 by the National Institute on Aging (NIA), the National Institute of Biomedical Imaging and Bioengineering (NIBIB), the Food and Drug Administration (FDA), private pharmaceutical companies, and non-profit organizations, as a $60 million, 5-year public-private partnership. The primary goal of ADNI has been to test whether the serial MRI, PET, other biological markers, and clinical and neuropsychological assessments can be combined to measure the progression of MCI and early AD. Determination of sensitive and specific markers of very early AD progression is intended to aid researchers and clinicians to develop new treatments and monitor their effectiveness, as well as lessen the time and cost of clinical trials.

The ADNI is the result of efforts of many co-investigators from a broad range of academic institutions and private corporations, and subjects have been recruited from over 50 sites across the U.S. and Canada. The initial goal of ADNI was to recruit 800 adults, aged 55 to 90, to participate in the research approximately 200 cognitively normal older individuals to be followed for 3 years, 400 people with MCI to be followed for 3 years, and 200 people with early AD to be followed for 2 years (see www.adni-info.org for up-to-date information). The research protocol was approved by each local institutional review board, and the written informed consent was obtained from each participant.

Subjects

The ADNI general eligibility criteria are described at www.adni-info.org. Briefly, subjects are between 55 and 90 years of age, and have a study partner able to provide an independent evaluation of functioning. Specific psychoactive medications were excluded. General inclusion/exclusion criteria are as follows: 1) healthy subjects: MMSE scores between 24 and 30, a Clinical Dementia Rating (CDR) of 0, non-depressed, non-MCI, and non-demented; 2) MCI subjects: MMSE scores between 24 and 30, a memory complaint, having objective memory loss measured by education adjusted scores on Wechsler Memory Scale Logical Memory II, a CDR of 0.5, absence of significant levels of impairment in other cognitive domains, essentially preserved activities of daily living, and an absence of dementia; and 3) Mild AD: MMSE scores between 20 and 26, CDR of 0.5 or 1.0, and meets the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association (NINCDS/ADRDA) criteria for probable AD.

MRI, PET, and CSF acquisition

A detailed description of the ADNI data acquisition of MRI, PET, and CSF can be found in (Zhang et al. 2011). Specifically, the structural MR scans were acquired from 1.5T scanners. We downloaded raw Digital Imaging and Communications in Medicine (DICOM) MRI scans from the public ADNI website (www.loni.ucla.edu/ADNI), reviewed for quality, and corrected spatial distortion caused by gradient nonlinearity and B₁ field inhomogeneity. The PET images were acquired 30–60 min post-injection, averaged, spatially aligned, interpolated to a standard voxel size, intensity normalized, and smoothed to a common resolution of 8 mm full width at half maximum. The CSF data were collected in the morning after an overnight fast using a 20- or 24-gauge spinal needle, frozen within 1 h of collection, and transported on dry ice to the ADNI Biomarker Core laboratory at the University of Pennsylvania Medical Center. In this study, we used Aβ₄₂, t-tau, and p-tau as CSF features.

Image pre-processing and feature extraction

All MRI and PET images were pre-processed by first performing an anterior commissure-posterior commissure (AC-PC) correction using the MIPAV software (CIT 2012). The AC-PC corrected images were resampled to 256×256×256, and the N3 algorithm (Sled et al. 1998) was used to correct intensity inhomogeneity. For the MRI images, a skull stripping method (Wang et al. 2011a) was performed, and the skull stripping results were manually reviewed to ensure clean skull and dura removal. The cerebellum was removed by first registering the skull stripped image to a manually-labeled cerebellum template, and then removing all voxels within the labeled cerebellum mask. FAST in FSL (Zhang et al. 2001) was then used to segment the human brain into three different tissues: grey matter (GM), white matter (WM), and cerebro-spinal fluid (CSF). We used HAMMER (Shen and Davatzikos 2002) for registration. After registration, the subject-labeled image was generated based on the Jacob template (Kabani et al. 1998) that dissects a brain into 93 manually labeled ROIs. Then, for each of 93 ROIs, we computed the GM tissue volume in an ROI as a feature. For the PET images, we used a rigid transformation to align them onto their respective MRT1 image of the same subject, and then computed the average intensity of each ROI as a feature. In total, for each subject, we extracted 189 features including 93 MRI features, 93 PET features, and 3 CSF features.

Overview

In Fig. 1, we illustrate the proposed framework for MCI conversion prediction based on our M2TL method. Specifically, our framework consists of three main components, i.e., (1) image pre-processing, (2) M2TL-based sample selection, and (3) M2TL-based classification. As shown in Fig. 1, we first pre-process all MRI and PET images, and extract features from each modality as described in the Image pre-processing and feature extraction section. Then, we select informative samples for building a generalized model via the proposed M2TL method. We finally make a decision using both the sample weights and the modality weights trained in our M2TL method.

Multimodal manifold-regularized transfer learning (M2TL)

Unlike the previous methods that only considered samples of the target domain in model training, in this work, we use samples of different domains as well as unlabeled samples to build a generalized model. Furthermore, we use multimodal samples. Hereafter, we denote M as the number of different modalities with an index m∈{1,…,M} throughout the whole paper. Assume that we have N_A samples with class labels in the auxiliary domain (i.e., AD and NC), denoted as $A_m={\left\{{\mathbf{x}}_{m,a}^A,{\mathrm{y}}_a^A\right\}}_{a=1}^{N_A}$, where ${\mathbf{x}}_{m,a}^A$ is the a-th sample and ${\mathrm{y}}_a^A\in \{+1,-1\}$ is its corresponding class label (e.g., AD as +1 and NC as −1). Also, assume that we have $N_T^L$ labeled samples of the target domain, denoted as $T_m^L={\left\{{\mathbf{x}}_{m,l}^L,{\mathrm{y}}_l^L\right\}}_{l=1}^{N_T^L}$, where ${\mathbf{x}}_{m,l}^L$ is the l-th sample and ${\mathrm{y}}_l^L\in \{+1,-1\}$ is the corresponding class label (e.g., MCI-C as +1 and MCI-NC as −1). Similarly, we have $N_T^U$ unlabeled samples of the target domain, denoted as $T_m^U={\left\{{\mathbf{x}}_{m,n}^U\right\}}_{u=1}^{N_T^U}$. We use $N_T=N_T^L+N_T^U$ to represent the total number of samples in the target domain, i.e., $T_m=\{T_m^L\cup T_m^U\}$. Also, $N=N_A+N_T^L+N_T^U$ is the total number of all samples.

In this work, we use a traditional regularized least square method (Belkin et al. 2006) to design our model for classification, and use all the available data from the auxiliary domain as well as the target domain to build a more generalized model. However, there may be some noise and irrelevant samples in the auxiliary domain as well as in the target domain, especially for the case of using multimodal biomarkers. To remove the noise and irrelevant samples from different modalities consistently, we introduce an L₁/L₂-norm based regularizer on weight matrix (i.e., W_2,1), which can simultaneously remove a common subset of samples relevant to all modalities (Zhang et al. 2012b). In addition, by simultaneously performing sample selection for multimodal data, it is very helpful to suppress noise in the individual modalities. Accordingly, the base model can be written as follows:

Where Y is the label vector and $\mathbf{Y}={\left[{\mathrm{y}}_1^A,\dots ,{\mathrm{y}}_{N_A}^A,{\mathrm{y}}_1^L,\dots ,{\mathrm{y}}_{N_T^L}^L,0_1,\dots ,0_{N_T^U}\right]}^{\prime }$, J is a diagonal matrix with the first $N_{\mathrm{A}}+N_T^L$ diagonal entries to be 1 and the remaining $N_T^U$ diagonal entries to be 0, λ_m is a modality weighting factor, W = [w₁,w₂,…w_M] ∈ R^{N × M} denotes a weight matrix whose i-th row wⁱ is the vector of coefficients associated with the i-th training sample across different modalities, and μ>0 is a sparsity control parameter. The symbol ' denotes the transpose of a matrix. It is worth noting that a `group sparsity' regularization in Eq. (1) is used for joint selection or un-selection of samples across different modalities based on the L₁/L₂-norm, i.e., ${\Vert \mathbf{W}\Vert }_{2,1}={\sum }_{i=1}^N{\Vert {\mathit{w}}^i\Vert }_2$. As for the selection of Y and J, according to the weight matrix W whose elements in some rows are all zeros, we just select those corresponding samples and their labels (Y and J) with non-zero weights. In Eq. (1), K_m is a compound cross-domain kernel matrix over A_m and T_m. In the following, we will introduce how to compute this cross-domain kernel matrix K_m for implementing the knowledge fusion from both auxiliary and target domains (including labeled and unlabeled samples).

Here, the instance-transfer approach (Dai et al. 2007) is used to link the auxiliary domain data to the target domain data. To be specific, we first define the kernel matrices from the auxiliary domain and the target domain as ${\mathit{K}}_m^{A,A}=\left[\mathit{k}\left({\mathbf{x}}_{m,i}^A,{\mathbf{x}}_{m,j}^A\right)\right]\in R^{N_A\times N_A}$ and ${\mathit{K}}_m^{T,T}=\left[\mathit{k}\left({\mathbf{x}}_{m,i}^T,{\mathbf{x}}_{m,j}^T\right)\right]\in R^{N_T\times N_T}$, respectively. Here, ${\mathbf{x}}_{m,i}^A\left({\mathbf{x}}_{m,j}^A\right)$ and ${\mathbf{x}}_{m,i}^T\left({\mathbf{x}}_{m,j}^T\right)$ are samples in the auxiliary and target domains, respectively, N_A and N_T are the numbers of samples in the auxiliary and target domains, respectively. Then, we define the cross-domain kernel matrices from the auxiliary domain to the target domain, and also from the target domain to the auxiliary domain as ${\mathit{K}}_m^{A,T}=\left[\mathit{k}\left({\mathbf{x}}_{m,i}^A,{\mathbf{x}}_{m,j}^T\right)\right]\in R^{N_A\times N_T}$, and ${\mathit{K}}_m^{T,A}=\left[\mathit{k}\left({\mathbf{x}}_{m,i}^T,{\mathbf{x}}_{m,j}^A\right)\right]\in R^{N_T\times N_A}$ respectively. Finally, the cross-domain kernel matrix K_m can be computed as: ${\mathbf{K}}_m=\left[\begin{array}{cc}\hfill {\mathit{K}}_m^{A,A}\hfill & \hfill {\mathit{K}}_m^{A,T}\hfill \\ \hfill {\mathit{K}}_m^{T,A}\hfill & \hfill {\mathit{K}}_m^{T,T}\hfill \end{array}\right]\in {\mathrm{R}}^{N\times N}$, which can be seen as the similarity between pairwise samples in the cross-domain for the m-th modality. In our study, the linear kernel function is used.

Note that the base model in Eq. (1) treats the samples from the auxiliary domain (i.e., AD and NC) and the unlabeled samples equally as the labeled samples from the target domain (i.e., MC-C and MC-NC) with no consideration of their own distributions. However, due to the potential distributional discrepancy between domains, i.e., the target domain (MCI-C and MCI-NC) and the auxiliary domain (AD and NC), the base model would not successfully combine them in learning. To this end, we utilize a maximum mean discrepancy (MMD) criterion (Borgwardt et al. 2006; Duan et al. 2012), which was originally designed to measure whether two sets of data are from the same or different probability distributions. Specifically, we use a kernel-based MMD criterion formulated as follows:

where $\mathbf{S}={\mathbf{ss}}^{\prime },\mathbf{s}={\left[\frac{1}{N_A},\dots ,\frac{1}{N_A},\frac{-1}{N_T},\dots ,\frac{-1}{N_T}\right]}^{\prime }$, tr(·) denotes a trace of a matrix, and the symbol' denotes the transpose of a matrix.

Regarding to the SSL method that utilizes the unlabeled samples to train a classifier, we define a compound cross-domain Laplacian matrix on the m-th modality as follows:

where ${\Lambda }_m^A={\mathit{D}}_m^A-{\mathit{C}}_m^A$ and ${\Lambda }_m^T={\mathit{D}}_m^T-{\mathit{C}}_m^T$ are the Laplacian matrices over the auxiliary domain and the target domain, respectively. Here, ${\mathit{C}}_m^A=\left[c_{\mathit{ij}}^A\right]\in R^{N_A\times N_A}$ and ${\mathit{C}}_m^T=\left[c_{\mathit{ij}}^T\right]\in R^{N_T\times N_T}$ are the similarity matrices for the samples of the auxiliary domain and the samples of the target domain, respectively, and ${\mathit{D}}_m^A=\left[d_{\mathit{ii}}^A\right]\in R^{N_A\times N_A}$ and ${\mathit{D}}_m^T=\left[d_{\mathit{ii}}^T\right]\in R^{N_T\times N_T}$ are the diagonal matrices with elements $d_{ii}^A={\sum }_j{c}_{ij}^A$ and $d_{ii}^T={\sum }_j{c}_{ij}^T$ respectively. In conjunction with the compound cross-domain kernel matrix K_m and the weight coefficient vector w_m, we define a manifold regularization function (Belkin et al. 2006) as follows:

By integrating a kernel-based MMD criterion in Eq. (2) and a manifold regularization function in Eq. (4) into the base model in Eq. (1), we define our objective function as follows:

where γ>0 is a regularization control parameter. We call our method a `multimodal manifold-regularized transfer learning method' (M2TL). In Eq. (5), the first term tr(K<sub>m</sub>S) is the kernel-based MMD criterion, which can help eliminate the potential negative effect introduced by the distributional difference between the auxiliary domain and the target domain. The manifold regularization term R(Λ<sub>m</sub>,K<sub>m</sub>,w<sub>m</sub>)=(K<sub>m</sub>w<sub>m</sub>)′Λ<sub>m</sub>(K<sub>m</sub>w<sub>m</sub>) can capture the geometry of the probability distribution between the labeled and unlabeled data via the compound cross-domain Laplacian matrix Λ<sub>m</sub>. By minimizing Eq. (5), we can learn a converged W among multi-domains, labeled and unlabeled data, and multimodal data. It is worth noting that, because of using `group sparsity', the elements of some rows in the common weight matrix W will be all zeros. For sample selection, we just keep those samples with non-zero weights.

To solve the optimization problem of Eq. (5), we employ an accelerated gradient descent (AGD) method (Chen et al. 2009). To be specific, we decompose the objective function of Eq. (5) into two parts of a smooth term G(W) and a non-smooth term H(W) as follows:

We then define the generalized gradient update rule as follows:

where ▽G(W_t) denotes the gradient of G(W) at the point W_t at the t-th iteration, h is a step size, 〈W–W_t, ▽G(W_t)〉= tr((W–W_t)′▽G(W_t)) is the matrix inner product, and ∥ ∥_Fdenotes a Frobenius norm. According to (Chen et al. 2009), the generalized gradient update rule of Eq. (8) can be further decomposed into N separate sub-problems with a gradient mapping update approach. We summarize the details of AGD algorithm in Algorithm 1.

Sample selection and classification

It is noteworthy that, due to the use of the group sparsity constraint in Eq. (5), after optimization, some row vectors in the optimized weight matrix W have their l₂-norm being close to or equal to zero. This implies that the corresponding samples are less informative for classification. This favorable property allows us to use the proposed M2TL method for sample selection in a data-driven manner, and when making a decision we can only use those selected samples.

We finally build our classifier by performing the proposed M2TL on the selected samples. After learning the optimal weight matrix ${\mathbf{W}}^{\ast }=[{\mathbf{w}}_1^{\ast },\cdots ,{\mathbf{w}}_{\mathrm{M}}^{\ast }]$, given a test sample $\mathbf{x}={\left\{{\mathbf{x}}_m\right\}}_{m=1}^M$, we can then make a decision with the following multi-kernel SVM function f*(x):

where ${\mathbf{K}}_m^{\ast }={\left[\mathit{k}({\mathbf{x}}_m,{\mathbf{x}}_m^i)\right]}_{i=1}^N\in R^{1\times N}$ is the testing sample's kernel vector on the m-th modality (between the testing sample x_m and the i-th selected training sample ${\mathbf{x}}_m^i$ in the cross-domain).

Experimental settings

We used the samples of 202 subjects (51 AD, 43 MCI-C, 56 MCI-NC, and 52 NC), for whom the baseline MRI, PET, and CSF data were all available. Also, for each of the three modalities, we included another set of unlabeled samples from 153 randomly selected subjects. We regarded the samples of 43 MCI-C and 56 MCI-NC subjects as the target domain data and also those of 51 AD and 52 NC subjects as the auxiliary domain data. It is worth noting that, for all 99 MCI subjects (43 MCI-C + 56 MCI-NC), during the 24-month follow-up period, 43 MCI subjects converted to AD and 56 remained stable.

To evaluate the performances of the proposed method as well as the competing methods, we used a 10-fold cross-validation strategy by partitioning the target domain data into training and testing subsets. In particular, 99 MCI samples in our target domain were partitioned into 10 subsets (each subset with a roughly equal size), and then one subset was successively selected as the testing samples and all the remaining subsets were used for training. To avoid the possible bias occurring during sample partitioning, we repeated this process 10 times. We reported the performances in terms of area under the receiver operating characteristic curve (AUC), accuracy (ACC), sensitivity (SEN), and specificity (SPE).

We compared the proposed method with a standard SVM, domain transfer SVM (DTSVM) (Cheng et al. 2012), and manifold-regularized Laplacian SVM (LapSVM) (Belkin et al. 2006). The main difference among these methods lies in how much information they use in learning from the available samples:

• SVM: labeled samples from the target domain;

• DTSVM: labeled samples from both the target and the auxiliary domains;

• LapSVM: both labeled and unlabeled samples from the target domain.

The standard SVM method was implemented using the LIBSVM toolbox (Chang and Lin 2001) with a linear kernel and a default value for the parameter C (i.e., C=1). We used a linear kernel for a Laplacian matrix in both M2TL and LapSVM methods. The optimal model parameters of γ and μ in our M2TL method were chosen from the range of {0.001, 0.01,0.03,0.06,0.09,0.1,0.2,0.4,0.6,0.8} by a nested 10-fold cross-validation on the training data.

In the experiments, both single-modal and multimodal features were used to evaluate the proposed method as well as other methods. A multi-kernel combination technique (Zhang et al. 2011) was adopted for multi-modality fusion. To be specific, the combination weights for multi-kernels were learned within a nested cross-validation via a grid search in the range of 0 and 1 at a step size of 0.1. The optimal parameter λ_m in the proposed M2TL method was determined in the same manner. Before training models, we normalized features by following (Zhang et al. 2011).

Comparison between M2TL and other methods

Here, we first compare the proposed M2TL method without sample selection with the competing methods, with results reported in Table 1. Note that Table 1 shows the averaged results of the 10-fold cross-validation performed on 10 independent experiments. We also presented the ROC curves achieved by different methods in Fig. 2. From Table 1 and Fig. 2, we can see that the proposed M2TL method achieved better performance than DTSVM, LapSVM, and SVM in terms of both accuracy and sensitivity. At the same time, in most cases, the proposed M2TL method outperformed the competing methods in terms of specificity and AUC. Specifically, by using multimodal data, M2TL achieved a classification accuracy of 77.8 %, which significantly outperformed DTSVM (69.4 %), LapSVM (69.1 %), and SVM (63.8). At the same time, by using single modality, the proposed M2TL method usually achieved better performances than DTSVM, LapSVM, and SVM. These results validate the efficacy of our M2TL method, which uses both labeled and unlabeled samples from the auxiliary domain (i.e., AD and NC) and the target domain (i.e., MCI-C and MCI-NC) in MCI conversion prediction.

Comparison between M2TL with sample selection and other methods

To investigate the influence of the proposed sample selection method, we also compare the proposed method without sample selection (M2TL) and with sample selection (M2TL+SS) to LapSVM with Sample Selection (LapSVM+SS), and also DTSVM with Sample Selection (DTSVM+SS). Specifically, for the methods of LapSVM+SS and DTSVM+SS, we first applied our M2TL method for sample selection and then trained the respective LapSVM and DTSVM on the selected samples. It is worth noting that, because labeled samples from the auxiliary and the target domains are more informative than unlabeled samples, we applied the sample selection strategy only for unlabeled samples. The experimental results are shown in Table 2.

From Table 2, we can see that M2TL+SS with multimodal data achieved a classification accuracy of 80.1 %, which is significantly better than M2TL (77.8 %), LapSVM+SS (71.6 %), and DTSVM+SS (71.3 %). With single modality, especially with PET, M2TL+SS still achieved better performance than M2TL, LapSVM+SS, and DTSVM+SS. Recalling the experimental results reported in Table 1, we could say that the proposed M2TL-based sample selection method has the effect of promoting the performance of MCI conversion prediction. These results validate the efficacy of the proposed M2TL-based sample selection.

Furthermore, we investigated the influence of the number samples from the auxiliary domain for M2TL+SS and M2TL by comparing to other transfer learning methods, i.e., DTSVM and DTSVM+SS. We randomly chose samples from the auxiliary domain and then reported the average accuracies in Fig. 3, from which we can see that the proposed M2TL+SS and M2TL consistently outperformed DTSVM+SS and DTSVM. In addition, with the increase of the number of samples from the auxiliary domain, the classification accuracy rises monotonically for M2TL+SS, M2TL, and DTSVM.

Finally, we investigated the influence of the number of unlabeled samples for the proposed M2TL+SS and M2TL methods, in comparison to two SSL methods (i.e., LapSVM+SS and LapSVM). The average accuracies achieved by these four different methods are reported in Fig. 4. Specifically, for M2TL and LapSVM methods, we randomly chose unlabeled samples and then performed M2TL and LapSVM for classification, respectively. On the other hand, for M2TL+SS and LapSVM+SS methods, we first conducted sample selection using M2TL to select samples from unlabeled samples and then performed M2TL and LapSVM for classification, respectively.

As we can see from Fig. 4, regardless of the number of unlabeled samples, the proposed M2TL+SS and M2TL methods outperformed LapSVM+SS and LapSVM in terms of classification accuracy. As the used number of unlabelled subjects changes from 0 to 15, there are obvious improvements in accuracy by using four methods, which explicitly demonstrates that using unlabelled samples can improve classification performance. In addition, Fig. 4 shows that the classification accuracies of M2TL and LapSVM methods (based on random sample selection) rise gradually with the increase of the number of unlabelled samples. On the other hand, for M2TL+SS and LapSVM+SS methods, their corresponding performances are first improved as the number of unlabelled samples increases, and then dropped when too many (e.g., over 75) unlabelled subjects are used. This implies that our proposed M2TL method for sample selection can effectively select informative unlabelled samples and also avoid noisy or irrelevant samples for the underlying classification task.

Use of all available samples in learning

In the field of neuroimaging-based brain disease diagnosis, there have been studies presenting relations among tasks of identifying different stages of disease, e.g., AD vs. NC and MCI-C vs. MCI-NC. Motivated by these studies, in this paper, we proposed a method that could use samples from different domains by means of transfer learning. Specifically, we adopted the AD/NC as an auxiliary domain to help the task of discriminating MCI-C from MCI-NC. From a machine learning point of view, transfer learning aims to apply the knowledge learned from one or more auxiliary domains to a target domain. However, due to the potential difference in distributions of auxiliary domains and the target domain, it is challenging to efficiently use such knowledge in learning. To handle the distributional discrepancy between domains, we used an MMD criterion to measure the similarity or dissimilarity between two sets of samples from different domains.

While it is difficult to get more labeled samples, it is relatively easy to obtain more unlabeled samples in general. In our previous work (Cheng et al. 2012), we used labeled auxiliary-domain samples to foster the generalization of a classifier in a target domain. In this work, we extended it to use unlabeled samples for further performance improvement. To be precise, we proposed a multimodal manifold-regularized transfer learning (M2TL) method for automatic selection of informative samples and also automatic rejection of uninformative samples to make a decision.

As a naïve way for transfer learning, we can apply the model trained on AD and NC to the task of MCI conversion prediction directly (Da et al. 2014; Eskildsen et al. 2013; Filipovych et al. 2011a; Young et al. 2013). We call this kind of method as a direct transfer learning (DTL), or direct semi-supervised transfer learning (DSSTL). In DTL, samples of AD and NC are treated as training data, while samples of MCI-C and MCI-NC are used for testing data; In DSSTL, samples of AD and NC are used as labeled data, while samples of MCI-C and MCI-NC are regarded as unlabeled data. Then, the model is trained based on both labeled and unlabeled data. Finally, samples of MCI-C and MCI-NC are treated as testing data to evaluate the performance of each learned model. In our additional experiment, we compared the proposed M2TL method with DTL and DSSTL using multimodal data (i.e., MRI+CSF+PET). The DTL method achieved a classification accuracy of 66.7 % and AUC of 0.702, and the DSSTL method achieved a classification accuracy of 70.9 % and AUC of 0.766. These results are much worse than those of the proposed M2TL method that produced the maximum classification accuracy of 80.1 % and AUC of 0.852. We believe that these results validated the advantage of the proposed M2TL method over other transfer learning methods.

Besides modalities used in this paper, i.e., MRI, PET, and CSF, there also exist other modalities (e.g., Diffusion Tensor Imaging (DTI) and Resting-state functional Magnetic Resonance Imaging (RS-fMRI)) which can be used for AD/MCI classification (Jie et al. 2014; Supekar et al. 2008; Wang et al. 2007; Wee et al. 2012, 2014; Zhu et al. 2014). It will be interesting to further investigate the incorporation of these modalities into our proposed M2TL model, which will be one of our future works.

M2TL model for sample selection

According to (Wang et al. 2011b), the sparse weight matrix can not only consistently select the informative samples from different modalities, but also indicate the contributions of different modalities and subjects in the prediction. Accordingly, we specifically count the number of non-zero elements from each column weight vector of W. Since we used a 10-fold cross-validation strategy in the experiments, we should count the frequency of each subject selected across all folds and all runs (i.e., a total of 100 times for 10-fold cross-validation with 10 independent runs) on the training set. Then, those subjects with frequency of 100 (i.e., always selected in all folds and all runs) are regarded as stable subjects.

For M2TL+SS, we obtained the number and percentage of stable subjects for different domains on the training set as follows: Auxiliary (90/103=87.38 %), Labeled target (69/90=76.67 %), and Unlabeled target (75/153=49.02 %). It shows that the most selected stable subjects are from the auxiliary domain, followed by the target domain. This observation is reasonable since AD and NC subjects are more separable than MCI-C and MCI-NC subjects, and thus the former is more important than the latter for robust classification.

In addition, we compute the sum of absolute values of each column weight vector, and find that the MRI modality is more important than the PET modality, and also the PET modality is more important than the CSF modality, i.e., w_MRI>w_PET>w_CSF, which is consistent with results in Tables 1 and 2. In our current study, we only report results of excluding subjects (by sample selection methods), and the experimental results show that excluding certain subjects can further improve the classification performance (e.g., as shown in Tables 1 and 2). But we did not report results of excluding any modality since our previous works (Cheng et al. 2013a; Zhang et al. 2011) show that using more modalities often leads to better performance.

Effect of feature selection

To investigate the influence of feature selection on the performances of the proposed methods, we further performed a set of experiments by using an extra feature selection step, i.e., based on t-test statistics (Zhang et al. 2011), before sample selection and classification. Fig. 5 shows the classification accuracies achieved by our M2TL+SS and M2TL methods with the t-test based feature selection, with respect to the different number of selected features. As can be seen from Fig. 5, feature selection can help further improve the classification accuracy compared with the original methods using all features (without feature selection). We expect that the use of more advanced feature selection methods in the future could further improve the performance of our M2TL model.

In the current study, we adopt a linear kernel to compute the kernel matrix, because it has been shown effective for multimodal classification of AD and MCI in our previous works (Zhang et al. 2011, 2012b). In future work, we will investigate using other kernel functions (e.g., Gaussian kernel) for computing a kernel matrix, which may provide more precise similarity measurement of the cross-domain kernel matrix.

Comparison with the state-of-the-art methods

Recently, many groups have focused on predicting the conversion of MCI to AD, i.e., identifying MCI-C and MCI-NC subjects (Cho et al. 2012; Coupé et al. 2012; Cuingnet et al. 2011; Davatzikos et al. 2011; Duchesne and Mouiha 2011; Eskildsen et al. 2013; Hinrichs et al. 2011; Lehmann et al. 2012; Leung et al. 2010; Misra et al. 2009; Wee et al. 2013; Westman et al. 2012; Young et al. 2013; Zhang et al. 2012a, b). In Table 3, we compare the performances of the proposed M2TL method with those of the state-of-the-art methods in terms of accuracy, sensitivity, specificity, and AUC, although some metrics are not available for certain studies. It should be noted that different modalities and different numbers of samples were used for different studies. Nevertheless, we would like to emphasize that, in most of performance measurements, our proposed method achieved better performance than the state-of-the-art methods in MCI conversion prediction.

Limitations

The proposed method is based on multimodal data (e.g., MRI, PET, and CSF) and thus requires each subject to have the complete dataset. Such a requirement prevents the proposed method from utilizing a huge amount of available samples with incomplete data, i.e., missing of one or two modalities. For example, in the ADNI database, many subjects have incomplete data due to unavailability of certain modalities. Hence, only a small number of subjects with complete data were used in our study. It will be our future research work to extend our method to deal samples with incomplete data for further performance improvement.

In addition, in our current study, the proposed M2TL model mainly focused on sample selection and classification rather than feature selection. Therefore, our proposed M2TL model is not able to directly identify the relevant biomarkers (i.e., features). In the future work, we will also extend our M2TL model to include a feature selection step for multimodal biomarkers selection.