A Pragmatic Ensemble Strategy for Missing Values Imputation in Health Records

1.1. Motivation

1.2. Missing Data Classification

1.3. Endeavours to Impute Missing Data

1.4. Importance of Imputing Missing Health Data for Entropy

1.5. Research Contributions

• We introduce a unique Ensemble Strategy for Missing Value to analyse healthcare data with considerable missing values to identify unbiased and accurate prediction statistical modelling. Overall, there are four computational benefits of the suggested model:
  • It can analyse huge amounts of health data with substantial missing values and impute them more correctly than standalone imputation procedures such as the k-nearest neighbour approach, iterative method, and so on.
  • It can discover essential characteristics in a dataset with many missing values.
  • It tackles the performance glitches of developing a single predictor to impute missing values, such as high variance, feature bias, and lack of precision.
  • Fundamentally, it employs an extreme gradient-boosting method, which includes L1 (Lasso Regression) and L2 (Ridge Regression) regularisation to avoid overfitting.
• The current study uses real-world healthcare data (snapshot presented in Figure 1) to conduct experiments and simulations of data with varying feature-wise missing frequencies indicating that the proposed technique surpasses standard missing value imputation approaches.

3.1. Data Pre-Processing

• Initially, the training data i.e., $D^{train}$, does not contain any missing values. Thus, a dataset, i.e., $D^{trai{n}_{mv}}$, is prepared by randomly eliminating the present values from the features present in ${Q}$.
• Three imputation techniques were chosen for the proposed ensemble methodology as unrelated base predictors since using unrelated base predictors may significantly reduce prediction errors in ensemble learning, as indicated in [40]. $D^{trai{n}_{mv}}$data is passed to three imputation methods, i.e., (1) simple mean imputer, (2) KNN imputer, and (3) iterative imputer, that have been chosen as base predictors in current research.
  • Simple mean imputer: Missing values are substituted in this imputer by the mean of all non-missing values in the corresponding parameter.
  • KNN imputer: By assessing respective distance measurements, the KNN method seeks the other k non-missing findings, most comparable to the missing one for every missing value. The missing data is subsequently replaced by a weighted average of the k nearby but non-missing values, with the scores determined by their Euclidean distances from the missing value.
  • Iterative imputer: Multiple copies of the same data are generated and then integrated to get the “finest” predicted value in this approach. The MICE technique has been used to provide iterative imputation based on completely conditional requirements.
• The values predicted to be imputed for the missing data in $D^{trai{n}_{mv}}$ by the base predictors are reserved in three 2-D matrices, i.e., $Pre{d}^1, Pre{d}^2, and Pre{d}^3,$ for simple mean, KNN, and iterative imputer, respectively.
• Corresponding to each attribute index in ${Q}$, a regressor model is trained. For training each $\mathrm{q}ϵ\left\{1,2,\dots ,{Q}\right\}$ regressor models, a corresponding matrix $P_q$ (structure presented in Equation (1)) is provided as input.

3.2. Model Training

3.3. Imputation

• In the first section (variable declaration), all the required datasets and matrices have been initialised.
• In the second section, the algorithm performs two sequential tasks.
  • The first task involves generation of training dataset using three imputation strategies, i.e., simple imputation, kNN imputation, and iterative imputation; after applying imputation method on the training dataset, the resultant dataset is stored in $Pre{d}^1$, $Pre{d}^2$, and $Pre{d}^3$, respectively. Now, for each attribute index present in ${Q}$, a corresponding matrix $P_q$ is formed that comprises of four attributes (simple, kNN, iterative, and actual). The first three attribute elements are represented by vector B denoting the values of qth attribute’s elements imputed by simple imputation, kNN imputation, and iterative imputation method, and the fourth attribute element is represented by vector A, denoting the known value of qth attribute’s elements.
  • The second task involves the training of a regressor model (XGB) using generated training dataset. The vectors B and A are passed into XGBRegressor method for training the model and the trained resultant regressor associated with the qth attribute is represented by reg[q].
• In the third section, the algorithm performs three sequential tasks.
  • The first task involves the preprocessing of the testing dataset as done in previous section and transform testing dataset representation into $P_q{}^{test}$ matrix associated with each missing valued attribute (q). $P_q{}^{test}$ matrix comprises of three attribute elements represented by vector $B^{test}$ denoting the values of qth attribute’s elements imputed by simple imputation, kNN imputation, and iterative imputation methods.

  Algorithm 1 Proposed Ensemble Model

  $\mathcal{D}$: testing dataset, ${Q}$: dataset with imputed instances ${Q}$: indexes of attributes having at least one MV. $D^{train}$: dataset with training instances. $D^{trai{n}_{mv}}$: dataset with training instances having randomly assigned MVs. reg[q]: regressor model associated with qth feature #Generating training Dataset and training the Model $Pre{d}^1=$ SimpleImputer($D^{trai{n}_{mv}}$, strategy = ‘mean’) $Pre{d}^2=$ kNNImputer($D^{trai{n}_{mv}}$, NN = 5) $Pre{d}^3=$IterativeImputer($D^{trai{n}_{mv}}$, max_itr = 5) for qth in ${Q}$: $P_q\left[0\right]=Pre{d}^1\left[q\right], P_q\left[1\right]=Pre{d}^2\left[q\right], P_q\left[2\right]=Pre{d}^1\left[q\right], P_q\left[3\right]=D^{train}\left[q\right]$  $B=(P_q\left[0\right],P_q\left[1\right],P_q\left[2\right]$)  $A=(P_q\left[3\right]$) reg[q] = XGBRegressor() reg[q].fit(B,A) reg[q].predict(B) #Applying trained ensemble models on $\mathcal{D}$ $Pre{d}^{1^{test}}=$ SimpleImputer($D^{trai{n}_{mv}}$, strategy = ‘mean’) $Pre{d}^{2^{test}}=$ kNNImputer($D^{trai{n}_{mv}}$, NN = 5) $Pre{d}^{3^{test}}=$ IterativeImputer($D^{trai{n}_{mv}}$, max_itr = 5) for qth in ${Q}$:  $P_q{}^{test}\left[0\right]=Pre{d}^{1^{test}}\left[q\right], P_q{}^{test}\left[1\right]=Pre{d}^{2^{test}}\left[q\right], P_q{}^{test}\left[2\right]=Pre{d}^{3^{test}}\left[q\right]$  $B^{test}=(P_q{}^{test}\left[0\right],P_q{}^{test}\left[1\right],P_q{}^{test}\left[2\right]$)  $B^{test}$ = $B^{test}$[$\mathcal{D}$[q].isna().index]  ${\mathcal{Y}}_q$= reg[q].predict($B^{test}$)  $i=-1$  for j in $\mathcal{D}$[q]:   if $\mathcal{D}$[q][j] = nan:    $\mathcal{D}$[q][j]= [i++]

  b.

  The second task involves the prediction of missing values in testing dataset using trained regressor models (XGB) reg[q] associated with each missing valued attribute (q). The predicted values are stored in a vector ${\mathcal{Y}}_q$

  c.

  Lastly, the third task involves the substitution of imputed results of missing values associated with qth attribute as stored in ${\mathcal{Y}}_q$ into the actual dataset $\mathcal{D}$. After substitution, the dataset is completed, and no missing value is present in it.

4.1. Real Time Dataset

4.2. Regressor Models

4.3. Evaluation Metrics

4.4. Results

• It has been discovered that primitive imputation strategies, such as iterative, kNN, and simple mean imputation do not perform well when imputing the missing values of huge datasets. When the imputed dataset is submitted to XGB regressor and random forest regressor to assess target values, dropping the records with missing values appears to be highly promising, as demonstrated in Table 5. On the contrary, while making predictions through a support vector regressor, dealing with a large dataset containing comparatively more missing values, dropping the missing values is not recommended. However, when the dataset is small and has fewer missing values, dropping the records holding missing values is the best option, as predicted by all three regression models.
• When working with a small dataset with fewer missing values, all imputation techniques produce similar outcomes when predicted by the SVR Model. On the contrary, in the case of regressor models XGB and RFR, significant variations in the performance of various imputation techniques are observed. The results achieved indicate that the proposed ensemble model outperforms all mentioned primitive imputation techniques when dealing with both large and small datasets by producing the lowest values for mean absolute and mean squared errors. The performance of kNN, iterative, and simple mean imputation to impute missing values individually has been observed to underperform compared to the technique of dropping the records holding missing values. However, the suggested ensemble imputations model outperformed all four scenarios, as validated by the three underlying regression models.