CoV2-Detect-Net: Design of COVID-19 prediction model based on hybrid DE-PSO with SVM using chest X-ray images

2.1. Support vector Machine (SVM)

A supervised classification algorithm SVM [32] is better suited for solving classification problems. In SVM, the main objective is to look for a hyperplane in $n$dimensional space in order to classify the data set. A non-linear function is set as $\theta (\beta )$ where $\beta$ is the training set and $\mu \ast \theta (\beta )+\beta =0$ represents the classifier hyperplane. A decision function is defined as: $f\left(\beta \right)=sign\left(\mu \ast \theta \left(\beta \right)+\sigma \right)$. In this equation, $\mu$represents the orientation and $\sigma$represents the related displacement position. Equation for hyperplane classification can be represented as:

where $h_i$ and ${\beta }_i$ are the training label and training data respectively. For each $i=1,2,\cdots n$ where n represents the training set size, ${\beta }_i\in R^{n}and{h}_i\in \left\{+1,-1\right\}.$ $c$ is capacity constant, ${\epsilon }_i$ is a non-negative variable also known as slack variable. This slack variable is an important attribute that demonstrate the violations of margin by sample ${\beta }_i$. The characteristics of linear and non-linear classifiers are almost same but with a difference of the Lagrange formulation in non-linear. So now the formulation of the Eq. (1) re-represented as:

where ${\alpha }_i$ is the support value, if $0\le {\sigma }_i\le c$, then $e_i$ is represented as the support vector (SV), $\gamma \left({\beta }_i,{\beta }_j\right)=\theta \left({\beta }_i\right)\ast \theta \left({\beta }_j\right)\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{k}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{e}\mathrm{l}\mathrm{f}\mathrm{u}\mathrm{c}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}$ [34].

2.2. Differential evolution algorithm (DE)

DE [22] is a population-based metaheuristic approach which is stochastic in nature. DE has been applied in many domains for solving different optimization problems which are complex in nature like medical image processing [35], [36], Feature selection [37], [31]. DE begins by population initialization as $X_i=\left\{x_{i1},x_{i2},\cdots ,x_{\mathit{ij}},..,x_{\mathit{iD}}\right\}$ where $X$is the candidate solution. This process is followed by mutation, crossover and selection operation as shown below:

2.3. Particle swarm optimization

Kennedy and Eberhart [23] in the year 1995 introduced a novel technique based on swarm behaviour termed as Particle Swarm Optimization (PSO). In this algorithm $X_i=\left(x_{i1},x_{i2},x_{i3}\cdots .x_{\mathit{iD}}\right)$ shows the position and $v_i=v_{i1},v_{i2},\cdots \cdots \cdots ,v_{\mathit{iD}}$ is the velocity for the particle $i$ in search space of dimension $D$. Every particle in search space maintains its local and global best position as $l_{\mathit{best}}$and $g_{\mathit{best}}$respectively. PSO update its position ($X_i)$ and velocity ($v_i)$ for each iteration.

Initially PSO was introduced for solving continuous optimization problems but later as there were multiple discrete optimization problems; like feature selection, PSO was augmented to binary PSO (BPSO) [38]. In BPSO position ($X_i),$ and global and local best $(g_{\mathit{best}}and{l}_{\mathit{best}})$ can only take 0 and 1 values. Therefore, for BPSO the velocity calculation is updated as:

where $a_1$, $a_2$,$b_1$ and $b_2$ are acceleration constants. In a generation $G,$For Particle $i$, $v_i$ represents the velocity in dimension $D$. $\mathit{Iw}$represents the Inertia weight that specifies the effect of previous velocity. $l_{\mathit{best}(i,D)}$ and $g_{\mathit{best}(i,D)}$ represents the local and global best values.

3.1. Feature extraction

In the next step feature extraction is performed on the segmented dataset collected from K-means clustering. Feature extraction is the process of capturing the relevant informative data from an image and build the feature set-in coarse-grained manner. It is like a screening process for identifying useful features. Feature extraction is useful for large dataset as it reduces the data dimensionality by obtaining the new features and remove repeated or redundant data. All the details around these feature types are stated in [39]. Table 2 showcase the procedure for feature extraction. In our model we have extracted following features.

3.2. Computational complexity

In our proposed feature selection approach based on hybrid DEPSO, the computation complexity lies in the selection of mutation and crossover strategy using sigmoid function. Also, on the parameter self-adjusting strategy in each iteration, the computational complexity of selection strategy is represented as $O((N\times D+N\times D+N)\times t_{\mathit{max}})$. The self-adjusting strategy has the complexity of $O\left(N\times M\times logN\times t_{\mathit{max}}\right)$ where, N is the population sizes, D is dimension and $t_{\mathit{max}}$ is the maximum iteration and $M$ is the count of objectives . Therefore, the computational complexity of our feature selection algorithm (DEPSO) is $O\left(\left(N\times D\right)\times t_{\mathit{max}}\right)+O\left(N\times M\right).$

As we have used SVM as a classifier and classifier’s accuracy is dependent on training data and feature set. Therefore, the complexity of SVM classifier during training is $O(n^{2}f+n^3)$ and for prediction $O(n_{s}f)$, where $n$ is the training sample, $f$ is the number of features, $n_s$ is the support vector count. This is due to the fact that during the training, classical SVM classifier needs to evaluate the kernel matrix $M$ represented as $M\left(x_i,y_j\right)$.

3.3. Dataset details

Currently RT-PCR test is performed to detect the COVID-19 infected individual. Lower and upper respiratory specimens like nasal, sputum or nasal aspirate are collected from the person suspected to be infected by COVID-19 by health specialist. But this test has limitations like the sample might malfunction by mistake and this impact the accurate diagnosis. Also, the test process is very time consuming and rate of detection is also very low. Radiological images like chest X-ray images can be used as an alternative for detection of COVID-19 infected individual. Therefore, in this study chest X-ray images available as public dataset are used in our model. We have collected the COVID-19 chest X-ray images from [13], [41]. Pneumonia infected patient’s X-ray images [42] are also used so that our model can be effectively prepared and tested. Total 13,975 X-ray images are available as public dataset. This data has 8066 normal, 5551 Pneumonia and 358 COVID-19 chest X-ray images. The sample X-ray images are shown in Fig. 2 .

3.4. Cluster analysis

In an unsupervised learning, clustering plays a very important role. The aim of clustering is to find the interesting pattern in an unlabeled data and group them together. Various clustering techniques are suggested and proposed by researchers [44]. As there are no absolute criteria for choosing the best clustering technique available and the clustering technique mostly dependent of the problem. From the literature we see that there is a manual approach in which clusters are defined for the dataset. Other approach is known as Overlapping clustering, in this technique data is clustered on fuzzy sets based on some membership value. The other approach is hierarchical clustering technique. This algorithm starts by creating clusters of similar objects and finally each object is grouped into the distinct clusters and final clusters will have similar objects. For this study we have used 2 most widely used clustering techniques as K-means and expectation–maximization (EM) with Gaussian mixture model (GMM-EM) [45] clustering technique. Table 4 shows the accuracy comparison results of K-means and GMM-EM approach. The results show that K-means clustering is able to correctly classify 87.42% of features whereas GMM-EM is able to achieve 85.4532% accuracy. The results show that K-means clustering is showing better results in comparison to GMM-EM and therefore, we have opted k-mean algorithm as a clustering approach in our proposed model.

For calculation of “K” in K-means clustering we have used Elbow method. The basic idea in calculating the value of “K” using elbow method is to run the clustering algorithm on various values from the dataset and then calculate the sum of square error. This square error is the square of the distance between each value. “Euclidian distance” is used to calculate this distance. Every observation of the data values fit to nearest mean value cluster. All these centers choose the clusters as per the density and centroid. As shown in Fig. 4 for value of K = 2 there is an “elbow point”, therefore we have used this value for clustering the image data. Image pre-processing is an important step for better clustering of images, we have applied Watershed transform of gradient magnitude as shown in Fig. 5 . This helps in separating the touching objects from image.

3.5. Performance Evaluation:

We have applied k-fold cross-validation method [46] in order to ensure the reliability of our classifier and strengthen the simulation results. In this approach we have used 10-fold cross-validation by randomly dividing the 10 set of training and testing samples. Each combination is executed independently for calculating the accuracy and other parameters as mentioned below.

To measure the performance of our model, we have considered following parameters.

• •

  Computation time: Total time spent in calculation.

• •

  Classification accuracy:

• •

  $F_{\mathit{score}}$: $F_{\mathit{score}}$ is calculated to measure the test’s accuracy.

where $\mathit{TP},TN,FP,FN$are the correctly classified class, opposite class classified, incorrectly classified and misclassified class. $\mathit{CRandCM}$ are correctness and completeness.

• •

  Friedman’s test: Non-parametric test for statistical analysis of the results.

3.6. Experimental results

For evaluation the performance of our proposed model, the experiment is performed using MATLAB with system configuration as 2.11 GHz, Intel ® Core™ i7-8650U and 16 GB RAM. The parameters used for DEPSO algorithm is mentioned in Table 5 . In the evolution stage setting up the right parameters is very important as performance is dependent on these control parameters, however choosing right parameters is a difficult process. In our approach values are adjusted in real time by monitoring the evolutionary process of each individual. During the process of evolution if any individual stagnates then the values need to be readjusted. In literature there is no guideline to set the values of control parameters, instead the random selection of values by trial and error is considered as the better option [47]. Since we have use DE and PSO algorithm in our model for feature selection and these algorithms are stochastic in nature, so accuracy is calculated by running our program over 100 iteration for 30 runs. To evaluate the performance of our proposed model we have also implemented SVM, DE with SVM (DE-SVM), PSO with SVM(PSO-SVM) classifier. We have also compared our proposed model’s efficiency with 8 recently proposed approaches (detailed mentioned in Table 6 ) in literature for COVID-19 detection.

In the preprocessing step data is split into 3 set as training, validation, and testing. This formation of split data is packaged as training data:80%, validation and testing data as 10% each. This validation data is required to better make the error prone objective function. Now with the separate test data set, all the images are resized to $227\times 227$ pixel and the input data is reshuffled in order to reduce the overfitting’s negative impact. Table 5 shows the distribution of data and classes. All the images are pre-processed as in step 1 and 230 features are extracted from each image. These feature set are put for feature selection process using our hybrid DEPSO algorithm. Finally, the optimized feature set are sent to classifier for classification.

Experimental results shown in Table 7 showcase the proposed algorithm performance with different values of $\alpha$ in fitness function. From the table we can see that in all the cases there is a reduction in features but with $\alpha =0.15,$ our proposed algorithm captures the smaller number of features with better classification accuracy as compared to other values. Our proposed approach obtains 99.35 of average accuracy in 40 runs and best accuracy of 100%. These results also show the impact of relevance and redundancy on the performance of classier. Therefore, it is important to choose the correct value of $\alpha$.

We have applied k-fold cross-validation method in order to ensure the reliability of our classifier and strengthen the simulation results. For our experiment we have used k as 10 and so 10 sets of equal sized training and test data samples are created. Further 9 sets are used for training and 1 is used for testing the model. We have repeated this process 10 times by applying data set on both training and testing stage. The average testing results are calculated with training and classification time and shown in Table 8 . Training time is the time spent in training a classifier on 10-fold cross-validation. Classification time is the time spent in predicting the Covid-19 positive and negative cases. The results show that DESVM is able to optimize the feature set and selects 116 features, PSOSVM selects the 112 optimized feature set and our proposed algorithm is able to optimize maximum and select 106 feature set. These optimized feature results also show the impact on classifier’s accuracy. We have also calculated the computational complexity of our algorithm. As we can see from the table the total time for training is 11 sec which is less as compared with DESVM and PSOSVM classifiers but more than SVM classifier. Similarly, the classification time complexity is also<1 s. The results show that our proposed hybrid approach working fast as compared to other evolutionary variants but slow in comparison to SVM classifier. This is due to feature selection step added in our framework and features subset selection from larger extracted feature set takes time but will results in fine tuning of classifier by reducing the complexity .Results also indicate that our feature selection approach is able to get more optimized feature set. This shows that adding the feature selection approach only improves the computational complexity of our classifier. These results show that our proposed algorithm is giving better performance as compared to other classifier in predication of COVID-19 positive and negative cases.

We have also compared our proposed approach performance with 8 recently proposed models. Experimental results are shown in Table 9 . From the results, we can see that the model proposed by Chowdhury et al. [17] is giving better results in terms of accuracy and correctness in comparison to other state of art algorithms. Almost similar performance is shown by the diagnosis model proposed by Ucar et al. [18]. If we compare both these models with our proposed approach, the proposed model with DEPSOSVM is showing better results with 99.34% of accuracy as compared with all the other approaches. These results show our proposed model is giving more accurate results as compared to other models.

Experimental results show that our proposed model is showing superior performance then other approaches. There are two reasons for the same. In our framework the initial step is to pre-process the images by applying K-means unsupervised learning algorithm which is best suited for creating a self-learning model. The second reason is the feature selection step. We have added a novel feature selection approach in which differential evolution algorithm is hybridized with PSO. This hybridization helps swarm to acquire population diversity and avoiding local optima convergence. This is due to our mutation strategy that helps algorithm to maintain the population diversity and to improve the local and global search ability by utilizing DE mutation strategy at initial and PSO strategy at the later stage. DE helps in adding population diversity and PSO enhances the convergence and accuracy. Following this process, optimized feature set are collected which are then put into SVM classifier to gaining better classification accuracy particularly on the larger population and complex solution set. In our approach, DE has been used to maintain the constant parameter value for every evolution, but individual parameter can dynamically get the optimized performance in each iteration. Every dataset is different and complex so need a dynamic strategy to choose the parameter for each iteration and this will improve the overall efficiency of the algorithm. Therefore, in our approach, parameters are self-adjusted based of the status. For example, if any individual parameter becomes stationary in consecutive generations, our model will readjust them by random values as:

where $F_{i,G},F_{\mathit{min}}and{F}_{\mathit{max}}$ are the scaling factor and min and max constant values of scaling factor respectively. ${\mathit{CR}}_{i,G},{\mathit{CR}}_{\mathit{max}}and{\mathit{CR}}_{\mathit{min}}$ represents the crossover rate, max and min constant values of crossover rate for $X_{i,G+1}$ in generation $G$. rand (1, D) is the random values in range [1, D} and D is the size (samples, 2). The min and max values for scaling factor are 0.8 and 1.0 and crossover rate are 0.8 and 1.0. The impact of these control parameters is crucial and therefore the values are chosen by trial and error.

For handling the population diversity and premature convergence problem, we have used a novel mutation operation to produce child generation followed by crossover with particles. These generated offspring have their position updated. Throughout the iteration these new offspring update their own best position in comparison to other generated offspring.

As shown in Fig. 6 , we have created the confusion matrix of our proposed framework in testing. In the figure, 1 represent the COVID data, 2 for the normal and 3 is for the Pneumonia data. Our proposed model is able to detect all the COVID-19 infected chest X-ray images of the patients. Out of 459 samples, there is only 3 cases where misclassification happened. If we check for the error rate, it is 0.007% for normal x-ray images and 0.01% for pneumonia cases. This shows the robustness and sustainability of our model. We have also captured the results using box plot as shown in Fig. 7 for good understanding.

We have also performed non-parametric Friedman’s test for evaluating the statistical performance of our proposed model. The null hypothesis is set as: For the same dataset, results Q is tested for the provided p value $\left(Z_{Q-1}^2>T\right)$, where T is test statistics [48]. For testing performance metrics is applied on accuracy, fitness and $F_{\mathit{score}}.$ Test statistics results and ranks for various approaches are shown in Table 10, Table 11 respectively. For this experiment, confidence level ($\alpha$) is taken as 0.5. From Table 10 we can observe the difference between various approaches are significant and so this null hypothesis is rejected. Ranking order of various approaches are given in Table 11. Lower the value, higher is the ranking and so our proposed model achieves highest ranking among all other algorithms. Therefore, statistical results also confirm the superiority of our proposed model.

Friedman’s non-parametric test results shows that there is a significant difference between the comparative algorithm, but we are not able to establish the interrelation between the algorithms in comparison. García et al. [49] and Derrac et al. [50] has suggested a post-hoc analysis for $N\times N$ comparison of multiple algorithms. Their studies suggested a non-parametric test should be conducted to determine whether there is any significant difference among the algorithms in comparison. Once this is established that there is a significant difference among the algorithms then post-hoc analysis is performed to detect which pair of algorithms are significantly different.

Post-hoc test are usually performed to control the type-1 error also known as family wise error rate (FWER) [51]. In a pairwise multiple comparison tests, FWER refers to the probability of rejecting at least one null hypothesis. With the Friedman test, the Bonferroni test is usually a suggested approach which adjust the p-values and controls the FWER [52]. But this approach is very conservative and there are modifications made to the original process [53], [54]. We have used Holm’s test to determine which of these algorithms are different in $N\times N$ comparison. This Holm’s test is also known as modified Bonferroni process and works on sequentially rejective step down approach for calculating the adjusted p-value [53]. This post-hoc analysis is based on the significant value of $\alpha .$ For this experiment we have used$\alpha as.05.$

We have applied post-hoc analysis to further strengthen the statistical analysis and to showcase that there is a significant difference between the proposed algorithm and rest of the algorithms in comparison. We have used Bonferroni-Holm process for this analysis. Table 12 shows the results from post-hoc analysis that includes Z-score, p-value (unadjusted), Holm’s coefficient adjustment and adjusted p-value. The results are compared with proposed model and other 4 algorithms. From the results it is evident that proposed model shows the significant difference in performance in comparison to other algorithms. Therefore, post-hoc analysis results also confirms the superiority of our model.