A Deep Learning Approach for COVID-19 $\&$ Viral Pneumonia Screening with X-ray Images A Deep Learning Approach for COVID-19 Viral Pneumonia Screening with X-ray Images

COVID-19, the illness caused by the severe acute respiratory syndrome coronavirus (SARS-CoV-2) enters your body through the mucous membranes of your face, such as your nose, mouth, and eyes by latching its surface proteins to receptors on healthy cells [1]. Gradually, the virus spreads down the respiratory tract, which can cause patients’ lungs to become inflamed and lead to difficulty in breathing. This can eventually develop into pneumonia, an infection of the alveoli inside the lung where the blood exchanges oxygen and carbon dioxide. Pneumonia occurs when parts of the lungs consolidate and collapse, as a result of an accumulation of pus in the infected airways. Patients also suffer from fever and cough due to the reduced surfactant in the alveoli from the viral attack, which diminishes oxygen uptake [2, 3].

If a doctor performs a chest CT scan of a patient with COVID-19 or pneumonia, they will likely see shadows or patchy areas called “ground-glass opacity” and are extremely helpful in the early detection and diagnosis of this disease [4]. However, differentiating between the occurrence of the two diseases can be inherently difficult, which is what inspired a deep learning approach to this problem. Currently, the most common diagnostic test used to identify positive cases of COVID-19 is the reverse-transcription polymerase chain reaction test, or RT-PCR. The procedure works by detecting viral RNA in a person's cells, usually collected from their nose. These tests can be highly inaccurate, as scientists from Johns Hopkins Medicine have shown that as many as 20% of RT-PCR tests may produce false negatives, incorrectly indicating that a patient does not have COVID-19, when they actually do [5]. In fact, RT-PCR tests generally do not perform well outside of ideal conditions, as researchers at the Foundation for Innovative New Diagnostics, a nonprofit research center in Geneva, Switzerland, achieved 100% sensitivity and at least 96% specificity on negative samples in a laboratory setting. However, with the abnormal and irregular testing conditions in the real world, clinical sensitivity of RT-PCR tests ranges from 66% to 80% [6]. On the basis of these findings, scientists state that it is important to exercise caution when interpreting results of RT-PCR tests for SARS-CoV-2, especially if the test took place in the early stages of infection.

Given the widespread presence of radiology imaging systems in modern healthcare systems, employing a deep learning approach to this problem can help increase efficiency in detecting positive cases of the disease. A trained deep learning model can be quickly deployed to detect abnormalities in the X-ray images and help differentiate between occurrences of COVID-19 and Viral Pneumonia. In fact, chest X-rays are often performed as part of a standard procedure for patients with a respiratory complaint, and they can be a great complement to the traditional RT-PCR testing to reach a diagnosis [7]. In the event that chest X-rays are used to screen for a diagnosis, the presence of expert radiologists would be required to interpret the images. However, given that visual indicators of COVID-19 and Viral Pneumonia can be quite elusive (as shown in Figure 1), even to experts, there is still a high chance of a misdiagnosis. Deep learning provides a fast, automated, and effective strategy to eliminate this problem, with an accuracy potentially higher than human capability. The computer learns to find the minuscule features in the X-rays that contribute to the diagnosis after analyzing hundreds of images, a process that can be very time-consuming for a radiologist to manually perform. The proposed model can be found at https://github.com/faizancodes/COVID-19-X-Ray-Classification and can potentially be utilized to screen for COVID-19 and assist physicians in their diagnosis of the patient.

In this research, the X-ray images were acquired from two Kaggle datasets [8, 9] that were created in collaboration with medical doctors to establish an open-source database to help researchers better understand COVID-19. The databases were created by extracting the X-ray images from various public sources such as publications as well as indirect collection from hospitals and physicians [10]. The dataset used in this article contains a total of 1,389 images, which includes 289 COVID-19, 550 Viral Pneumonia, and 550 Normal X-rays.

The proposed model contains five convolutional layers, with each being followed by batch normalization and max pooling layers, along with dropout. Processing the final convolutional layer is a fully connected layer with 512 neurons, followed by the last layer with three neurons representing each category of X-ray. ReLU was used as the activation function for each layer, and softmax was used for the final dense layer. The model architecture is shown in Figure 2 and details for each layer are shown in Table 1.

The number of filters in each convolutional layer was gradually increased from 32 to 64 to 128, then back down to 64 and 32, with the strides for each layer being $3\times 3$. Batch normalization was used to standardize the inputs, and max pooling was used to provide spacial variance, used to account for varying appearances.

The model was initially trained on a dataset containing 200 COVID-19 X-rays, 250 Viral Pneumonia X-rays, and 250 Normal X-rays, with each image being resized to a height and width of 200 by 200 pixels. Eighty percent of the data were used for training and 20% were used for validation, with 560 images being part of the training set and 140 images being part of the validation set. The developed model consists of 453,027 parameters and uses categorical crossentropy for the loss function, rmsprop for the optimizer, and a batch size of 5. The learning rate was initially set to 0.0001, and to prevent the accuracy from plateauing during training, we performed a learning rate reduction by a factor of 0.5 if the validation accuracy does not increase for two steps.

For the initial procedure, the model was tested on a dataset containing 89 COVID-19 X-rays, 250 Viral Pneumonia, and 250 normal X-rays. To further evaluate the model, we performed 5-fold cross-validation on the entire dataset of 1,389 X-ray images, where 80% of the dataset was used for training and 20% percent was used for testing. Of the training set, 20% was used as the validation set, with 889 total images used for training, 223 images used for validation, and 277 images used for testing for each fold of cross-validation. A visual representation of this procedure is shown in Figure 3 and Figure 4.

To better account for variations in the X-ray images, the ImageDataGenerator class in Keras¹ was used to generate batches of image data with real-time augmentation. The rotation, shear, zoom, width shift, and height shift ranges were all adjusted to to account for variations in the images and best train the model.

For the initial procedure, the proposed model achieved an accuracy of 93% on the testing set, with an F1-score of 93%. The confusion matrix of the model performance is shown in Figure 5, which indicates that the model is successfully able to differentiate between the occurrence of the diseases, with high precision and recall for each category of X-ray.

The model performed similarly for each fold of cross-validation, with an average accuracy of 90.64% and F-1 Score of 89.88%, as shown in Table 2 and the confusion matrices in Figure 6.

Using saliency maps, we can better understand the features in the X-rays and visualize what areas of the image are of high importance. To visualize the saliency, the gradient of the output category was computed with respect to the input image. This tells us how the output category value changes with respect to a small change in input image's pixels. All the positive values in the gradients tell us that a small change to that pixel will increase the output value. Visualizing these gradients, which are the same shape as the image, should provide some insight on the most important features of the image [11].

As shown in Figures 7 –9, the areas of yellow gradient have the greatest influence on the model's prediction. These visualizations can be shown to radiologists to confirm their significance, as they help us better interpret the trained model.

From evaluating the COVID-19 saliency maps in Figure 7, it is evident that the areas around the cardiophrenic angle are what the model deems most important, along with areas around the left and right hilum. The areas along the azygo-esophageal recess and right middle thorax are also important in screening for all three categories of X-rays as well, as indicated by their respective saliency maps. These areas can be classified as containing patches of ground-glass opacity, which is critical for early detection of COVID-19 and Viral Pneumonia, hence the absence of the patches in the normal X-rays. Radiologists can analyze these visualizations to better understand each disease and discover new ways to screen for them.

Despite achieving a high accuracy with our deep learning model, there are several factors that prevent it from being adapted in the healthcare industry as of yet. The biggest issue we are faced with is the lack of sufficient data for COVID-19, since it is still a new disease. There are limited datasets available for X-ray scans of COVID-19 patients, and to determine the diagnostic performance of our model, a clinical study would have to be conducted. Due to the small dataset used, there can be hidden biases in the data, especially with no metadata on age, gender, different pathologies also present in the patients, and other information necessary to spot these kinds of biases [12]. It was also found that convolutional neural networks are learning patterns in the X-ray images that are not correlated to the presence of COVID-19 at all, where similar classification performance can be obtained using X-ray images that do not contain most of the lungs [13]. Until COVID-19 data become more widely available and clinical studies are conducted, the proposed model cannot be used solely as a diagnostic measure.

COVID-19 is a new phenomenon for scientists, especially in the AI community. However, some research has been done that focuses on image processing and chest X-ray images. In this section, we explain the current research that has used AI and machine learning techniques for X-rays and chest area cancer detection.

We believe our system will be helpful to detect and determine X-ray images from patients that have similar characteristics to other diseases where symptoms imaging has similar patterns, i.e., COVID-19 and other illness categories. We also expect our system will detect different imaging features that infected with COVID-19 disease or similar disease such as flu ground. Our system especially will be useful to detect early stages of an outbreak and can be used in various government organizations or clinics that use X-ray images.

We also expect, if our system is used in a healthcare organization that is facing a new outbreak, healthcare professionals, doctors, physicians, and admin offices could be alerted due to high accuracy of our model. Our system will assist to provide more hints and signals on outbreak and identifying further image-based indication and classification of infectious images. Using this system also delivers more awareness into patient information system in organization for future usage by prognosis and treatment.

Finally, with the number of positive cases of the disease rising in cities worldwide and limited resources available in hospitals, our proposed deep learning model can potentially be used to screen for COVID-19 and detect instances of the disease much faster and more accurately than the traditional RT-PCR testing method, which can have high rates of false negative results. By detecting and isolating those who are infected with COVID-19 faster, cities can greatly benefit from lower rates of infection and be able to recover their economy as a whole.

After evaluating the model's performance, it is apparent that deep learning can successfully be used to screen for COVID-19 and Viral Pneumonia in chest X-rays. With an average classification accuracy of 90.64% and F-Score of 89.88% after performing 5-fold cross-validation, the model offers an automated, end-to-end solution for testing. It can potentially be used as an alternative or in conjunction with the standard RT-PCR testing methods currently used to increase efficiency and efficacy in detecting positive cases of each disease. In countries and areas where there are limited COVID-19 testing kits, utilizing this deep learning approach for testing can drastically help prevent the spread of the disease. However, due to the shortage of COVID-19 X-rays used, we aim to apply this model on a larger dataset to determine its true efficacy. With a clinical study, the true diagnostic performance of the model can be interpreted to then be used in healthcare facilities around the world.