A systematic review on cough sound analysis for Covid-19 diagnosis and screening: is my cough sound COVID-19?

Studies on non laboratory confirmed datasets

In Table 1, we summarize work in accordance with how features were extracted (shallow learning or deep learning). In other words, we consider handcrafted features (with typical machine learning classifiers) and deep learning algorithms to detect cough sounds in COVID-19 patients. Regardless of how samples were collected, we start with popular features, such as MFCCs, typical machine learning classifiers, and end with deep features. Table 1 follows the order of the description (provided below).

• Handcrafted features (shallow learning): MFCCs contributed a lot in audio/speech processing. In Dash et al. (2021), the authors used the MFCCs of speech signals. From the Coswara dataset, designated database-1, the authors used 570 participants, and each participant contributed nine audio files to various categories of samples (3,470 are clean, 1,055 are noisy, and the rest are highly degraded sound samples). They did not, however, mention if a laboratory has verified the data. Next, the authors used a crowdsourced database designated database-2, with 6,631 users; 235 users were declared COVID-19 positive. The authors suitably optimized the frequency range and the conversion scale of the recordings. The authors used adaptive synthetic sampling approach for imbalanced learning to create synthetic data. In their experiments, they reported accuracy of 0.86 (0.74) for database-2 (database-1) cough sounds.

In a similar fashion, Mouawad, Dubnov & Dubnov (2021) analyzed symbolic recurrence quantification measures derived from MFCC features to detect COVID-19 cases using cough sounds and speech. They used recurrence dynamics and variable Markov model on sustained vowel ‘ah’ recordings and showed that their model is robust for detecting the disease in sustained vowel utterances. The dataset was composed of 1,927 cough sound samples with only 32 sick patients and 1,488 speech records with only 20 sick patients. The authors ensured that the classification model was not biased towards the majority class and experimented with a wide range of data sampling techniques such as oversampling the minority class or under-sampling the majority class. They reported the following test results: an accuracy of 0.91 and an AUC of 0.84.

In addition to handcrafted features, other studies highlight the use of typical machine learning classifiers, such as Neural Networks (NNs), Support Vector Machine (SVM), Random Forest (RF), and Logistic Regression (LR). In Shimon et al. (2021), the authors used multiple different shallow learning techniques like SVM and RF. Their dataset was from non-publicly available crowdsourced data that took multiple types of recordings from people over multiple days. The type of recording used in this article was both cough sounds and the vowel ‘a’. In total, 1,296 cough recordings were used, and 428 “a” recordings were used. The experimenters used multiple different techniques to extract the audio feature, including the openSMILE toolkit and Librosa. After the classifiers classified each audio sample, the experimenters used simple majority voting to classify the patient as COVID-19 infected or non-COVID-19 infected. SVM showed the best result of the different classifiers with an accuracy of 0.78 and 0.74, an AUC of 0.64 and 0.60, a sensitivity of 0.95 and 0.90, and a specificity of 0.36 and 0.35 for ‘a’ and cough, respectively.

In Despotovic et al. (2021), the authors used wavelet scattering features and deep audio embeddings with multiple different shallow learning techniques. Their dataset used crowdsourced data with 1,103 participants where each produced two to five samples each. From these samples, the authors created a balanced dataset with a total of 496 cough samples. They used five-fold leave one out cross-validation for their experimental set-up. Boosting with the wavelet scattering features seemed to have the best overall performance with a 0.88 accuracy, a 0.87 sensitivity, and a 0.89 specificity. In Melek (2021), the authors used multiple different shallow learning architectures to classify cough as COVID-19 Positive or Negative. Among the different architectures are Polynomial-SVM, Linear-LDA, and Euclidean-kNN. They used a composition of two different datasets, 121 from Virufy and 59 from NoCoCoDa. Once combined, the dataset consisted of 107 positive samples and 73 negative samples. Unfortunately, the paper did not specify how they split their data, which limits the reproducibility of this model and limits interpretation of the results. Euclidean-kNN achieved their best results on a single cough with a 0.98 accuracy, 1.0 Specificity, 0.97 Sensitivity, and 0.99 AUC.

• Deep features (deep learning):

The authors also introduced a Deep Neural Network (DNN) to analyze cough sounds. In Nessiem et al. (2021), the authors explore the usage of deep learning models as a ubiquitous, low-cost, pre-testing method for detecting COVID-19 from audio recordings of breathing or coughing taken with mobile devices or via the web. First, they collected 1,427 audio files from a crowdsourced database. Then, they adapted an ensemble of three Convolutional Neural Networks that utilize breathing and coughing raw audio, spectrograms, and Mel-spectrograms to analyze if a speaker is infected with COVID-19 or not. In their experiments, they reported an unweighted average recall/sensitivity of 0.75, AUC of 0.81, and accuracy of 0.73 by ensembling NNs.

In Gokcen et al. (2021), the authors used a “Deep neural network” for the classification of COVID-19 infected individuals. However, the paper did not give an excellent description of their neural network, and from what was gathered, they used some dense net with neurons. However, it is clear that the paper had a sample size of 822 coughs from the MIT open dataset. This seemingly small neural network produced the following results: an accuracy of 0.79 and a recall of 0.75.

In Feng et al. (2021), the authors used an rNN to classify COVID-19 infected individuals from non-infected individuals. They used two different datasets, Coswara for the one they split into validation and training sets, the next is a small Virufy dataset for its final test set. Coswara consisted of 1,433 participants, which they used a 0.80, 0.20 split. The small Virufy test set consisted of 16 recordings from seven patients. Their model had an encouraging result for the training and validation set; however, when they tried it on the Virufy dataset, they could only get around a 0.81 accuracy with an AUC of 0.79.

In Coppock et al. (2021), the authors used a deep neural network on a crowdsourced dataset. This study used a “CIdeR” Convolutional Neural Network (CNN) which is based on the ResNet architecture to help classify in four tasks: (a) COVID positive and stratum healthy-no-symptoms (62 vs 245 subjects); (b) COVID-positive with COVID cough and stratum healthy-with-cough (23 vs 30 subjects); (c) COVID-positive with COVID cough and stratum asthma-with-cough (23 vs 19 subjects); and (d) COVID positive and COVID negative (62 vs 293 subjects). In total, this dataset consisted of 517 samples from 355 participants, and each sample was chopped up into segments, then used a voting mechanism with a mean average across the segments to break ties. Their respective AUCs are: (a) 0.83, (b) 0.57, (c) 0.91, and (d) 0.85.

In Lella & Pja (2021), the authors extended the work of Brown et al. (2020) (using the exact same dataset) using a Deep CNN with multi-feature channels and data augmentation. In contrast to previous work, their results were improved. It was, however, not mentioned whether the improvement was due to data augmentation, the use of the DCNN classifier, or the increase in the dataset’s size. These authors are also ambiguous in declaring whether their data is entirely non-laboratory confirmed, as they mention that users have been admitted ‘into the clinic.’ To classify between COVID-19 positive and negative cases, an accuracy of 0.95 was reported. No other metrics such as sensitivity and specificity were used. In Pahar et al. (2021), they used multiple different machine learning techniques on a crowdsourced publicly available dataset called the ‘Coswara’ dataset. It was composed of 92 (1,079) COVID-19 positive samples (negative samples). The authors generated synthetic COVID-19 samples to build dataset balanced. Of many classifiers (e.g., LR, LSTM, CNN and ResNet50), ResNet50 achieved the best-reported accuracy (0.95). However, throughout the entire reported results, sensitivity remained in the lower nineties (percentage-wise). Later, the researchers also tested their model on a clinically validated dataset called Sarcos after training on the Coswara dataset. Their best performing model was a combination of LTSM + SFS which had an accuracy of 0.9291, AUC of 0.938, sensitivity of 0.91, and a specificity of 0.96.

In addition, the authors have found that biomarkers are important features. In Laguarta, Hueto & Subirana (2020), their tool helped extract and leverage acoustic biomarker features to help pre-screen COVID-19 recordings. The dataset was composed of 475 laboratory-confirmed COVID-19 infected patients, 962 doctor assessments, and 1,223 personal assessments. This study used under-sampling to pull coughs from a group of ‘hundreds of thousands’ of non-infected COVID-19 people to balance the dataset with 224 official tests, 523 personal assessments, and 1,913 personal assessments. However, this study does not thoroughly explain what a ‘personal assessment’ is. However, this study does explain that a diagnostic test must have been complete within seven days of of collection of the sample, with symptom onset no later than 20 days. In total, 4,256 cough samples were used for training and 1,064 for validation. The CNN incorporates multiple biomarker feature models. These biomarkers included muscular degradation, vocal cords, sentiment, and lung and respiratory tract. They achieved a forced cough COVID-19 screening accuracy of 0.97, an AUC of 0.97, a sensitivity of 0.98, and a specificity of 0.94. Furthermore, asymptomatic patients achieved a sensitivity of 1.0 with 0.83 specificity. This work admits that both the inclusion of doctor assessments and personal assessments hurts the generalizability of their model, and are conducting future tests. Since the results are high, their methods of biomarker use should be further explored. In Jayachitra et al. (2021), the authors used a deep neural network architecture. The paper proposed a RNN that would fuse each mode of input’s predictions and add the symptoms to the mix of fusions. The dataset used a mixture of audio and radiological pictures to get a yes or no for COVID-19 infection. After gathering many datasets, their cough sample size consisted of 153 positive samples and 348 negative samples. Again, artificial data was used in their sample size. Their testing procedure did not include a separate test set, so there is a potential bias toward the dataset with the .80 .20 split. However, they report that their multi-modal dataset scored 1.0 across the board on all metrics with just cough at 0.97 accuracy, 0.97 precision, and 0.94 recall. As the model is still progress, it may not be generalized well to the population.

Studies on laboratory confirmed datasets

In Table 2, we observe the use of CNN and/or DNN as well as ensemble learning to detect COVID-19 in the cough sounds of human subjects. Based on the authors’ reports, we categorize their studies with laboratory confirmed datasets. In what follows, we discuss CNN-based works as well as a new paradigm of physiological impact.

CNN has been popularly used to develop AI-guided tools across many different computer vision tasks, and COVID-19 screening using cough sounds is no exception. In Imran et al. (2020), the app called ‘AI4COVID-19’ recorded three seconds of sound and provided results in less than 2 min. This app used a CNN to identify cough sounds. In their first test, they employed Deep Transfer Learning-based Multi-Class classifier (DTL-MC) CNN with multiple output classifiers for COVID-19 and three other diseases. The second used a Classical Machine Learning-based Multi-Class classifier (CML-MC) to judge whether the first one suffered from over-fitting issue. The last test employed a Deep Transfer Learning-based Binary Class classifier (DTL-BC) (similar to the first one), but it was limited to a binary output (yes/no) for a possible COVID-19 infection. For training, they used 1,838 cough sounds and 3,597 non-cough environmental sounds, and for testing, they used 96 bronchitis, 130 pertusses, 70 COVID-19, and 247 normal cough samples. The authors did not mention whether they were laboratory-confirmed, but they referred to the people that provided the samples as “patients,” so we assume these samples were collected in a hospital/clinical environment, and thus are laboratory confirmed. The DTL-BC classifier in this study provided the best overall accuracy at 0.93 in differentiating COVID-19 from non-COVID-19 cough sounds and had a sensitivity of 0.96, and specificity of 0.91.

Another form of NN called Recurrent Neural Network (RNN) was used in the literature. In Pinkas et al. (2020), the authors collected self-recordings from phones (vocal utterances, speech and cough sound), and used RNN to produce specialized sub-models for the SARS-CoV-2 classification. From 29 laboratory-confirmed COVID-19 patients and 59 negative control subjects, 235 samples were used for training the model and 57 samples for testing. An ensemble stacking fused the predictions of the sub-models and pre-training, bootstrapping and regularization techniques were used to prevent over-fitting. They reported an accuracy of 0.79 (with a corresponding sensitivity of 0.79) based on leave-one-out validation protocol.

Cough sounds may not be sufficient to completely analyze COVID-19 positive cases, and physiological impact is another important data type that could help better analyze. In Lonini et al. (2021), the authors introduced a novel paradigm based on recording the physiological responses elicited by a short sequence of 2-min activities (physical activity, cardio-respiratory function and cough sounds). While validating the data, they employed a novel body-conforming soft wearable sensor placed on the suprasternal notch to capture physical activity data and cardio-respiratory function. Combining these features on snapshots from 19 COVID-19 positive and 14 healthy cases provided an AUC of 0.94 as compared to 0.64 (with only forced cough sounds).