Abstract
Sepsis is a life-threatening condition which may arise from an extreme response of the body to an infection. While worldwide figures are unknown, it is estimated that sepsis causes millions of yearly fatalities. In this context, it is important to develop tools for decision support and training of healthcare professionals. This paper proposes that artificial intelligence tools be used for prognosis of septic patients. The model used is a neural network trained and validated with cross-validation. The information used includes data regarding patient history and treatment. More importantly, this paper also presents a principled approach of using sensitivity analysis for the identification of discriminatory variables when these are of mixed types such as binary, categorical, and integer. While initial results, validated on over 5000 patients, already show both specificity and sensitivity above 80% and good model robustness against errors in most inputs, even better performance is attained through the utilization of sensitivity analysis to select the variables used as inputs. This work presents a promising tool for input selection in contexts of limited data availability, and successfully applies this technique to obtain a high-performance model for prognosis of septic patients.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abadi, M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D., Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F., Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems Software available from tensorflow.org.
Bouch, DC, Thompson JP. Severity scoring systems in the critically ill. Continuing Education in Anaesthesia, Critical Care & Pain 2008;8:181–85.
Chicco, D, Jurman G. 2020. The advantages of the Matthews Correlation Coefficient (MCC) over F1 score and accuracy in binary classification evaluation BMC Genomics 21.
Dietterich, TG. Approximate statistical test for comparing supervised classification learning algorithms. Neural Comput 1998;10:1895–23.
Fawcett, T. An introduction to ROC analysis. Pattern Recogn Lett 2006;27:861–74.
Flanagan, JR, Pittet D, Li N, Thievent B, Suter P, Wenzel R. Predicting survival of patients with sepsis by use of regression and neural network models. Clinical Performance and Quality Health Care 1996;4:96–103.
Fleischmann, C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, Angus DC, Reinhart K. Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations. Am J Respir Crit Care Med 2016;193:259–72.
Gevrey, M, Dimopoulos I, Lek S. Review and comparison of methods to study the contribution of variables in artificial neural network models,. Ecol Model 2003;160:249–64.
Innocenti, F, Tozzi C, Donnini C, De Villa E, Conti A, Zanobetti M, Pini R. SOFA score in septic patients: incremental prognostic value over age, comorbidities, and parameters of sepsis severity. Intern Emerg Med 2018;13:405–12.
Jaimes, F, Farbiarz J, Alvarez D, Martínez C. Comparison between logistic regression and neural networks to predict death in patients with suspected sepsis in the emergency room. Critical Care 2005;9: R150.
Kam, HJ, Kim HY. Learning representations for the early detection of sepsis with deep neural networks. Comput Biol Med 2017;89:248–55.
Kibe, S, Adams K, Barlow G. Diagnostic and prognostic biomarkers of sepsis in critical care,. J Antimicrob Chemother 2011;66:ii33–40.
Kingma, D, Ba J. 2014. Adam: A method for stochastic optimization International Conference on Learning Representations.
Liu, X, Shen Y, Li Z, Fei A, Wang H, Ge Q, Pan S. Prognostic significance of APACHE II score and plasma supar in Chinese patients with sepsis: a prospective observational study. BMC Anesthesiol 2015; 16:46.
Machado, FR, Cavalcanti AB, Bozza FA, Ferreira EM, Carrara FSA, Sousa JL, Caixeta N, Salomao R, Angus DC, Azevedo LCP, et al. The epidemiology of sepsis in Brazilian intensive care units (the Sepsis PREvalence Assessment Database, SPREAD): an observational study. The Lancet Infectious Diseases 2017;17:1180–89.
Martin, GS. Sepsis, severe sepsis and septic shock: changes in incidence, pathogens and outcomes,. Expert Rev Anti-Infect Ther 2012;10:701–06.
McLymont, N, Glover GW. 2016. Scoring systems for the characterization of sepsis and associated outcomes Annals of Translational Medicine 4.
Morris, MD. Factorial sampling plans for preliminary computational experiments. Technometrics 1991;33:161–74.
Nachimuthu, SK, Haug PJ. Early detection of sepsis in the emergency department using dynamic Bayesian networks. AMIA Annual Symposium Proceedingsinfo American Medical Informatics Association 653; 2012.
Nguyen, HB, Van Ginkel C, Batech M, Banta J, Corbett SW. Comparison of predisposition, insult/infection, response, and organ dysfunction, acute physiology and chronic health evaluation ii, and mortality in emergency department sepsis in patients meeting criteria for early goal-directed therapy and the severe sepsis resuscitation bundle. Am J Crit Care 2012;27:362–69.
Okuyama, IF. 2015. Neural Networks For Sepsis Prediction 005/2015 Instituto Tecnológico de Aeronáutica, São José dos Campos, Brazil.
Pedregosa, F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M., Édouard Duchesnay. Scikit-learn: Machine learning in Python. J Mach Learn Res 2011; 12:2825–30.
Pentoś, K. The methods of extracting the contribution of variables in artificial neural network models–comparison of inherent instability. Comput Electron Agric 2016;127:141–146.
Polikar, R, Upda L, Upda S, Honavar V. Learn++: An incremental learning algorithm for supervised neural networks. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 2001;31:497–508.
Rathour, S, Kumar S, Hadda V, Bhalla A, Sharma N, Varma S. PIRO concept: staging of sepsis. J Postgrad Med 2015;61:235.
Refaeilzadeh, P, Tang L, Liu H. Cross-validation,. Encyclopedia of Database Systems Springer pp. 532–538; 2009.
Ruiz, GO, Castell CD. Epidemiology of severe infections in Latin American intensive care units. Revista Brasileira de Terapia Intensiva 2016;28:261–63.
Sadaka, F, EthmaneAbouElMaali C, Cytron MA, Fowler K, Javaux VM, O’Brien J. Predicting mortality of patients with sepsis: A comparison of APACHE II and APACHE III scoring systems,. Journal of Clinical Medicine Research 2017;9:907.
Schuh, CJ. Sepsis and septic shock analysis using neural networks. Fuzzy Information Processing Society, NAFIPS’07 Annual Meeting of the North American IEEE, pp. 650–654; 2007.
Singer, M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche J-D, Coopersmith CM, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). Jama 2016;315:801–10.
Taylor, RA, Pare JR, Venkatesh AK, Mowafi H, Melnick ER, Fleischman W, MHall K. Prediction of in-hospital mortality in emergency department patients with sepsis: A local big data–driven, machine learning approach,. Acad Emerg Med 2016;23:269–78.
Tu, JV. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 1996;49:1225–31.
Zurada, JM, Malinowski A, Cloete I. Sensitivity analysis for minimization of input data dimension for feedforward neural network. Circuits and Systems, ISCAS’94., 1994 IEEE International Symposium on 6 IEEE pp. 447–450; 1994.
de Alencar Saraiva, JL, Júnior M, Júnior O, Kadirkamanathan V, Silva E, Kienitz K. Sepsis patient outcome prediction using machine learning. Costa-Felix R., Machado J., Alvarenga A. (eds) XXVI Brazilian Congress on Biomedical Engineering. IFMBE Proceedings, vol 70/1. Springer, pp. 795–799; 2019.
Acknowledgments
Support from FAPESP (Brazil), as well as from from CNPq (Brazil), is gratefully acknowledged. We also thank Marcus Victor Henrique Júnior and Igor Franzoni Okuyama for support and valuable discussions in early stages of this research.
Funding
This study was funded by Fapesp (grants number 2017/11272-2 and 2017/25497-6) and by CNPq (306900/2018-1).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Author José Lucas de Alencar Saraiva declares that he has no conflict of interest. Author Otávio Monteiro Becker Júnior declares that he has no conflict of interest. Author Eliezer Silva declares that he has no conflict of interest. Author Visakan Kadirkamanathan declares that he has no conflict of interest. Author Karl Heinz Kienitz declares that he has no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Appendix
Appendix
This appendix presents the mean effects of the inputs which were ranked with the highest relevance, the mean effect having been defined in “Sensitivity analysis”.
Ranking | Input | Mean |
|---|---|---|
Effect | ||
1 | Length of hospital stay | 19% |
2 | Length of ICU stay | 13% |
3 | Days of mechanical ventilation | 4.6% |
on ICU | ||
4 | Age | 3.5% |
5 | Country | 3.2% |
6 | AIDS | 1.7% |
7 | Patient moved intermediate | 1.7% |
8 | Other disabled condition | 1.6% |
9 | Other immunosuppression | 1.5% |
condition | ||
10 | Utilization of albumin | 1.2% |
11 | Days of vasopressor on ICU | 1.0% |
12 | Apache Score | 0.9% |
13 | Serious trauma | 0.9% |
14 | Sedation days on ICU | 0.9% |
15 | Mechanical ventilation days | 0.9% |
16 | Utilization of antibiotics | 0.8% |
17 | Location prior ICU | 0.8% |
18 | Enteral treatment | 0.8% |
19 | Days of vasopressor | 0.8% |
20 | Radiotherapy | 0.7% |
21 | Xigris (drotrecogin alfa) utilization | 0.7% |
22 | Severe sepsis diagnosis | 0.7% |
in internation | ||
23 | Tachpynea | 0.6% |
24 | Renal replacement utilization | 0.6% |
25 | Mechanical ventilation utilization | 0.6% |
26 | Renal failure | 0.6% |
27 | Sedation days | 0.6% |
28 | Other typical presentation condition | 0.5% |
29 | Days of PRNTRL medication | 0.5% |
on ICU | ||
30 | Respiratory failure | 0.5% |
31 | Fluid resuscitation | 0.5% |
32 | Days of renal replacement | 0.5% |
33 | Strong severe sepsis suspicion | 0.5% |
34 | Hepatic failure | 0.5% |
35 | Days of unfractionated mechanical | 0.5% |
heart valve | ||
36 | Vasopressor utilization | 0.5% |
37 | Platelet transfusion | 0.4% |
38 | Days of albumin on ICU | 0.4% |
39 | Site of infection | 0.4% |
40 | Chronic steroids | 0.4% |
41 | Gamma globulin treatment | 0.4% |
42 | Days of Xigris on ICU | 0.4% |
43 | Days of enteral treatment on ICU | 0.4% |
44 | Days of low-dose steroids | 0.4% |
45 | Clinical status | 0.4% |
46 | Neurological failure | 0.4% |
47 | Tachycardia | 0.3% |
48 | Fluid resuscitation days on ICU | 0.3% |
49 | Renal replacement days | 0.3% |
50 | Low-dose steroids utilization | 0.3% |
51 | Renal replacement days on ICU | 0.3% |
52 | Cardiovascular failure | 0.2% |
t53 | Utilization of low molecular | 0.2% |
weight heparin | ||
54 | Chemotherapy | 0.2% |
55 | Days of platelet transfusion on ICU | 0.2% |
56 | Utilization of unfractionated heparin | 0.2% |
57 | Liver disease | 0.2% |
58 | High dose steroids utilization | 0.2% |
59 | Beginning of sepsis | 0.2% |
60 | Utilization of sedatives | 0.2% |
61 | Days of enteral treatment | 0.2% |
62 | Active cancer | 0.2% |
63 | Type of ICU | 0.2% |
64 | Chronic lung disease | 0.2% |
65 | Heart disease | 0.1% |
66 | Days of unfractionated heparin on ICU | 0.1% |
67 | Severe sepsis primary diagnosis | 0.1% |
68 | Days of fluid resuscitation on ICU | 0.1% |
69 | Surgical drainage | 0.1% |
70 | Utilization of PRNTRL medication | 0.1% |
71 | Days of high dose steroids | 0.1% |
72 | SIRS immune dysregulation | 0.1% |
73 | Days of nitric oxide | 0.1% |
74 | Days of low-dose steroids on ICU | 0.1% |
75 | Utilization of antithrombin | 0.1% |
76 | Days of Xigris (drotrecogin alfa) | 0.1% |
on ICU | ||
77 | Days of nitric oxide on ICU | 0.1% |
78 | Chronic renal insufficiency | 0.1% |
79 | Hematological failure | 0.1% |
80 | Other surgical procedure | 0.1% |
81 | Days of high-dose steroids on ICU | 0.0% |
82 | Type of infection | 0.0% |
83 | Removal of obstruction | 0.0% |
84 | Days of antithrombin treatment | 0.0% |
85 | Days of antithrombin treatment on ICU | 0.0% |
86 | Gender | 0.0% |
87 | Proven severe sepsis | 0.0% |
88 | Days of gamma globulin treatment | 0.0% |
89 | Utilization of nitric oxide | 0.0% |
90 | HIV | 0.0% |
91 | Leukocytosis | 0.0% |
92 | Usage of unfractionated mechanical | − 0.1% |
heart valve | ||
93 | Days of low molecular weight heparin | − 0.1% |
on ICU | ||
94 | Days of unfractionated heparin | − 0.1% |
95 | Metabolic failure | − 0.1% |
96 | Days of albumin | − 0.1% |
97 | Days of unfractionated mechanical heart | − 0.1% |
valve on ICU | ||
98 | Days of PRNTRL medication | − 0.1% |
99 | Diabetes | − 0.1% |
100 | Days of gamma globulin on ICU | − 0.1% |
101 | Days of low molecular weight heparin | − 0.2% |
102 | Origin of infection | − 0.2% |
Rights and permissions
About this article
Cite this article
de Alencar Saraiva, J.L., Becker, O.M., Silva, E. et al. Sensitivity analysis–based sepsis prognosis using artificial intelligence. Res. Biomed. Eng. 36, 449–461 (2020). https://doi.org/10.1007/s42600-020-00083-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42600-020-00083-7