Skip to main content

Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost

Abstract

Background

Sepsis is a significant cause of mortality in-hospital, especially in ICU patients. Early prediction of sepsis is essential, as prompt and appropriate treatment can improve survival outcomes. Machine learning methods are flexible prediction algorithms with potential advantages over conventional regression and scoring system. The aims of this study were to develop a machine learning approach using XGboost to predict the 30-days mortality for MIMIC-III Patients with sepsis-3 and to determine whether such model performs better than traditional prediction models.

Methods

Using the MIMIC-III v1.4, we identified patients with sepsis-3. The data was split into two groups based on death or survival within 30 days and variables, selected based on clinical significance and availability by stepwise analysis, were displayed and compared between groups. Three predictive models including conventional logistic regression model, SAPS-II score prediction model and XGBoost algorithm model were constructed by R software. Then, the performances of the three models were tested and compared by AUCs of the receiver operating characteristic curves and decision curve analysis. At last, nomogram and clinical impact curve were used to validate the model.

Results

A total of 4559 sepsis-3 patients are included in the study, in which, 889 patients were death and 3670 survival within 30 days, respectively. According to the results of AUCs (0.819 [95% CI 0.800–0.838], 0.797 [95% CI 0.781–0.813] and 0.857 [95% CI 0.839–0.876]) and decision curve analysis for the three models, the XGboost model performs best. The risk nomogram and clinical impact curve verify that the XGboost model possesses significant predictive value.

Conclusions

Using machine learning technique by XGboost, more significant prediction model can be built. This XGboost model may prove clinically useful and assist clinicians in tailoring precise management and therapy for the patients with sepsis-3.

Background

Sepsis is a common and economically significant disease which has become an important public health issue globally and led to over 5.3 million people dies annually with an approximately overall mortality of 30%, particularly in the intensive care unit (ICU) [1,2,3]. Sepsis is defined as a syndrome of physiologic, pathologic, and biochemical abnormalities induced by infection which results in life-threatening organ dysfunction caused by dysregulated host response [3]. Different from those previous diagnostic criteria for sepsis, sepsis-3 highlighted the strong association between infection and organ failure according to the Third International Consensus Definitions for Sepsis and Septic Shock in February 2016 [2], hence, the early identification and diagnosis for sepsis are essential, which could provide meaningful information for clinicians to assess patients' condition and improve survival outcomes through prompt and appropriate treatment. Due to the complex of vague sepsis syndrome definitions, unknown sources of infection and higher mortality, it is necessary to establish a reliable and effective prognostic model for sepsis. With the help of these prognostic models, strong evidences for clinical decision-making and rational allocation of public health care resources can be provided.

The establishment of prognosis model for sepsis patients has always been a hot topic in critical care medicine. Some sensitive serum markers, such as Ang-2, PCT, interleukin-6, pentraxin 3, etc. [1, 4, 5], have been widely used to facilitate sepsis prognosis, however, their prognostic values are limited, not only rarely available but often lack of sensitivity or specificity. On the other hand, traditional prediction models based on small sample data such as logistic regression analysis and scoring systems including acute physiology and chronic health evaluation-II (APHACHE-II), Simplified acute physiology score-II (SAPS-II) and etc. [6,7,8], are still providing comprehensively clinical importance of identifying patients who are at risk of unfavourable prognostic outcomes, but these methods and scores require the statistical assumption of the independent and linear relationship between explanatory and outcome variables or preclude the analysis of a large number of valuable variables. In addition, insufficient prognostic strength, large fluctuation range, poor stability and operability, tedious process, and other shortcomings exist in these predictive serum markers, models and scores to a certain extent.

Recently, novel machine learning techniques have demonstrated improved predictive performance compared to traditional prediction methods. Moreover, the evolution of statistical theory, computer technology and the establishment of specialized database for critical care medical such as MIMIC-III could help machine learning get more attention and recognition by clinicians. eXtreme Gradient Boosting (XGBoost) is a machine learning technique with the remarkable features of processing the missing data efficiently and flexibly and assembling weak prediction models to build a accurate one [9]. As an open source package, XGBoost has been widely recognized in a number of machine learning and data mining challenges, for example, 17 solutions used XGBoost among the 29 challenge winning solutions published at Kaggle’s blog in 2015 and the top-10 winning teams used XGBoost in KDD Cup 2015 [10].

Therefore, the goal of the study was twofold: firstly, we attempted to compare the performance of machine learning (XGboost) model with traditional prediction models (conventional logistic regression model and SAPS-II score model) in the prediction of the 30-days mortality in MIMIC-IIIpatients with sepsis-3. Secondly, we planned to plot nomogram and clinical impact curve (CIC) to validate the XGboost model.

Methods

Database

We used the Medical Information Mart for Intensive Care III database version 1.4 (MIMIC III v1.4) for the study. MIMIC-III, a publicly available single-center critical care database which was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center (BIDMC, Boston, MA, USA) and the Massachusetts Institute of Technology (MIT, Cambridge, MA, USA), includes information on 46,520 patients who were admitted to various ICUs of BIDMC in Boston, Massachusetts from 2001 to 2012 [11,12,13]. The database contains charted events such as demographics, vital signs, laboratory tests, fluid balance and vital status; documents International Classification of Diseases and Ninth Revision (ICD-9) codes; records hourly physiologic data from bedside monitors validated by ICU nurses; and stores written evaluations of radiologic films by specialists covering in the corresponding time period. The use of the data in the database, provided by clinicians, data scientists, and information technology personnel and unidentified health information of patients, has been deemed not human subjects research and there was no requirement for individual patient consent because of the unidentified health information [12, 13]. The users, whereas, must pass a test to qualify to register for the database and be approved by MIMIC-III database administration staff. After passing a training course “Protecting Human Research Participants” on the website of National Institutes of Health (NIH), an author (NZ Hou) was approved to extract data from this database for research purposes (certification number: 37258322).

Study population

Adult patients who were diagnosed with sepsis-3 were included in our study. The inclusion criteria were: (I) patients who were older than 18 years old; (II) length of stay in the ICU was over 24 h to ensure sufficient data for analysis; (III) patients with the diagnosed of sepsis according to The Third International Consensus Definitions for Sepsis and Septic Shock (sepsis-3) [2]. Because MIMIC-III database has shifted their date of birth to obscure their age, we excluded patients who were over 89 years, and if a patient had multiple admissions with sepsis, only the first admission was analyzed. As it is common with missing data in the MIMIC-III database, we also removed the variables with more than 20% observations missing to facilitate and ensure the accuracy of the review. However, for those with less than 20% missing data or randomly missing data, we explored and visualized them with Templ’s method (R Package “VIM”) [14] and multiple imputation method (R Package “mice”) [15] for further analysis respectively.

Data extraction

We obtained the raw data about patients who were diagnosed with “sepsis”, “severe sepsis” and “septic shock” on discharge using pgAdmin PostgreSQL tools (version 1.22.1) and Navicat Premium (version 12.0.28). After that, R software (version 3.4.3, CRAN) was used for further process. The code, supporting the MIMIC-III documentation and generating the descriptive statistic, is publicly available and contributions from the community of users are encouraged (https://github.com/MIT-LCP). The detailed process of data extraction is shown in Fig. 1. Following demographic data were extracted: age, gender, ethnicity, weight, height and body mass index (BMI), length of stay in hospital, length of stay in the ICU, hospital expire flag (in-hospital death recorded in the hospital database) at the first ICU admission. Then, we collected vital signs of the patients from the first 24 h of ICU stay, including heart rate (HR), systolic blood pressure (SBP), diastolic blood pressure (DBP), mean arterial pressure (MAP), temperature (TEMP), respiratory rate (RR) and oxyhemoglobin saturation (SpO2). Afterwards, laboratory values, such as blood routine examination, liver and kidney function, blood glucose, and arterial blood gas (ABG) were abstracted. Furthermore, advanced cardiac life support (mechanical ventilation, renal replacement therapy, etc.) and accompanied diseases (diabetes, malignant tumour, etc.) were accessed. Because of the high sampling frequency, we use the maximum, minimum and the mean value when incorporating the characteristics of vital signs and related laboratory indicators. Ultimately, we obtained the list of anonymized patients with sepsis from the Table 1 (Additional file 1).

Fig. 1
figure1

The detailed process of data extraction

Table 1 Baseline characteristics, vital signs, laboratory parameters and statistic results of mimic-III patients with sepsis

Statistical analysis

Patients were divided into two groups based on whether death or alive within 30 days and variables were displayed and compared between groups. We revealed and excluded these confounders of the independent risk factors, then, performed correlation analysis to determine the impact of them on 30-days mortality. Normally and non-normally distributed continuous variables were summarized as the mean ± SD and the median respectively. Continuous variables of normal distribution were tested by Kolmogorov–Smirnov test. Student’s t test, One-way ANOVA, Mann–Whitney U or Kruskal–Wallis H test were used to compare continuous data of non-normally distribution, if appropriate. Categorical variables were expressed as numbers or percentage and assessed using Chi-square test or Fisher’s exact test according to different sample sizes as proper.

In the model-development phase, we constructed three predictive models: conventional logistic regression model, SAPS-II score model and XGBoost algorithm model. Firstly, the conventional logistic regression model was conducted using these significant variables identified by backward stepwise analysis with Chi-square test. Then we chose an entry probability of < 0.05 by the stepwise selection method. Secondly, in the construction of SAPS II model, we used these time-stamp variables to do prediction based on the methods provided by the original literature of SAPS II [16]. Thirdly, we performed XGBoost model [17, 18] to analysis the contribution (gain) of each variable to 30-days mortality, at the same time, backward stepwise analysis was processed to select the variable with a threshold of p < 0.05 according to the Akaike information criterion (AIC) [19]. After identifying the variables through XGBoost, we used these clinical and laboratory variables included to construct the XGBoost algorithm model. In the model-comparison phase, we tested and compared the performances of the three predictive models by area under curves (AUCs) of the receiver operating characteristic curves (ROC) and decision curve analysis (DCA), then, selected the model that achieved the highest overall diagnostic value for further verification. At last, nomogram and clinical impact curve (CIC) were plotted to evaluate the clinical usefulness and applicability net benefits of the model with the best diagnostic value. All the analyses above were conducted using R software, and p value < 0.05 was defined as statistically significant.

Results

Baseline characteristics

A total of 4559 sepsis-3 patients are included in our study, in which, 889 patients were death and 3670 survival within 30 days, respectively. In these patients of death, the age, ethnicity, admission type, heartrate_mean, sysbp_min, diasbp_mean, meanbp, meanbp_min, resprate_mean, tempc_min/max, spo2_mean, aniongap (AG)_min/max, creatinine_min, hemoglobin_min/max, lactate_min, potassium_min, sodium_max, bun (blood urea nitrogen)_min/max, wbc (white blood cell)_min/max/mean, INR (international normalized ratio)_max/mean, urine output, score system, comorbidity and common sources of infection differ significantly compared these of survived, however, the sex, heart rate_min, chloride_min, platelet_min, sodium_min and advanced life support show no significant difference between the two groups. Figure 1 is a flow chart describing the procedure for subjects selection; Table 1 is a summary concluding the comparisons of the baseline characteristics, vital signs, laboratory parameters between the non-survivors and the survivors within 30 days, and the overall ethnicity characteristics/the common sources of infection are listed in Fig. 2.

Fig. 2
figure2

Characteristics of MIMIC-III patients with sepsis by ethnicities (a) and characteristics of MIMIC-III patients with sepsis by common sources of infection (b)

Features selected in models

As shown in Table 2, the most important features, which were identified by the results of backward stepwise analysis and strongly associated with mortality in 30 days, were applied in conventional logistic regression model, all of which with p value < 0.05. Moreover, according to the analysis results of each features’ contribution by XGBoost model (Table 3 and Fig. 3), urine output, lactate, Bun, sysbp, INR, age, cancer, SpO2, sodium, AG, and creatinine were the top 11 most important features of the data set and these variables are also included to construct XGBoost predictive models in our study.

Table 2 Features selected in the conventional logistic regression
Table 3 Features selected in the XGboost model
Fig. 3
figure3

Top 11 features selected using XGBoost and the corresponding variable importance score. X-axis indicates the importance score which is the relative number of a variable that is used to distribute the data, Y-axis indicates the top 11 weighted variables

Model comparisons

In the model-development and validation phase, the three models (traditional logistic regression model, SAPS-II score model and XGBoost algorithm model) showed good discriminatory power with AUCs of 0.819 (95% CI 0.800–0.838), 0.797 (95% CI 0.781–0.813), and 0.857 (95% CI 0.839–0.876), respectively (Fig. 4). The XGBoost algorithm model showed the largest test AUC but the traditional logistic regression model was the smallest. According to the DCA of the three prediction model, the net benefit for XGboost model was larger over the range of traditional logistic model and SAPS-II score model, which means XGboost model is the optimal and the SAPS-II score model inferior (Fig. 5).

Fig. 4
figure4

The receiver operating characteristic (ROC) curves. a traditional logistic regression model, area under curves (AUC) is 0.819 [95% confidence interval (CI); 0.800–0.838]; b SAPS-II score model, AUC is 0.797 [0.781–0.813]; c XGboost model, AUC is 0.857 [0.839–0.876], the best performance of the models was the XGboost model

Fig. 5
figure5

Decision curve analysis (DCA) of the three prediction models. The net benefit curves for the three prognostic models are shown. X-axis indicates the threshold probability for critical care outcome and Y-axis indicates the net benefit. Solid green line = XGboost model, solid red line = traditional logistic model, solid blue line = SAPS-II score mode. The preferred model is the XGboost model, the net benefit of which was larger over the range of traditional logistic model and SAPS-II score model

Optimal model analysis

For visualization of the XGboost predictive model, the risk nomogram that integrated 11 selected variables for the incidence of mortality within 30 days is shown in Fig. 6. Clinical impact curve (CIC) analysis was performed in Fig. 7 to evaluate clinical applicability of risk prediction nomogram. CIC visually showed that the nomogram had a superior overall net benefit within the wide and practical ranges of threshold probabilities and impacted patient outcomes, which indicates that the XGboost model possesses significant predictive value.

Fig. 6
figure6

Nomogram to estimate the risk of mortality in sepsis patients. To use the nomogram, we first draw a line from each parameter value to the score axis for the score, the points for all the parameters are then added, finally, a line from the total score axis is drawn to determine the risk of mortality on the lower line of the nomogram

Fig. 7
figure7

Clinical impact curve (CIC) of XGboost model. The red curve (number of high-risk individuals) indicates the number of people who are classified as positive (high risk) by the model at each threshold probability; the blue curve (number of high-risk individuals with outcome) is the number of true positives at each threshold probability. CIC visually indicated that nomogram conferred high clinical net benefit and confirmed the clinical value of the XGboost model

Discussion

Sepsis, which is associated with profound mortality and substantial economic burden, is no longer defined simply as serious infection. In a systematic review and meta-analysis, Reinhart et al. [20] concluded that the mortality rate estimate of ICU- and hospital-treated sepsis patients were 41.9% and 26.7% respectively, or one out of four sepsis patients did not survive their hospital stay. Torio et al. [21] estimated sepsis accounted for 6.2% of the aggregate costs for all hospitalizations, or 23.7 billion USD in 2011. Furthermore, Moss et al. [22] conducted a study spanning two decades (from 1979 to 2000), which reported the annual increase of sepsis cases was around 8.7%. The improvement of sepsis prevention, recognition, and treatment has been a global health priority since the declaration repeatedly by the World Health Organization (WHO) in 2017 [23]. Progressive exacerbation of sepsis can lead to organ failure and death, but early aggressive therapy also forestalls further progression and rescues a decompensating patient. Unfortunately, in ICU it is very difficult for clinicians to predict which patients will respond favorably and could be out of the crisis or will deteriorate despite all interventions and resuscitative efforts. At present, these findings indicate the urgent need to increase efforts to promote reliable prediction models to identify patients with sepsis who are at increased risk of developing organ dysfunction and to prognosticate their mortality.

In this present study, the AUCs and DCAs we developed have demonstrated the benefit of using a XGboost model- as opposed to the classic logistic regression analysis and traditional SAPS II scoring system for early prediction of probability of septic mortality. Moreover, CIC and nomogram were plotted to evaluate the clinical usefulness and applicability net benefits of the model with the best diagnostic value. Logistic regression analysis as one of the classic regression analyses is widely used to test the association between sepsis and mortality. For instance, through the logistic regression analysis, Vivien et al. [24] observed an association between mortality at day 28 and the tidal volume indexed on ideal body weight (VTIBW) in pre-hospital mechanically ventilated patients with septic shock; Wu et al. [25] revealed that dynamic changes of serum S100B levels from day 3 to 1 were more associated with mortality than those on day 1 in patients with sepsis; Oud et al. [26] indicated that sepsis was associated with most of the short-term deaths among ICU patients with SLE despite its relatively low mortality; Song et al. [5] revealed that combined biomarkers approach showed good performance in predicting 28-day all-cause mortality among patients diagnosed with either sepsis or septic shock according to the sepsis-3 definition, however, the differences might not be statistically proven. Furthemore, some studies [27, 28] found conventional logistic regression had a relatively low indicator of performance as measured by AUCs for ROC curves or showed higher prediction error and worsen performance compared to some novel techniques.

Several conventional prognostic scoring systems have been developed to provide relevant evaluation results considering the hospital mortality of ICU patients. The advantages of such scoring systems are easy to calculate and interpret. SAPS II, as one of the commonly used model, has better discrimination, calibration and power to predict deaths on ICU than the sequential organ failure assessment score (SOFA), which has been recommended for the identification and mortality prognostication of patients in ICU by sepsis-3 [7]. Moreover, the ability of SAPS II to discriminate between survivors and non-survivors is as excellent as APACHE II score and other scores and even to help to play in end-of-life decision-making in ICUs [8]. However, the specificity and sensitivity of scoring systems such as SAPS II are low, and the predictive performance is worse than that of multivariate predictive models. Last but not least, the evaluation systems and the accurate outcomes depended heavily on the practitioner’s experience [6].

In recent years, various machine learning algorithms, a subset of artificial intelligence and a data analysis technique that develops algorithms to predict outcomes by “learning” from data, have been investigated for early detection of sepsis-3 and outperformed than conventional or classic statistic methods, which could automatically analyze complex data and produce significant results. Following is four notable examples of such algorithms. Buchman et al. [29] concluded that machine learning-based CDS tools can accurately predict the onset of sepsis in an ICU patient 4–12 h prior to clinical recognition. Seymour et al. [30] performed different machine learning methods and suggested 4 clinical phenotypes may help in understanding the heterogeneity of treatment effects for patients with sepsis. Kashyap et al. [31] used JMP statistical software to conduct a supervised machine learning for identification of sepsis and septic shock and found it’s a reliable and efficient alternative to manual chart review. Winslow et al. [32] applied machine learning to features calculated from patient with sepsis to estimate whether or not a patient enters this pre-shock state. However, all those articles mentioned above haven’t verified the superiority of machine learning models or done relevant further analysis or offered interpretation compared to other types of prediction model. More importantly, the primary outcomes of these studies are the emergence of detection of sepsis rather than poor clinical outcomes (i.e. mortality) of sepsis. XGBoost, a decision-tree-based algorithm, has been found to be the best algorithm for machine learning and prediction competition hosted by Kaggle.com [10, 33]. Due to its best precision value and performance, XGBoost-based algorithm machine learning is increasingly emphasized as a competitive alternative to regression analysis and used in predicting clinical adverse outcomes.

In terms of the prognosis of sepsis, an artificial intelligence algorithm based on XGBoost has been published by Yuan et al. [9] in 2020. Nevertheless, both of our articles about XGboost models have its own merits. Firstly, there are several limits in Yuan’s study mentioned by himself. For instance, the features selected were according to clinical experience but not algorithm; the representativeness of features may not clear in sepsis and some important dynamic features were not included; left or right censoring may be resulted from incomplete recording of electronic medical records (EMR) when patients transfer or discharge; besides, there were no validations for the XGboost model and no traditional regression analysis was used as a control. Secondly, there are some superiorities in our model compared to Yuan’s machine learning: the features selected were according to backward stepwise analysis which increased representativeness and accuracy; some important features are not missing such as lactate, AG, etc.; data was from MIMIC-III which is an updated database and provides detailed information; classic logistic regression analysis with AUCs and DCAs were used to contrast with XGboost except for traditional scoring system; crucially, nomogram and CIC were plotted to evaluate the clinical usefulness and applicability net benefits of the model. Thirdly, of course, some common limitations exist in both of our articles: measurement bias within calculation is possible due to the method is based on experts’ opinion; sepsis could happen at any time during ICU admission (even possibly hours before labelled), although with the help of algorithm, it’s still difficult for intensivists to integrate the data of point-of-care vital signs, latest lab reports and etc. all the time and to determine the patient condition with sepsis or not according to any database.

An interesting finding in our study is that the features included in the XGBoost-based model and logistic regression model showed consistent, which indicated the excellent performances of XGBoost model were significant, although the two models may fit and perform differently in different datasets. However, these recognitions of the features and sepsis-induced mortality cannot be entirely explained. Hence, further studies and efforts are needed to investigate the mechanisms underlying the role of these variables included in patients with sepsis-3. Following is a brief summary of remarkable or controversial features included in the XGBoost model. Among these features, the weight of urine output is the greatest which represents it is the most important predictor for 30-day mortality MIMIC-III patients. This result is compatible with some clinical studies. Vieira et al. [34] reported higher urine output is associated with successful enteral nutrition therapy in septic shock patients. Laranja et al. [35] concluded that septic patients with no acute kidney injury (AKI) had a more preserved urine output compared to that in all groups with AKI or AKI/chronic kidney disease (CKD). Lin et al. [36] indicated decreased urine output could be manifested as a compensatory mechanism to maintain intravascular volume, and also imply intrinsic renal injury for patients in sepsis. Teixeira et al. [37] confirmed that the use of diuretics was inversely associated with mortality and itself may exert a protective effect. Sodium_max is an interesting feature in our XGBoost-based model. Hypernatremia can be an independent predictor of poor outcome in septic patients in the ICU, which is similar to some views [38]. However, another study [39] showed the risk of death increased by 71.6% when serum sodium was < 129 mmol/L for patients with sepsis. Lactate and AG are typical metabolic indicators. Patients with a normal lactate level alone should not be excluded life-threatening sepsis, and with high AG levels regardless of lactate levels, have high rates of mortality and should also be considered for early, aggressive therapy [40]. However, Liu and Velissaris et al. [41, 42] clearly pointed out that plasma lactate were associated with poor outcomes in patients with sepsis and predicted mortality. INR is another crucial predictive factor in the machine learning model. Several studies [43, 44] found septic patients with elevated INR and platelet count appeared to have a greater risk of death compared with those without coagulopathy. There is no doubt that age and metastatic cancer as basic demographic information could be included in the model which plays unfavorable effects for the mortality. Whereas, survival in critically ill cancer patients with sepsis improved significantly over time but reasons or mechanisms for this condition haven't been identified [45]. In consideration of the source of infection, we found blood infection ranks the highest (38.49%), followed by MRSA screen (35.49%) and urine (17.36%), which indicates that we can perform empirical antibiotics treatment, but de-escalation or determination of whether or not to stop antibiotics or successful implementation of antimicrobial stewardship may help to improve a patient's clinical prognosis while preventing adverse outcomes [46].

The strength of this study was mainly that it was the first time to predict the 30 day mortality of MIMIC-III patients with sepsis-3 using the XGBoost model, and compared to traditional regression analysis and clinical scoring system, and meanwhile verified by nomogram and CIC. We must acknowledge some other limitations of our study: firstly, because the data come from only one database and the majority of patients were white, potential bias may occur; secondly, further exploration for the database was not performed, which may lead to the abandonment of some key variables; thirdly, the proposed model was not designed to be validated by developing set from the database or our clinical data. Even so, we believe that the proposed model may contribute to further our understanding of the prognosis of patients suffering from sepsis in ICU.

Conclusions

In conclusion, this study shows that the machine learning based on XGboost algorithm does outperform conventional logistic regressions and scoring system. This XGboost model may prove clinically useful and assist clinicians in tailoring precise management and therapy for the patients with sepsis-3 which is essential for maximizing the patient’s chance of survival.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

ICU:

Intensive care unit

Ang-2:

Angiopoietin-2

PCT:

Procalcitonin

APHACHE-II:

Acute physiology and chronic health evaluation-II

SAPS-II:

Simplified acute physiology score-II

XGBoost:

EXtreme Gradient Boosting

KDD:

Knowledge Discovery and Data Mining

MIMIC-III:

Medical Information Mart for Intensive Care III

BIDMC:

Beth Israel Deaconess Medical Center

MIT:

Massachusetts Institute of Technology

ICD-9:

International Classification of Diseases and Ninth Revision

NIH:

National Institutes of Health

SQL:

Structure query language

BMI:

Body mass index

HR:

Heart rate

SBP, sysbp:

Systolic blood pressure

DBP, diasbp:

Diastolic blood pressure

MAP:

Mean arterial pressure

TEMP:

Temperature

RR, Resprate:

Respiratory rate

SpO2:

Oxyhemoglobin saturation

ABG:

Arterial blood gas

Meanbp:

Mean blood pressure

Tempc:

Temperature

Bun:

Blood urea nitrogen

Wbc:

White blood cell

INR:

International normalized ratio

Vent:

Ventilation

Max:

Maximum

Min:

Minimum

AIC:

Akaike information criterion

AUCs:

Area under curves

ROC:

The receiver operating characteristic curves

OR:

Odds ratio

CI:

Confidence interval

DCA:

Decision curve analysis

CIC:

Clinical impact curve

WHO:

World Health Organization

SOFA:

The sequential organ failure assessment

qSOFA:

Quick SOFA

VTIBW:

The tidal volume indexed on ideal body weight

MED:

Medical-general service for internal medicine

CMED:

Cardiac medical-for non-surgical cardiac related admissions

EMR:

Electronic medical records

AKI:

Acute kidney injury

CKD:

Chronic kidney disease

References

  1. 1.

    Song JY, Park DW, Moon SW, et al. Diagnostic and prognostic value of interleukin-6, pentraxin 3, and procalcitonin levels among sepsis and septic shock patients: a prospective controlled study according to the sepsis-3 definitions. BMC Infect Dis. 2019a;19(1):968.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  2. 2.

    Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA. 2016;315(8):801–10.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  3. 3.

    Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90–2.

    CAS  PubMed  Article  Google Scholar 

  4. 4.

    Hong TH, Chang CH, Ko WJ, et al. Biomarkers of early sepsis may be correlated with outcome. J Transl Med. 2014;12:146.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  5. 5.

    Song J, Moon SW, Park DW, et al. Biomarker combination and SOFA score for the prediction of mortality in sepsis and septic shock. Medicine. 2020;99(22):e20495.

    PubMed  Article  PubMed Central  Google Scholar 

  6. 6.

    Majdan M, Brazinova A, Rusnak M, et al. Outcome prediction after traumatic brain injury: comparison of the performance of routinely used severity scores and multivariable prognostic models. J Neurosci Rural Pract. 2017;8(1):20.

    PubMed  PubMed Central  Article  Google Scholar 

  7. 7.

    Song JY, Park DW, Moon SW, et al. Validation of APACHE II and SAPS II scales at the intensive care unit along with assessment of SOFA scale at the admission as an isolated risk of death predictor. Anaesthesiol Intensive Ther. 2019b;51(2):107–11.

    Article  Google Scholar 

  8. 8.

    Godinjak A, Iglica A, Rama A, et al. Predictive value of SAPS II and APACHE II scoring systems for patient outcome in a medical intensive care unit. Acta Med Acad. 2016;45(2):97–103.

    PubMed  Article  PubMed Central  Google Scholar 

  9. 9.

    Yuan KC, Tsai LW, Lee KH, et al. The development an artificial intelligence algorithm for early sepsis diagnosis in the intensive care unit. Int J Med Inform. 2020;141:104176.

    PubMed  Article  Google Scholar 

  10. 10.

    Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining-KDD 2016, San Francisco, CA, USA; 2016. p. 785–94.

  11. 11.

    Johnson AEW, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Sci Data. 2016;3:160035.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  12. 12.

    Oweira H, Schmidt J, Mehrabi A, et al. Comparison of three prognostic models for predicting cancer-specific survival among patients with gastrointestinal stromal tumors. Future Oncol. 2018;14(4):379–89.

    CAS  PubMed  Article  Google Scholar 

  13. 13.

    Goldberger A, Amaral L, Glass L, et al. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation. 2000;101(23):e215–20.

    CAS  PubMed  Article  Google Scholar 

  14. 14.

    Templ M, Alfons A, Filzmoser P. Exploring incomplete data using visualization techniques. Adv Data Anal Classif. 2012;6:29–47.

    Article  Google Scholar 

  15. 15.

    Van Buuren S, Groothuis-Oudshoorn CG. mice: multivariate imputation by chained equations in R. J Stat Softw. 2011;45:67.

    Article  Google Scholar 

  16. 16.

    Gall JRL, Lemeshow S, Saulnier F, et al. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270:2957–63.

    Article  Google Scholar 

  17. 17.

    Zhang Z, Ho KM, Hong Y. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care. 2019;23(1):112.

    PubMed  PubMed Central  Article  Google Scholar 

  18. 18.

    Livne M, Boldsen JK, Mikkelsen IK, et al. Boosted tree model reforms multimodal magnetic resonance imaging infarct prediction in acute stroke. Stroke. 2018;49:912–8.

    PubMed  Article  Google Scholar 

  19. 19.

    Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD). Circulation. 2015;131(2):211–9.

    PubMed  PubMed Central  Article  Google Scholar 

  20. 20.

    Fleischmann-Struzek C, Mellhammar L, Reinhart K, et al. Incidence and mortality of hospital and ICU-treated sepsis: results from an updated and expanded systematic review and meta-analysis. Intensive Care Med. 2020;46(8):1552–62.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  21. 21.

    Torio CM, Moore BJ. National inpatient hospital costs: the most expensive conditions by payer, 2013: statistical brief #204. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs, Rockville (MD); 2006.

  22. 22.

    Martin GS, Mannino DM, Eaton S, et al. The epidemiology of sepsis in the united states from 1979 through 2000. N Engl J Med. 2003;348:1546–54.

    PubMed  Article  Google Scholar 

  23. 23.

    World Health Organization. World Health Assembly 70, resolution 70.7: improving the prevention, diagnosis and clinical management of sepsis. 2017. http://apps.who.int/gb/ebwha/pdf_fles/WHA70/A70_R7-en.pdf.

  24. 24.

    Jouffroy R, Saade A, Vivien B, et al. Pre-hospital mechanical ventilation in septic shock patients. Am J Emerg Med. 2019;37(10):1860–3.

    PubMed  Article  Google Scholar 

  25. 25.

    Wu L, Feng Q, Ai ML, et al. The dynamic change of serum S100B levels from day 1 to day 3 is more associated with sepsis associated encephalopathy. Sci Rep. 2020;10(1):7718.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  26. 26.

    Oud L. Epidemiology and outcomes of sepsis among hospitalizations with systemic lupus erythematosus admitted to the ICU: a population-based cohort study. J Intensive Care. 2020;8:3.

    PubMed  PubMed Central  Article  Google Scholar 

  27. 27.

    Xiao J, Ding RF, Xu XL, et al. Comparison and development of machine learning tools in the prediction of chronic kidney disease progression. J Transl Med. 2019;17(1):119.

    PubMed  PubMed Central  Article  Google Scholar 

  28. 28.

    Li YM, Li ZL, Chen F, et al. A LASSO-derived risk model for long-term mortality in Chinese patients with acute coronary syndrome. J Transl Med. 2020;18(1):157.

    PubMed  PubMed Central  Article  Google Scholar 

  29. 29.

    Nemati S, Holder A, Razmi F, et al. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med. 2018;46(4):547–53.

    PubMed  PubMed Central  Article  Google Scholar 

  30. 30.

    Seymour CW, Kennedy JN, Wang S, et al. Derivation, validation, and potential treatment implications of novel clinical phenotypes for sepsis. JAMA. 2019;321(20):2003–17.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  31. 31.

    Dhungana P, Serafim LP, Ruiz AL, et al. Machine learning in data abstraction: a computable phenotype for sepsis and septic shock diagnosis in the intensive care unit. World J Crit Care Med. 2019;8(7):120–6.

    PubMed  PubMed Central  Article  Google Scholar 

  32. 32.

    Liu R, Greenstein JL, Granite SJ, et al. Data-driven discovery of a novel sepsis pre-shock state predicts impending septic shock in the ICU. Sci Rep. 2019;9(1):6145.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  33. 33.

    LANL Earthquake Prediction. 2019. https://www.kaggle.com/c/LANL-EarthquakePrediction. Accessed 15 Mar 2020.

  34. 34.

    Franzosi OS, Nunes DSL, Klanovicz TM, et al. Hemodynamic and skin perfusion is associated with successful enteral nutrition therapy in septic shock patients. Clin Nutr. 2020;S0261–5614(20):30151–5.

    Google Scholar 

  35. 35.

    Pinheiro KHE, Azêdo FA, Areco KCN, et al. Risk factors and mortality in patients with sepsis, septic and non septic acute kidney injury in ICU. J Bras Nefrol. 2019;41(4):462–71.

    PubMed  PubMed Central  Article  Google Scholar 

  36. 36.

    Lin PC, Huang HC, Komorowski M, et al. A machine learning approach for predicting urine output after fluid administration. Comput Methods Programs Biomed. 2019;177:155–9.

    PubMed  Article  Google Scholar 

  37. 37.

    Teixeira C, Garzotto F, Piccinni P, et al. Fluid balance and urine volume are independent predictors of mortality in acute kidney injury. Crit Care. 2013;17(1):R14. https://doi.org/10.1186/cc12484.

    Article  PubMed  PubMed Central  Google Scholar 

  38. 38.

    Shirazy M, Omar I, Abduljabbar D, et al. Prevalence and prognostic impact of hypernatremia in sepsis and septic shock patients in the intensive care unit: a single centre experience. J Crit Care Med. 2020;6(1):52–8.

    Article  Google Scholar 

  39. 39.

    Zhang K, Lv D, Deng Y, et al. STAPLAg: a convenient early warning score for use in infected patients in the intensive care unit. Medicine. 2020;99(22):e20274.

    PubMed  Article  Google Scholar 

  40. 40.

    Ding XF, Yang ZY, Xu ZT, et al. Early goal-directed and lactate-guided therapy in adult patients with severe sepsis and septic shock: a meta-analysis of randomized controlled trials. J Transl Med. 2018;16(1):331.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  41. 41.

    Liu YL, Zheng J, Zhang DS, et al. Neutrophil-lymphocyte ratio and plasma lactate predict 28-day mortality in patients with sepsis. J Clin Lab Anal. 2019;33(7):e22942.

    CAS  PubMed  PubMed Central  Google Scholar 

  42. 42.

    Velissaris D, Karamouzos V, Pantzaris ND, et al. Relation between central venous, peripheral venous and arterial lactate levels in patients with sepsis in the emergency department. J Clin Med Res. 2019;11(9):629–34.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  43. 43.

    Lyons PG, Micek ST, Hampton N, et al. Sepsis-associated coagulopathy severity predicts hospital mortality. Crit Care Med. 2018;46(5):736–42.

    PubMed  Article  Google Scholar 

  44. 44.

    Casado-Méndez M, Fernandez-Pacheco J, Arellano-Orden V, et al. Relationship of thromboelastography and conventional clotting test values with severe bleeding in critically ill patients with coagulopathy: a prospective study. Int J Lab Hematol. 2019;41(5):671–8.

    PubMed  Article  Google Scholar 

  45. 45.

    Lemiale V, Pons S, Mirouse A, et al. Sepsis and septic shock in patients with malignancies: a Groupe de Recherche Respiratoire en Réanimation Onco-Hématologique study. Crit Care Med. 2020;48(6):822–9.

    CAS  PubMed  Article  Google Scholar 

  46. 46.

    Seok H, Jeon JH, Park DW. Antimicrobial therapy and antimicrobial stewardship in sepsis. Infect Chemother. 2020;52(1):19–30.

    PubMed  PubMed Central  Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Affiliations

Authors

Contributions

NH, ML, LH, and KW designed the work. BX, LW, RZ and YY record and summarized the patient of MIMIC features. XS, ZP and KW analyzed datasets. NH, ML and KW wrote this paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kai Wang.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Extracted raw data from the MIMIC-III.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hou, N., Li, M., He, L. et al. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost. J Transl Med 18, 462 (2020). https://doi.org/10.1186/s12967-020-02620-5

Download citation

Keywords

  • MIMIC-III
  • Sepsis-3
  • Machine learning
  • Xgboost
  • Logistic regression
  • SAPS-II score