A novel explainable online calculator for contrast-induced AKI in diabetics: a multi-centre validation and prospective evaluation study
Journal of Translational Medicine volume 21, Article number: 517 (2023)
In patients undergoing percutaneous coronary intervention (PCI), contrast-induced acute kidney injury (CIAKI) is a frequent complication, especially in diabetics, and is connected with severe mortality and morbidity in the short and long term. Therefore, we aimed to develop a CIAKI predictive model for diabetic patients.
3514 patients with diabetes from four hospitals were separated into three cohorts: training, internal validation, and external validation. We developed six machine learning (ML) algorithms models: random forest (RF), gradient-boosted decision trees (GBDT), logistic regression (LR), least absolute shrinkage and selection operator with LR, extreme gradient boosting trees (XGBT), and support vector machine (SVM). The area under the receiver operating characteristic curve (AUC) of ML models was compared to the prior score model, and developed a brief CIAKI prediction model for diabetes (BCPMD). We also validated BCPMD model on the prospective cohort of 172 patients from one of the hospitals. To explain the prediction model, the shapley additive explanations (SHAP) approach was used.
In the six ML models, XGBT performed best in the cohort of internal (AUC: 0.816 (95% CI 0.777–0.853)) and external validation (AUC: 0.816 (95% CI 0.770–0.861)), and we determined the top 15 important predictors in XGBT model as BCPMD model variables. The features of BCPMD included acute coronary syndromes (ACS), urine protein level, diuretics, left ventricular ejection fraction (LVEF) (%), hemoglobin (g/L), congestive heart failure (CHF), stable Angina, uric acid (umol/L), preoperative diastolic blood pressure (DBP) (mmHg), contrast volumes (mL), albumin (g/L), baseline creatinine (umol/L), vessels of coronary artery disease, glucose (mmol/L) and diabetes history (yrs). Then, we validated BCPMD in the cohort of internal validation (AUC: 0.819 (95% CI 0.783–0.855)), the cohort of external validation (AUC: 0.805 (95% CI 0.755–0.850)) and the cohort of prospective validation (AUC: 0.801 (95% CI 0.688–0.887)). SHAP was constructed to provide personalized interpretation for each patient. Our model also has been developed into an online web risk calculator. MissForest was used to handle the missing values of the calculator.
We developed a novel risk calculator for CIAKI in diabetes based on the ML model, which can help clinicians achieve real-time prediction and explainable clinical decisions.
As radiography and percutaneous coronary intervention (PCI) have grown more widely used, contrast-induced acute kidney injury (CIAKI) has risen to become the third most prevalent cause of iatrogenic acute kidney injury , especially in patients with diabetes mellitus (DM) due to the poor vascular conditions [2, 3]. As many as 21.2% of DM patients may suffer from CIAKI , which may lead to as high as 30% mortality from CIAKI . Therefore, an early predictive system applied in diabetic patients according to their risk of CIAKI is crucial to reduce the frequency of CIAKI.
Serum creatinine levels are still used in the current definition of CIAKI, which could delay the diagnosis of CIAKI. Although some novel biomarkers have been proven to predict CIAKI [6, 7], cost-effectiveness limits their widely applications . Clinical risk scores like the Mehran score  have been introduced into clinical practice for decades. However, the predictive power was inadequate in different races or populations. Recently, several studies have demonstrated that the machine learning (ML) model has an excellent prediction performance in kidney disease compared with the traditional statistics model [10,11,12,13]. ML model has a more accurate prediction ability because it gives the probability of events individually rather than risk groups.
However, these ML models rarely explained the models’ variables because of the shortcomings of the black box in ML algorithms. Most studies often lacked the verification of external data sets. Furthermore, there are few prediction models based on the website for clinical use. We intended to apply a range of ML algorithms to establish ML models and compare the models’ prediction performance to the Mehran score . In addition, we used data from multi-centre hospitals as an external cohort and one of the centers as a prospective cohort to validate our model. Then, we established a dynamic and explainable website tool for predicting CIAKI in patients with diabetes.
Study design and participants
The study was divided into two steps. Firstly, we retrospectively reviewed the medical records from multi-center hospitals to build and validate the predictive model. The multi-centre hospitals included Affiliated Sir Run Run Hospital of Nanjing Medical University, Nanjing First Hospital, Affiliated Shu Yang Hospital of Nanjing University of Chinese Medicine and Xu Zhou Medical University Hospital. The study population included diabetic patients who underwent coronary angiography (CAG) and PCI between January 2014 and January 2020. We excluded patients based on the following criteria: (1) missing serum creatinine levels prior to and after CAG and PCI; (2) needing dialysis before CAG and PCI; (3) repeated hospitalization for PCI; and (4) acute kidney injury prior to CAG and PCI. Our research was carried out in respect to the Declaration of Helsinki. Due to its retrospective design, our hospitals gave their approval for the study and waived the need for informed consent.
Secondly, we conducted a prospective study in Affiliated Sir Run Run Hospital of Nanjing Medical University to determine early prediction of CIAKI with our CIAKI online calculator. The study population included adult diabetic patients that underwent CAG and PCI from June 2021 to April 2022. The Ethnic Committee approved this study (Ethics number: 2021-SR-041) and waived the requirement for informed permission to use identifiable data. We reported our work following the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement guideline , Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement  and guidelines for ML predictive modeling .
CIAKI was the primary outcome of our study, based on the Contrast Media Safety Committee (CMSC), described as an increment of serum creatinine value at least 44.2 μmol/L (0.5 mg/dl) or 1.25 times comparing the baseline level within 72 h exposure to contrast agent, eliminating alternative causes of acute kidney injury. The baseline creatinine was the lowest serum creatinine level within 7 days before CAG. In the 72 h following CAG and PCI, all serum creatinine values were collected. Dialysis, stroke, length of in-hospital stay, the new-onset or recurrence of myocardial infarction and other adverse cardiovascular events such as worsening heart failure and death were also included as outcomes.
DM was defined if the patient’s treatment included dietary, oral, or insulin therapy or if patients’ fasting blood glucose value was 126 mg/dl based on the practice guidelines of the American Diabetes Association . Congestive heart failure (CHF) was diagnosed if the patients were grouped into New York Heart Association (NYHA) class III or higher based on the categorization system of the NYHA or history of pulmonary edema. Clinicians comprehensively diagnosed acute coronary syndromes (ACS) according to the symptoms of myocardial ischemia, changes in electrocardiogram, and myocardial injury biomarkers . According to the definition of chronic kidney disease (CKD), patients with proteinuria, estimated glomerular filtration rate (eGFR) < 60 ml/min/1.73m2, or both on at least two occasions more than or equal to three months apart [19, 20].
Data collection and preprocessing
In each institution, demographic data, preoperative medications, and laboratory tests were collected, including gender, age, pre-CAG blood pressure, body mass index (BMI), coronary artery disease, primary disease, contrast agents, and periprocedural biochemical markers. We removed characteristics absent in 11% or more of the samples. The abnormal value of variables were rechecked in electronic hospital records. Otherwise, they were treated as missing values. Categorical variables were processed with one-hot encoding and label encoding. One-hot encoding creates a separate binary feature for each category and is suitable for categorical variables without a specific order or hierarchy. For example, we converted the gender “male” or “female” to “female or not”. Label encoding assigns a unique numerical label to each category. Each category is mapped to a different integer value. Label encoding is suitable for categorical variables with a clear order or hierarchy, such as ordinal variables. For example, the variable “Diabetes history (yrs)” with categories “ < 1 year, 1–5 years, 5–10 years, 10–20 years, > = 20 years” were converted to “1, 2, 3, 4, 5”. Variance inflation factor (VIF) and generalized variance inflation factor (GVIF) were used to deal with collinearity between continuous and categorical variables, respectively. The continuous variables with VIF > 10 were removed. For categorical variables, we set the category with the largest proportion in each categorical variable as the reference level and considered the categorical variables with GVIF^[1/(2 × Df)] > 10^(1/2) to have high collinearity and removed them, where Df refers to the degree of freedom. We divided the data into the cohort of training, internal validation and external validation. We randomly used 80% of the data from Nanjing First Hospital for model training, 20% from Nanjing First Hospital for model internal validation, and other centres for model external validation. We used the missForest method, which can handle missing values with a combination of continuous and categorical variables to fill each remaining measurement’s missing data in the three cohorts separately . Meanwhile, variables were standardized before training and prediction by removing the mean and scaling to unit variance.
In the prospective design, we recorded the time of each variable in the CIAKI model and the time of clinical diagnosis of CIAKI to obtain the earliest time when the model predicted the occurrence of CIAKI. Because of the prospective design, none of the required variables had a missing value.
To solve the imbalance between positive and negative samples, we adopt a variety of balancing methods in the training set, including oversampling and undersampling. Oversampling includes Synthetic Minority Oversampling Technique (SMOTE), ADAptive SYNthetic (ADASYN) technique, and random oversampling. Undersampling includes random undersampling and TomekLinks (Additional file 1: Table S4). Finally, we found that each balancing method performed equally on the internal validation set, but TomekLinks performed better in the external validation set, so we chose to use TomekLinks. Specifically, TomekLinks focuses on neighboring pairs of samples, where one sample belongs to the minority class and the other belongs to the majority class. These sample pairs are close to each other and form links. These links are considered potential noise or outliers, which may have a negative impact on model training and performance. By identifying and addressing these links, we can reduce noise or outliers in the data and improve the performance of the classification model.
Mehran risk score
Mehran risk score  is calculated with 8 variables: hypotension, CHF, intra-arterial balloon pump (IABP), anemia, age, diabetes, contrast media volume, serum creatinine or eGFR. We calculated the total Mehran risk scores for each patient based on the sum of the scores corresponding to the 8 variables which were 5 points for hypotension (Systolic blood pressure is less than 80 mmHg for at least 1 h and inotropic assistance is required), 5 points for IABP (IABP is used), 5 points for CHF (NYHA classification III/IV or history of pulmonary edema), 4 points for age (more than 75 years old), 3 points for anemia (men’s hematocrit less than 39% while women’s less than 36%.), 3 points for diabetes, 1 point for contrast media volume per 100ml and 4 points for serum creatinine > 1.5mg/dl, or 2 points for eGFR 40–60 ml/min/1.73m2, 4 points for eGFR 20–40 ml/min/1.73m2, and 6 points for eGFR < 20 ml/min/1.73m2.
Machine learning development
Six ML models were constructed, including extreme gradient boosting trees (XGBT) model, random forest (RF) model, support vector machine (SVM) model, logistic regression (LR) model, least absolute shrinkage and selection operator (LASSO) with LR (Lasso + LR), and gradient boosted decision trees (GBDT) model. Additional file 1 included a full explanation of the six ML models.
ML models were also trained using ten-fold cross-validation (Additional file 1: Figure S1 and Additional file 1: Table S3). The initial samples were randomly split into ten equal-sized subsamples, one of which was used to validate the results and the other nine as training samples. For each model, in order to select the ideal hyperparameters, a grid search method with ten-fold cross validation was used. Furthermore, we constructed the SHapley Additive exPlanation (SHAP), which demonstrates each variable’s impact on the overall model as well as its contributions to the model. Additionally, the SHAP plot function was also used to reveal the XGBT model’s complicated link between factors and results. Finally, to forecast the risk of CIAKI in diabetics, we developed an explainable online web-based risk calculator.
All models were evaluated in internal as well as external validation sets. Each model’s area under the receiver operating characteristic curve (AUC), accuracy, positive predictive value (PPV), negative predictive value (NPV), sensitivity, specificity, and F1 score were also compared. Additionally, we chose the CIAKI prediction threshold by maximizing the F1 score in the training set. A 95% confidence interval (CI) was performed in 1000 iterations of bootstrap sampling with replacement. To examine the agreement between calculated likelihood and observed CIAKI prevalence in the population, a calibration curve was utilized. Moreover, the net benefit of each model was calculated using decision curve analysis (DCA) based on the difference between the predicted benefit and the expected risk associated with CIAKI.
For descriptive analyses, categorical variables were expressed as quantities and percentages. To compare categorical variables, chi-square tests were utilized. Analysis and expression of continuous variables using the mean and standard deviation or median and interquartile range were compared using either the Independent-sample T-test or the Mann–Whitney U test. All analyses were carried out with Python version 3.9.7, R version 4.1.0, and SPSS version 22.0. P < 0.05 was used as the statistical significance level.
From January 1, 2014 to January 30, 2020, a total of 5052 diabetic patients underwent CAG + PCI. Based on the inclusion and exclusion criteria, 3514 patients were included in the study. The characteristics of included patients and excluded patients were compared in Additional file 1: Table S1. Patients that were excluded had more CKD stage 4–5, higher blood urea nitrogen levels and more vessels of coronary artery disease. The main reason was that the exclusion criteria included patients who were on dialysis and had developed acute kidney injury before CAG + PCI which resulted in excluded patients having worse kidney function. Moreover, the exclusion criteria included repeated hospitalization and repeated CAG which suggested that the excluded patient’s coronary artery disease might be more serious like more vessels of coronary artery disease and required multiple CAG examinations. The remaining characteristics were not statistically different between the included and excluded patients. We randomly partitioned patients from Nanjing First Hospital into the cohort of training (80%, 2368 patients) and the cohort of internal validation (20%, 592 patients). Patients from the other three hospitals were used as the external validation cohorts (554 patients) (Fig. 1). The frequency of CIAKI in training, and internal and external validation sets was 447/2368 (18.9%), 107/592 (18.1%), and 80/554 (14.4%), respectively (Fig. 1). Additional file 1: Table S2 displayed the three cohorts’ initial characteristics. The median age in the cohort of training, internal validation and external validation was 67-year-old (interquartile range [IQR]: 60–74), 67-year-old (IQR: 59–73) and 65-year-old (IQR: 58–73), respectively. Patients with CIAKI were older, had worse heart dysfunction, higher frequency of CKD 3–4 stage, coronary artery disease and anemia, and higher uric acid, urine protein, and total cholesterol (Table 1).
634 patients (18.04%) developed CIAKI, and 50 (1.42%) occurred adverse cardiovascular events. Of them, 21 (0.60%) patients revealed worse heart failure, and 12 (0.34%) reoccurred or experienced a new-onset myocardial infarction. On top of that, we observed 7 (0.20%) patients developed stroke, and 11 (0.31%) death. CIAKI was linked to an increased risk of myocardial infarction (0.79% vs 0.24%, P = 0.033) and overall adverse cardiovascular events (2.5% vs 1.2%, P = 0.01) and increased hospitalization stay (9.23 ± 4.87 vs 7.38 ± 4.87, P < 0.001) (Table 2).
During the study, 61 essential characteristics from electronic medical records were chosen, including 5 demographic data, 13 medical history characteristics, 9 intraoperative indicators, 6 preoperative medications, 16 preoperative laboratory tests, and 6 postoperative serum creatinine and postoperative blood urea nitrogen at 24, 48, and 72 h. In addition, 6 clinical endpoint variables were included. After data cleaning, missing values greater than or equal to 11% (Killip classification, glycated hemoglobin and NYHA classification) were removed. At the same time, collinear features were removed, including height, weight, nonionic iso-osmolar and total cholesterol. Since this study used pre-CAG + PCI and intraoperative indicators to predict the occurrence of postoperative CIAKI, we removed postoperative serum creatinine and postoperative blood urea nitrogen at 24, 48, and 72 h. We retained preoperative serum creatinine and preoperative blood urea nitrogen. In addition, 6 clinical endpoint variables occurred postoperatively, and we did not include them as risk factors As a result, 37 features, including 20 categorical features and 17 continuous variables, were retained in the training cohort to establish ML models. Further, we screened 23 features by the LASSO + LR model for CIAKI (Additional file 1: Figure S2). We also showed the top 20 features for CIAKI in each ML model (Additional file 1: Figure S3). Figure 2a illustrated the scaled importance rank of all features in the six ML models for identifying CIAKI risk in diabetic patients. Figure 2b, a subset of Fig. 2a, showed the importance of the final 15 variables screened in the six ML models. Figure 2c showed the relative importance of the 15 variables in Fig. 2b in the six ML models. As shown in Figs. 2b and c, ACS presented the most crucial feature in all ML models.
In the internal validation cohort, all ML models achieved higher AUC than Mehran score, of which the XGBT model achieved the best AUC (0.816, 95% CI 0.777 to 0.853) (Fig. 3a). Additionally, all of the ML models outperformed the Mehran score in terms of F1 score, accurary, sensitivity, spensitivity, PPV and NPV. (Table 3). The XGBT model demonstrated superior performance than the other 4 ML models. The calibration curves of the 5 ML models and Mehran score were shown in Fig. 3b. The DCA indicated that the risk threshold of the XGBT model ranged from 0.20 to 0.50, which was superior to the ranges associated with other models (Fig. 3c). Additionally, the net benefit of the XGBT, RF, LASSO + LR and LR were optimistic when the risk threshold was in the range of 0 to 0.55.
The AUC performance in the cohort of external validation was comparable to that in the cohort of internal validation. The better performance of the ML models than the Mehran score remained consistent. XGBT model achieved better AUC (0.816, 95% CI 0.770 to 0.861) than others (Fig. 3d). Furthermore, the calibration curves of all models still performed well (Fig. 3e). According to the DCA, the XGBT model provided a clinical net benefit when models ranged from 0.10 to 0.65 (Fig. 3f).
The best prediction model determination
Among all the models, XGBT achieved the best AUC in both internal and external validation cohorts. In general, the AUC of the model will increase as more features are selected. Nevertheless, adding more features does not improve clinical practice. To discover the significant features, we sorted the importance of XGBT features in descent order in the training set. The performances of AUC, sensitivity, specificity and accuracy corresponding to the top n number of features in XGBT were shown in Fig. 4a1. When the number of features was increased to the top 15, the AUC rose to 0.883, and the accuracy and sensitivity had significant improvements. With the number of features growing to 22, the AUC steadily reached its maximum which was 0.896. However, the sensitivity had a small decrease from 0.644 to 0.611 when the number was from the top 15 to 22. The performance of AUC, specificity, sensitivity and accuracy tended to be stable and no longer changed after more than 27 features. Considering more features were not beneficial to clinical use and practice, and the AUCs after 15 variables were overall stable, to facilitate the application in clinical practice, we selected the top 15 critical variables as the brief CIAKI prediction model for diabetes, called the BCPMD model: ACS, Urine protein level, Diuretics, left ventricular ejection fraction (LVEF)(%), Hemoglobin, CHF, Stable Angina, Uric acid, Preoperative DBP, Contrast Volumes, Albumin, Baseline creatinine, Vessels of coronary artery disease, Glucose and Diabetes history (Fig. 4a1 and a2). The corresponding risk threshold of BCPMD was 0.3338 which was based on maximizing the F1 score. Violin plots were analyzed to demonstrate the distribution of 8 continuous characteristics contained in BCPMD between CIAKI patients (n = 634) and non-CIAKI patients (n = 2880) (Fig. 2d and Additional file 1: Table S5). Also, relationships between 7 categorical features and CIAKI were observed in Fig. 2d. Besides, The AUCs of the BCPMD for CIAKI in the cohort of training, internal validation, and external validation were 0.883 (95% CI 0.867–0.898), 0.819 (95% CI 0.783–0.855), and 0.805 (95% CI 0.755–0.850), respectively (Fig. 4b). The expected calibration errors (ECE) in calibration curves of BCPMD were 0.073 for the cohort of internal validation and 0.135 for external validation (Fig. 4c and e). The net benefits of the BCPMD in the cohort of external validation were reduced than in the cohort of internal validation (Fig. 4d and f).
Table 4 displayed the BCPMD’s prospective predictive performance. AUC, accuracy, sensitivity, specificity, PPV, NPV, and F1 scores of BCPMD were 0.801 (95% CI 0.688–0.887), 0.826 (95% CI 0.779–0.866), 0.684 (95% CI 0.500–0.846), 0.843 (95% CI 0.793–0.887), 0.351 (95% CI 0.219–0.485), 0.956 (95% CI 0.924–0.979), 0.464 (95% CI 0.311–0.586), respectively. Moreover, CIAKI occurred in 19/172 (11.0%) in the prospective cohort. Of the 19 patients with true CIAKI, BCPMD correctly predicted 13 patients. Additional file 1: Table S6 displayed the prospective validation cohort’s basic characteristics.
SHAP values evaluate feature importance
We explained the BCPMD through the SHAP diagram. After inputting each variable, the model’s positive or negative contribution could be observed (Fig. 5c). The SHAP summary plot demonstrated that ACS, hemoglobin, diuretics, LVEF (%) and uric acid (umol/L) ranked as the top 5 important features. Moreover, the SHAP plot revealed that ACS, lower hemoglobin (g/L), using diuretics, lower LVEF (%) and higher uric acid (umol/L) were correlated with a greater SHAP value generated in BCPMD, implying a higher risk of CIAKI. Figure 5a showed two cases (Patient No.2, No.17) by SHAP decision plot, which simulated the path of each feature decision. In addition, the different feature values of BCPMD represented different positive and negative contributions to the final SHAP value output. Red values represented higher risk factors, and blue values represented lower risk factors (Fig. 5b). It reflects the personalized interpretation function of SHAP and helps physicians make clinical decisions on the individual level.
CIAKI Web calculator development
Based on the BCPMD, we built a dynamic and explainable website to calculate the risk of CIAKI in diabetes. The URL is here: http://188.8.131.52/. When a patient plans to undergo CAG and PCI, the physician can enter the associated risk factor values into the website, which will immediately produce the projected risk values for CIAKI. Furthermore, the risk of CIAKI was judged as negative or positive according to the threshold of 0.3338 on the platform. Moreover, we used the missForest method to impute the missing data to predict the risk of CIAKI even when features are unknown. Notably, we developed a dynamic and explainable waterfall diagram to show the positive or negative contribution of different risk factor values, in which red presents higher risk and blue presents lower risk. Figure 6 showed an example that our web calculator predicted CIAKI in a case within 1 h of admission.
In this study, we employed ML algorithms to develop an innovative prediction tool. Compared to Mehran risk scores, our results showed that ML models were superior to traditional logistic regression. Notably, in both the cohort of internal and external validation, the XGBT model performed best. Further, we determined the top 15 important predictors in the XGBT model as BCPMD model variables as these variables can be collected easily in medical activities. Similarly, AUC for CIAKI in the cohorts of internal validation, external validation, and prospective validation was shown by BCPMD to be 0.819 (95% CI 0.783–0.855), 0.805 (95% CI 0.755–0.850) and 0.801 (95% CI 0.688–0.887), respectively. In addition, we constructed SHAP to provide personalized interpretation for each patient. An online web risk calculator model of CIAKI in diabetes was then established to predict the occurrence of CIAKI within 1 h when patients arrived at the hospital.
The previous study indicated Mehran’s score could predict CIAKI with an AUC of 0.67 in the validation cohort . Our results verified the AUC of Mehran score was 0.654 in the cohort of internal validation and 0.656 in external validation for CIAKI in patients with diabetes. Mehran score models were updated in 2021, with model 1 including indicators before CAG, and model 2 adding procedural features, giving a better AUC of 0.84 . However, with the development of biomarkers and algorithms, ML technology is gradually emerging as a better tool for establishing prediction models. Yin et al.  constructed a CIAKI prediction model using 13 preprocedural indicators through an RF algorithm, revealing an AUC of 0.907 and an accuracy of 80.8%. Other researchers also found that GBDT  and RNN  could perform well in predicting CIAKI. Moreover, Sun et al.  exhibited that in patients with ACS, the LASSO + LR-based nomogram model provided a better prediction of CIAKI than the Mehran score (AUC was 0.835 and 0.762, respectively). According to our results, in diabetic patients, ML models (including LASSO + LR, GBDT, XGBT, and SVM) demonstrated better discriminative power than traditional LR and Mehran score in developing predictive models. Additionally, our data displayed that XGBT performed best, which was an ensemble of weak prediction trees . The XGBT algorithms can capture complex relationships in data without explicit specification of higher-order interactions and nonlinear functions . Furthermore, XGBT algorithms prevent overfitting through cross-validation and regularization .
The BCPMD model included 15 features, which were easily accessible in clinical activities. Although the 15 features were readily accessible, missing data could still occur in different regions or circumstances, affecting the model’s performance and delaying the prediction time. As a result, we adopted missForest  to handle mixed-type data with both missing continuous and categorical patient variables to make our web predictive tool perform well.
Notably, our model suggested that ACS was the most significant risk factor for CIAKI in diabetic individuals, consistent with current studies [31,32,33]. In addition to the signal pathway regulation and contrast medium’s harmful effects on renal tubular cells , ACS may have a comparable mode of action with diabetes, leading to the superposition of kidney injury. On the one hand, they both affect renal perfusion. Patients with ACS often have unstable hemodynamics. In the case of cardiac vascular stenosis, cardiac ejection function is impaired, and hypotension occurs, which may result in decreased renal perfusion and kidney injury . Likewise, acute myocardial ischemia can activate renin angiotensin aldosterone system (RAAS). Vasopressin, catecholamine and interleukin are produced, and the level of nitric oxide is reduced, damaging endothelial cells and bringing about decreased renal blood flow [36, 37]. On the other hand, ACS can give rise to kidney inflammation and oxidative stress damage, like diabetes [38, 39].
Additionally, our results revealed that hyperuricemia constituted a significant risk factor for CIAKI in diabetes. A recent study from China proved that hyperuricemia was associated with CIAKI (OR = 2.363, 95% CI 1.653–3.377, P < 0.001) . What’s more, it was also shown that patients with uric acid levels above 8.0 mg/dL not only had a greater risk of CIAKI but also an increased risk of hemodialysis . Uric acid can promote oxidative stress and release a variety of proinflammatory factors, resulting in renal vasoconstriction and endothelial dysfunction . At the same time, contrast agents can give rise to acute uricosuria , further aggravating kidney injury. Besides, diuretics were one of the important factors in the model. This may be because diuretics can accelerate the excretion of iodine and improve urine viscosity . Whereas more and more studies believed that diuretics are independent predictors of CIAKI in recent years [45, 46]. The National Kidney Foundation and the American College of Radiology proposed that it was not recommended to use drugs that can affect renal function within 48 h before and after iodine contrast agents, including diuretics . Considering the hypoxia and inflammatory reaction induced by diuretics, using diuretics during the perioperative period of PCI may be a potential risk of CIAKI . Our study also confirmed the increased risk of CIAKI among patients suffering from heart failure, worse renal function, anemia, poor blood glucose control and more contrast volumes, underlining the need for early prevention strategies for these patients at high risk.
Of note, our web CIAKI risk calculator could be used as a guide for clinicians compared with previous studies that only stayed in constructing models, lacking practical value. Evidence has shown that early clinical intervention could improve CIAKI patients’ outcomes [49, 50]. The time window between evidence of increased CIAKI risk in the prediction platform and the occurrence of clinical CIAKI is an ideal period for clinical intervention. When combining the platform’s prediction and early intervention, the risk of CIAKI is expected to be reduced.
Our study has several strengths, the first of which was generalizability. We assessed the BCPMD model in multi-centre hospitals and prospectively constructed the web platform based on BCPMD. Our results also showed that despite the difference in our data distribution in the external set, it did not affect the model’s predictive ability, indicating that the BCPMD model is generalizable. Secondly, the feature of BCPMD was readily accessible in routine clinical practice. We also found that it was not the greater the number of predictor variables, the higher the model’s prediction ability. Therefore, we screened out a certain number of optimal subsets according to the model effect of different feature numbers to make the model more efficient and straightforward. Thirdly, our model can be used for clinical practice. We developed a dynamically interpretable prediction web platform for the first time. Meanwhile, we set the missing value filling for the platform. Additionally, considering the ML models’ black-box flaws, we used SHAP to explain whether features contributed positively or negatively to ML models, which can explain how each characteristic affects the overall forecast of the model and how our model features affect CIAKI at the individual level. Our web calculator provides a tool that can real-time predict high-risk CIAKI patients and helps clinicians simply and intuitively understand how different values of a single feature affect the model's predictions, which can be as a reference for other disease models.
We also have some limitations in our research. Firstly, 30% of our CAG + PCI patients were excluded from the inclusion criteria. Although most of our characteristics did not differ between excluded patients and included patients, there are still some characteristics that we did not pay attention to that might have possible bias. Therefore, a large sample size of data is needed for verification in the future. Secondly, even though our model’s AUC in the prospective validation set was performing well, we observed that not all risk thresholds were beneficial for patients. In the prospective validation set, a risk threshold lower than 0.30 has no benefit. However, it can identify more lower-risk CIAKI patients, who can give routine interventions such as closely detecting the serum creatinine. In the risk threshold of 0.55 to 0.70, patients with CIAKI can benefit and be identified more accurately. More comprehensive intervention methods, such as adequate hydration, should be given to these high-risk patients. However, using a higher risk threshold means part of CIAKI patients may not be identified ahead of time. It needs to be set according to the patient characteristics of different institutions. Thirdly, we still used serum creatinine for the definition of CIAKI. More early diagnostic markers and clinical features could be added to increase the prediction probability of CIAKI in the future.
In conclusion, we developed a web tool based on the BCPMD model that could identify high-risk CIAKI patients in diabetes and accurately stratify the risk of CIAKI. In the future, early kidney injury prevention combined with artificial intelligence are expected to improve outcomes in patients with CIAKI.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Percutaneous coronary intervention
Contrast-induced acute kidney injury
Gradient-boosted decision trees
Least absolute shrinkage and selection operator
Extreme gradient boosting trees
Support vector machine
Area under the curve
Positive predictive value
Negative predictive value
Decision curve analysis
- 95% CI:
95% Confidence interval
Brief CIAKI prediction model for diabetes
Shapley additive explanations
Acute coronary syndromes
Left ventricular ejection fraction
Body mass index
Chronic kidney disease
Congestive heart failure
Systolic blood pressure
Diastolic blood pressure
Calcium channel blocker
Angiotensin-converting enzyme inhibitor
Angiotensin receptor blocker
Estimated glomerular filtration rate
Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis
Strengthening the reporting of observational studies in epidemiology
Contrast media safety committee
New York Heart Association
Variance inflation factor
Synthetic minority oversampling technique
Adaptive synthetic technique
Intra-arterial balloon pump
Pattharanitima P, Tasanarong A. Pharmacological strategies to prevent contrast-induced acute kidney injury. Biomed Res Int. 2014;2014:236930–50.
Nough H, Daryachahei R, Hadiani L, Najarzadegan MR, Mirzaee M, Hemayati R, et al. Ascorbic acid effect on CIN incidence in diabetic patient after coronary angiography. Adv Biomed Res. 2016;5:69–75.
Liu L, Liang Y, Li H, Lun Z, Ying M, Chen S, et al. Association between diabetes mellitus and contrast-associated acute kidney injury: A systematic review and Meta-analysis of 1.1 million contrast exposure patients. Nephron. 2021;145:451–61.
Qin Y, Tang H, Yan G, Wang D, Qiao Y, Luo E, et al. A high triglyceride-glucose index is associated with contrast-induced acute kidney injury in Chinese patients with type 2 diabetes mellitus. Front Endocrinol (Lausanne). 2020;11:522883–90.
From AM, Bartholmai BJ, Williams AW, Cha SS, McDonald FS. Mortality associated with nephropathy after radiographic contrast exposure. Mayo Clin Proc. 2008;83:1095–100.
Scridon A, Somkereki C, Nicoară TR, Oprica M, Demian L. Neutrophil gelatinase-associated lipocalin monitoring reveals persistent subclinical kidney injury following intraarterial administration of iodinated contrast agents. Sci Rep. 2022;12:19464–680.
Zdziechowska M, Gluba-Brzózka A, Franczyk B, Rysz J. Biochemical markers in the prediction of contrast-induced acute kidney injury. Curr Med Chem. 2021;28:1234–50.
Ugwuowo U, Yamamoto Y, Arora T, Saran I, Partridge C, Biswas A, et al. Real-time prediction of acute kidney injury in hospitalized adults: implementation and proof of concept. Am J Kidney Dis. 2020;76:806–14.
Mehran R, Aymong ED, Nikolsky E, Lasic Z, Iakovou I, Fahy M, et al. A simple risk score for prediction of contrast-induced nephropathy after percutaneous coronary intervention: development and initial validation. J Am Coll Cardiol. 2004;44:1393–9.
Lee AM, Hu J, Xu Y, Abraham AG, Xiao R, Coresh J, et al. Using machine learning to identify metabolomic signatures of pediatric chronic kidney disease etiology. J Am Soc Nephrol. 2022;33:375–86.
Yue S, Li S, Huang X, Liu J, Hou X, Zhao Y, et al. Machine learning for the prediction of acute kidney injury in patients with sepsis. J Transl Med. 2022;20:215–32.
Zhang Y, Yang D, Liu Z, Chen C, Ge M, Li X, et al. An explainable supervised machine learning predictor of acute kidney injury after adult deceased donor liver transplantation. J Transl Med. 2021;19:321–39.
Tseng PY, Chen YT, Wang CH, Chiu KM, Peng YS, Hsu SP, et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Crit Care. 2020;24:478–93.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:7594–602.
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147:573–7.
Luo W, Phung D, Tran T, Gupta S, Rana S, Karmakar C, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res. 2016;18:323–35.
Genuth S, Alberti KG, Bennett P, Buse J, Defronzo R, Kahn R, et al. Follow-up report on the diagnosis of diabetes mellitus. Diabetes Care. 2003;26:3160–7.
Collet JP, Thiele H, Barbato E, Barthélémy O, Bauersachs J, Bhatt DL, et al. 2020 ESC guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J. 2021;42:1289–367.
Foundation NK. K/DOQI clinical practice guidelines for chronic kidney disease: evaluation, classification, and stratification. Am J Kidney Dis. 2002;39(2 Suppl 1):1–266.
Stevens LA, Schmid CH, Greene T, Zhang YL, Beck GJ, Froissart M, et al. Comparative performance of the CKD epidemiology collaboration (CKD-EPI) and the modification of diet in renal disease (MDRD) study equations for estimating GFR levels above 60 mL/min/1.73 m2. Am J Kidney Dis. 2010;56:486–95.
Gao Y, Cai GY, Fang W, Li HY, Wang SY, Chen L, et al. Machine learning based early warning system enables accurate mortality risk prediction for COVID-19. Nat Commun. 2020;11:5033–42.
Mehran R, Owen R, Chiarito M, Baber U, Sartori S, Cao D, et al. A contemporary simple risk score for prediction of contrast-associated acute kidney injury after percutaneous coronary intervention: derivation and validation from an observational registry. Lancet. 2021;398:1974–83.
Yin WJ, Yi YH, Guan XF, Zhou LY, Wang JL, Li DY, et al. Preprocedural prediction model for contrast-induced nephropathy patients. J Am Heart Assoc. 2017;6:4498–508.
Huang C, Murugiah K, Mahajan S, Li SX, Dhruva SS, Haimovich JS, et al. Enhancing the prediction of acute kidney injury risk after percutaneous coronary intervention using machine learning techniques: A retrospective cohort study. Plos Med. 2018;15:2703–22.
Yun D, Cho S, Kim YC, Kim DK, Oh KH, Joo KW, et al. Use of deep learning to predict acute kidney injury after intravenous contrast media administration: prediction model development study. JMIR Med Inform. 2021;9:27177–93.
Ma K, Li J, Shen G, Zheng D, Xuan Y, Lu Y, et al. Development and validation of a risk nomogram model for predicting contrast-induced acute kidney injury in patients with non-ST-elevation acute coronary syndrome undergoing primary percutaneous coronary intervention. Clin Interv Aging. 2022;17:65–77.
Zhang Z, Ho KM, Hong Y. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care. 2019;23:112–25.
Friedman JH. Stochastic gradient boosting. Comput Stat Data an. 2002;38:367–78.
Chen T, Guestrin C. XGBoost: a scalable tree boosting system. New York: ACM Press; 2016.785–94.
Stekhoven DJ, Bühlmann P. MissForest–non-parametric missing value imputation for mixed-type data. Bioinformatics. 2012;28:112–8.
Krasinski Z, Krasińska B, Olszewska M, Pawlaczyk K. Acute renal failure/acute kidney injury (AKI) associated with endovascular procedures. Diagnostics (Basel). 2020;10:274–301.
Gao YM, Li D, Cheng H, Chen YP. Derivation and validation of a risk score for contrast-induced nephropathy after cardiac catheterization in Chinese patients. Clin Exp Nephrol. 2014;18:892–8.
Zeng JF, Chen SQ, Ye JF, Chen Y, Lei L, Liu XQ, et al. A simple risk score model for predicting contrast-induced nephropathy after coronary angiography in patients with diabetes. Clin Exp Nephrol. 2019;23:969–81.
Sudarski D, Naami R, Shehadeh F, Elias A, Kerner A, Aronson D. Risk of worsening renal function following repeated exposures to contrast media during percutaneous coronary interventions. J Am Heart Assoc. 2021;10:21473–87.
Giacoppo D, Madhavan MV, Baber U, Warren J, Bansilal S, Witzenbichler B, et al. Impact of contrast-induced acute kidney injury after percutaneous coronary intervention on short- and long-term outcomes: pooled analysis from the HORIZONS-AMI and ACUITY trials. Circ Cardiovasc Interv. 2015;8:2475–83.
Muslem R, Caliskan K, Akin S, Yasar YE, Sharma K, Gilotra NA, et al. Effect of age and renal function on survival after left ventricular assist device implantation. Am J Cardiol. 2017;120:2221–5.
Denegri A, Mehran R, Holy E, Taramasso M, Pasotti E, Pedrazzini G, et al. Post procedural risk assessment in patients undergoing trans aortic valve implantation according to the age, creatinine, and ejection fraction-7 score: advantages of age, creatinine, and ejection fraction-7 in stratification of post-procedural outcome. Catheter Cardiovasc Interv. 2019;93:141–8.
Yao ZF, Shen H, Tang MN, Yan Y, Ge JB. A novel risk assessment model of contrast-induced nephropathy after percutaneous coronary intervention in patients with diabetes. Basic Clin Pharmacol Toxicol. 2021;128:305–14.
Toprak O, Cirit M, Yesil M, Bayata S, Tanrisev M, Varol U, et al. Impact of diabetic and pre-diabetic state on development of contrast-induced nephropathy in patients with chronic kidney disease. Nephrol Dial Transpl. 2007;22:819–26.
Hu Y, Wang X, Xiao S, Sun N, Huan C, Wu H, et al. A clinical nomogram based on the triglyceride-glucose index to predict contrast-induced acute kidney injury after percutaneous intervention in patients with acute coronary syndrome with diabetes mellitus. Cardiovasc Ther. 2022;2022:5443880–93.
Wu MJ, Tsai SF, Lee CT, Wu CY. The predictive value of hyperuricemia on renal outcome after contrast-enhanced computerized tomography. J Clin Med. 2019;8:1003–15.
Ejaz AA, Dass B, Kambhampati G, Ejaz NI, Maroz N, Dhatt GS, et al. Lowering serum uric acid to prevent acute kidney injury. Med Hypotheses. 2012;78:796–9.
Aslan G, Afsar B, Sag AA, Camkiran V, Erden N, Yilmaz S, et al. The effect of urine pH and urinary uric acid levels on the development of contrast nephropathy. Kidney Blood Press Res. 2020;45:131–41.
Seeliger E, Ladwig M, Sargsyan L, Cantow K, Persson PB, Flemming B. Proof of principle: hydration by low-osmolar mannitol-glucose solution alleviates undesirable renal effects of an iso-osmolar contrast medium in rats. Invest Radiol. 2012;47:240–6.
Yuan Y, Qiu H, Hu X, Zhang J, Wu Y, Qiao S, et al. A risk score model of contrast-induced acute kidney injury in patients with emergency percutaneous coronary interventions. Front Cardiovasc Med. 2022;9:989243–51.
Karauzum I, Karauzum K, Hanci K, Gokcek D, Kalas B, Ural E. The utility of systemic immune-inflammation index for predicting contrast-induced nephropathy in patients with ST-segment elevation myocardial infarction undergoing primary percutaneous coronary intervention. Cardiorenal Med. 2022;12:71–80.
Davenport MS, Perazella MA, Yee J, Dillman JR, Fine D, McDonald RJ, et al. Use of intravenous iodinated contrast media in patients with kidney disease: consensus statements from the american college of radiology and the national kidney foundation. Radiology. 2020;294:660–8.
Hu W, Lian X, Lin J, Chen Y, Wu Y, Liu W, et al. The incidence, characteristics, and use of suspected nephrotoxic drugs in elderly patients with community-acquired acute kidney injury. Clin Interv Aging. 2021;16:35–42.
Briguori C, D’Amore C, De Micco F, Signore N, Esposito G, Visconti G, et al. Left ventricular end-diastolic pressure versus urine flow rate-guided hydration in preventing contrast-associated acute kidney injury. JACC Cardiovasc Interv. 2020;13:2065–74.
Qian G, Fu Z, Guo J, Cao F, Chen Y. Prevention of contrast-induced nephropathy by central venous pressure-guided fluid administration in chronic kidney disease and congestive heart failure patients. JACC Cardiovasc Interv. 2016;9:89–96.
We thank all the nurses, physicians, and providers who contributed to the care of these patients. We thank the study participants for their contributions.
This study was supported by the National Natural Science Foundation of China (82170698), the project for Scientific Research and Practice Innovation of Jiangsu Province (JX52013950), the project for Medical Science and Technology Development of Nanjing (YKK19169), Nanjing Medical University Big Data Clinical Research Project and the project for Life Science and Technology of Nanjing (2021SX00000234).
Ethics approval and consent to participate
Our hospitals’ Ethics Committee gave their approval for the study and waived the need for informed consent.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Methods. Description of the six ML models. Table S1. Baseline characteristics between included and excluded patients. Table S2. Baseline characteristics of patients in the three groups. Table S3. Ten-fold cross-validation results of AUC and accuracy in models. Table S4. Model performance using different balancing methods. Table S5. The median [IQR] of the continuous features in BCPMD. Table S6. Baseline characteristics of patients in the prospective cohort. Figure S1. Ten-fold cross-validation results of AUC and accuracy in models. Figure S2. Feature screening process of LASSO. (a) Lasso ten-fold cross-validation determines the number of important features according to the binomial deviation (λ=0.005774419, n=23). (b) The dynamic change in risk factors with the penalty coefficient; the vertical line indicates the optimal λ (n=23). Figure S3. The top 20 features of ML models. Figure S4. The prediction process of a patient on the dynamic explainable CIAKI risk calculator.
About this article
Cite this article
Ma, M., Wan, X., Chen, Y. et al. A novel explainable online calculator for contrast-induced AKI in diabetics: a multi-centre validation and prospective evaluation study. J Transl Med 21, 517 (2023). https://doi.org/10.1186/s12967-023-04387-x
- Contrast-induced acute kidney injury
- Machine learning
- Web calculator
- Predictive models
- Diabetes mellitus