Development of a risk prediction model for bloodstream infection in patients with fever of unknown origin
Journal of Translational Medicine volume 20, Article number: 575 (2022)
Bloodstream infection (BSI) is a significant cause of mortality among patients with fever of unknown origin (FUO). Inappropriate empiric antimicrobial therapy increases difficulty in BSI diagnosis and treatment. Knowing the risk of BSI at early stage may help improve clinical outcomes and reduce antibiotic overuse.
We constructed a multivariate prediction model based on clinical features and serum inflammatory markers using a cohort of FUO patients over a 5-year period by Least Absolute Shrinkage and Selection Operator (LASSO) and logistic regression.
Among 712 FUO patients, BSI was confirmed in 55 patients. Five independent predictors available within 24 h after admission for BSI were identified: presence of diabetes mellitus, chills, C-reactive protein level of 50–100 mg/L, procalcitonin > 0.3 ng/mL, neutrophil percentage > 75%. A predictive score incorporating these 5 variables has adequate concordance with an area under the curve of 0.85. The model showed low positive predictive value (22.6%), but excellent negative predictive value (97.4%) for predicting the risk of BSI. The risk of BSI reduced to 2.0% in FUO patients if score < 1.5.
A simple tool based on 5 variables is useful for timely ruling out the individuals at low risk of BSI in FUO population.
Bloodstream infection (BSI) remains a common and important cause of death in infected patients. Population-based studies estimated that there were 575,000–677,000 episodes of BSI and 79,000–94,000 deaths per year in North America, more than 1,200,000 episodes of BSI and 157,000 deaths per year in Europe [1, 2]. A good progress has been achieved in the diagnosis and treatment of BSIs, but the mortality associated with some pathogens such as methicillin-resistant Staphylococcus aureus, vancomycin-resistant enterococci, carbapenem-resistant Enterobacteriaceae is still high .
The causes of fever of unknown origin (FUO) have shifted during the past century, but infections remain the leading causes of FUO . BSI may also be a cause underlying FUO. In West China Hospital, 7.13% of the FUO in the patients older than 14 years of age were finally confirmed as BSI . It was also reported that in West Australia, bacteremia was prevalent in 15% of the FUO patients, and 16% of the deaths of FUO patients were attributed to BSI . The diagnosis of BSI poses an especially great challenge in case of FUO. A patient suffering from protracted FUO is usually treated with broad-spectrum antimicrobial agents empirically, which may suppress the growth of fastidious pathogens and confer a predisposition to antibiotic resistance. Therefore, inappropriate empiric antimicrobial therapy may further impede the early diagnosis of BSI in case of FUO, and thus delay effective therapy.
It may be helpful to improve patient outcomes and reduce antibiotic overuse if the clinicians can know the risk of BSI at early stage in FUO patients. Serum inflammatory markers such as C-reactive protein (CRP) and procalcitonin (PCT) are generally used to diagnose BSI and assess the treatment effectiveness in clinical practice . These markers perform well with high sensitivity but poor specificity. DNA-based diagnostics, such as microbial next-generation sequencing (mNGS), which can rapidly detect multiple pathogens, are unfortunately expected to be available only in high-income settings and its real-world performance for BSI diagnosis has been questioned .
To our knowledge, few studies have assessed the risk of BSI in FUO cases and no models have been specifically developed for predicting the risk of BSI in FUO patients. In the present study, we aimed to develop a simple scoring system for predicting the risk of BSI in FUO patients, which can be calculated from readily available routine clinical variables within 24 h after admission.
The FUO patients were consecutively enrolled for analysis at Huashan Hospital, Fudan University, a 2000-bed tertiary care teaching hospital in Shanghai, China. The cohort consisted of retrospectively enrolled patients between January 2014 and June 2017 and those prospectively enrolled between December 2017 and May 2019].
Ethical approval was granted by the Institutional Review Board of Huashan Hospital, Fudan University for prospectively collecting the patient data. All the prospectively enrolled patients signed the informed consent form before data collection. This cohort study was registered at Chinese trial register (www.chictr.org.cn, ChiCTR1800020037) . The principles for transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) were observed in this study  (Additional file 2).
All of the enrolled patients were adults (at least 18 years of age) satisfying the criteria for classic FUO, i.e., body temperature above 38.3 °C on three or more occasions and at least 3-week duration of illness, in which no diagnosis was made after 1 week of hospital admission [11, 12]. The patients were excluded in case of: (1) pregnant; (2) readmission within 3 months after discharge; (3) discharge from hospital before identifying FUO etiology; or (4) missing data of CRP, PCT levels or other relevant test results within 24 h after admission.
BSI was defined by positive blood cultures in a patient with systemic signs of infection, excluding bacteria such as Corynebacterium species which were often associated with blood culture contamination. However, multiple positive cultures for common commensals (e.g., coagulase-negative Staphylococcus) were considered true BSI events if the second culture occurred on the same or subsequent day .
The following clinical and laboratory data were collected, including patient demographics, clinical features, comorbidities, laboratory test results, patient outcomes, and discharge diagnoses. Invasive procedure included endoscopy, puncture, tissue biopsy, hemodialysis and acupuncture. Catheter use including urinary catheter, peripherally inserted central catheter, drainage tube, nasobiliary duct and gastric tube. Implantations included double J tubes, pacemakers, artificial joints, heart valves, artificial ossicles, coronary stents, inferior vena cava filters and fracture internal fixation devices. Corticosteroid use referred to receiving an equivalent dose of ≥ 15 mg prednisone per day; chemotherapy referred to receiving cytotoxic anti-tumor drugs to cure or alleviate cancer.
The results of blood culture within 48 h of admission were also recorded and carefully evaluated. CRP and PCT (Upper Bio-Tech Pharma, Shanghai, China) were measured for all the prospectively enrolled FUO patients within 24 h after admission per the manufacturer’s instruction. The CRP and PCT data within 24 h after admission were retrieved from the electronic case record for the retrospectively enrolled FUO patients.
The continuous random variables were firstly evaluated by Kolmogorov–Smirnov test to verify the normality of their distribution, then compared between groups by Student t-test or Mann–Whitney U-test. The categorical variables were compared between groups by Pearson’s Chi-square test or Fisher’s exact test. The difference was considered statistically significant when the two-sided p value was < 0.05.
The missing data for a variable were subjected to imputation when no more than 20%, otherwise, the variable was removed from the analysis. Predictive mean matching (PMM), Logistic regression, and Bayesian polytomous regression methods were used to impute the missing values for continuous, dichotomous, and categorical variables, respectively. The continuous random variables were evaluated by restricted cubic splines with four knots at the 5th, 35th, 65th, and 95th percentiles of the distributions for their possible non-monotonic relationship to the dependent variable (BSI). The variables monotonously related to BSI were converted to binary variables according to the best cutoff value. Those non-monotonously related variables were converted to multi-category variables according to the spline curve graph. Considering the relatively small number of observed events in this study, least absolute shrinkage and selection operator (LASSO) regression augmented with tenfold cross validation was adopted to minimize the potential collinearity of variables measured from the same patient and over-fitting of variables, which used the λ operator to penalize the absolute value of regression coefficients. Specifically, the larger the value of λ operator would shrink the smaller absolute value of regression coefficient to zero, leaving only the strongest predictors. For model concision, a value of λ with five predictors left for variable screening were selected for this study. The variables identified by LASSO regression analysis were subsequently entered into logistic regression. The statistically significant variables were used to construct the final predictive model.
The receiver operating characteristic (ROC) curve and the corresponding area under the curve (AUC) were used to evaluate model discrimination. Hosmer–Lemeshow test was used to evaluate the model calibration. After internal validation via 1000 repetitions of Bootstrap resampling, the resulted regression model was further simplified to a scoring system according to the method previously described . Calibration plot was used to assess the goodness of fit. A schematic diagram comparing pre- and post-test probability of BSI was presented to assess the clinical utility of the scoring system . All statistical analyses were performed using R statistical software (version 4.0.2, R Foundation) and SPSS software (version 25.0, IBM).
Overall, 527 retrospective FUO patients and 185 prospective FUO patients were enrolled. Fifty-five cases of BSI were identified, including 48 in the retrospective cohort, and 7 in the prospective cohort. The cohort of 712 FUO patients and 55 BSI patients were used to develop the prediction model.
The patients with BSI had higher body temperature than the patients without BSI, and associated more frequently with chills, diabetes mellitus, liver cirrhosis, solid tumor, catheter use, and implant device (Table 1). Single pathogen was finally isolated from each of the 55 patients with BSI by blood culture, including Escherichia coli (36.3%), Streptococcus spp. (10.9%), and Klebsiella pneumoniae (10.9%) (Additional file 1: Table S1).
Performance of serum inflammatory markers in identifying BSI
The levels of CRP (Fig. 1A), PCT (Fig. 1B) and serum ferritin (Fig. 1C) within 24 h after admision are dpicted. BSI was associated with significantly higher CRP (62.80 versus 34.30 mg/L, p < 0.001) and PCT (0.38 versus 0.12 ng/mL, p < 0.001) levels, but significantly lower ferritin level (341.50 versus 535.40 mg/L, p < 0.05) compared with the FUO patients without BSI. The AUC of CRP, PCT, and ferritin for diagnosing BSI was 0.70 (95% CI 0.65–0.75), 0.73 (95% CI 0.66–0.80), and 0.62 (95% CI 0.55–0.69), respectively (Fig. 2B).
Development and utility of the prediction model
The continuous variables age, CRP, and erythrocyte sedimentation rate (ESR) showed non-monotonic U shape association with BSI. These variables were converted to 3-category variables for analysis (Additional file 1: Fig. S1). Only 5 strong predictors remained in the prediction model when the λ of LASSO regression was 0.0415, i.e., presence of diabetes mellitus and chills, neutrophil percentage > 75%, initial CRP level of 50–100 mg/L, and procalcitonin > 0.3 ng/mL within 24 h after admission (Additional file 1: Fig. S2). Multivariate analysis demonstrated that these variables were also independent predictors of BSI (Table 2).
A logistic regression model based on these five predictors showed good discrimination (C-statistic, 0.85) and calibration (Hosmer–Lemeshow Chi-square statistic, 7.02; p > 0.05). Bootstrap simulation suggested no significant overoptimism (optimism-corrected C-statistic, 0.83; calibration slope, 0.91).
Each of these five independent variables was scored according to the corresponding β-coefficient of the variable. The sum of the score was calculated as the risk score of BSI for an individual patient, which ranged from 0 to 6 points (Table 2). The risk score (median, interquartile range [IQR]) was significantly higher among the FUO patients with BSI (3.5, IQR [2.0–4.0]) compared with those without BSI (1, IQR [0.0–2.0]; p < 0.001). The incidence rate of BSI predicted by the risk score was well calibrated with the observed rate (Fig. 2A). This scoring tool could identify BSI appropriately, evidenced by an AUC of 0.85 (95% CI 0.80–0.90) (Fig. 2B). The optimal cut-off value of this scoring system was 2.5 points. Its performance in predicting the risk of BSI among the FUO cohort showed good sensitivity (74.5%) and specificity (78.7%), low positive predictive value (22.6%), but excellent negative predictive value (97.4%). The positive and negative likelihood ratios (LRs) were 3.5 and 0.32, respectively. The pre-test probability (prevalence) of BSI in this FUO cohort was 8.0%. When the risk score is < 1.5, the risk of BSI decreases to 2%, when the risk score is ≥ 4.5, the risk of BSI increases to 48% (Fig. 3).
We developed a novel practical diagnostic instrument to predict the risk of BSI in FUO patients following the TRIPOD guidance. To our knowledge, no such prediction tool has been reported specifically for the FUO patients. The scoring system consisted of five variables routinely assessed for FUO patients in clinical practice: diabetes mellitus, chills, CRP, PCT, and neutrophil percentage. These variables are readily tested or available to clinicians for calculating the risk score of BSI. The newly developed scoring tool was internally validated and showed good discrimination and calibration abilities.
Previous studies have shown that the commonly used inflammatory markers such as PCT and CRP alone are not accurate enough in identifying BSI [16,17,18]. PCT generally showed higher predictive value for BSI than CRP, but its AUC varied (0.70–0.85) when used in different wards and populations [19,20,21]. In our FUO cohort, PCT level > 0.3 ng/mL within 24 h after admission alone provided only a moderate AUC (0.73, 95% CI 0.66–0.80) in identifying BSI, with high specificity to discriminate BSI (80.1%) but low sensitivity (60.0%). CRP alone for diagnosing BSI is far from satisfactory . Serum CRP > 150 mg/L was not an independent risk factor for BSI in febrile adults in emergency department . A retrospective study of FUO population also revealed a higher median level of CRP in connective tissue diseases than in infectious diseases . It’s worth noting that a U-shape relation between CRP and BSI were observed in our cohort (Additional file 1: Fig. S1). More than half (50.9%) of the patients with BSI had CRP levels between 50 and 100 mg/L. However, CRP > 100 mg/L was reported in more cases of non-infectious inflammatory diseases (38 cases) and other bacterial infections (41 cases) than in BSIs (12 cases).
Some authors have used the prediction model combining risk factors and clinical manifestations with inflammatory markers to identify BSI at early stage. Su et al. developed a scoring tool to predict BSI for the patients (> 15 years of age) in the emergency department . Their scoring system adopted serum markers CRP (> 10 mg/dL) and PCT (> 0.5 ng/mL) in addition to the clinical predictors such as fever and sinus tachycardia. The reported AUC for this tool was 0.85 in diagnosing BSI, and 0.79 even in the febrile subset. However, the risk factors of BSI such as invasive procedures and indwelling catheter were not considered as the variables for screening in constructing their model. Recently, some researchers have also used machine learning algorithms to predict BSI. The results varied greatly with the variables included, algorithms, and target populations. Ratzinger et al developed a stochastic forest model using 29 parameters including clinical variables, routine laboratory tests, and cytokines for diagnosing bacteremia in the systemic inflammatory response syndrome (SIRS) population in standard care wards, which resulted in an AUC of 0.74 . Roimi et al. developed a prediction model using the datasets of ICU patients in two hospitals. The model included 50 clinical and laboratory variables. The diagnostic performance (AUC) was 0.87 and 0.93 in internal validation . However, the machine learning approach requires that an electronic database containing all the patient characteristic parameters is available for the algorithm to work, so it is not readily applicable to most of the healthcare settings. In the present study, the scoring system was designed for FUO patients specifically. It is a simple and practical instrument that can be used at bedside. The predictors are well defined, easily measured, and readily available within 24 h after patient admission. It performed well in identifying BSI in FUO patients (AUC, 0.85).
The unnecessary antimicrobial therapy increases the risk of adverse drug reactions and secondary Clostridioides difficile infections, and promotes the spread of antibiotic-resistant bacteria. Inappropriate antibiotic use is a common problem in FUO population. Mert et al. reviewed the records of 20 FUO patients, who finally were diagnosed as adult onset Still disease, and found that 18 of the patients received unnecessary antibiotics prescription . At present, the expert consensus on the diagnosis and treatment of FUO states that FUO without agranulocytosis should not be routinely treated with empirical antimicrobial therapy [4, 28]. However, for BSI, early use of antibiotics can improve the patient outcome. Therefore, identification of an individual at high likelihood of BSI among FUO patients has the potential to help clinicians make reasonable treatment decision. This scoring tool can help clinicians assess the possibility of BSI in patients with typical FUO within 24 h of admission and decide whether anti-infective treatment needs to be initiated immediately. In our center, the pre-test probability (prevalence) of BSI is 8.0% among FUO patients. Therefore, if the score of an individual is less than 1.5 after admission, his/her risk of BSI is very low (only 2%), suggesting that doctors can consider suspending antibiotic prescription.
This study has some limitations. More than 700 FUO cases were included over 5 years, but there are still few cases of BSI. The results were from a single center and have not been externally verified. Therefore, it is required to further evaluate the utility of this scoring tool in other medical institutions. Other relevant markers such as IL-6 were also used in the diagnosis of severe infections, but they were not routinely tested for FUO patients in the study center at that time, so they were not included as possible variables in the screening. The diagnostic performance of this model is unclear for the fungal BSI in FUO cases because no fungal pathogen was isolated in this study.
In summary, a simple scoring system was constructed with a combination of five variables: diabetes mellitus, chills, CRP (50–100 mg/L), PCT (> 0.3 ng/mL), and neutrophil percentage (> 75%) in the cohort of 712 FUO patients. The tool is helpful for clinicians to identify the FUO patients at low risk of BSI, and so avoid inappropriate antibiotic use. The clinical utility of this scoring tool still needs to be confirmed by external validation with larger sample size.
Timsit JF, Ruppé E, Barbier F, Tabah A, Bassetti M. Bloodstream infections in critically ill patients: an expert statement. Intensive Care Med. 2020;46:266–84.
Goto M, Al-Hasan MN. Overall burden of bloodstream infection and nosocomial bloodstream infection in North America and Europe. Clin Microbiol Infect. 2013;19:501–9.
Kern WV, Rieg S. Burden of bacterial bloodstream infection—a brief update on epidemiology and significance of multidrug-resistant pathogens. Clin Microbiol Infect. 2020;26:151–7.
Ghady H, Nina S. Fever of unknown origin. N Engl J Med. 2022;386:463–77.
Zhou G, Zhou Y, Zhong C, et al. Retrospective analysis of 1,641 cases of classic fever of unknown origin. Ann Transl Med. 2020;8:690.
Ingarfield SL, Celenza A, Jacobs IG, Riley TV. Outcomes in patients with an emergency department diagnosis of fever of unknown origin. Emerg Med Australas. 2007;19:105–12.
von Dach E, Albrich WC, Brunel AS, et al. Effect of C-reactive protein-guided antibiotic treatment duration, 7-day treatment, or 14-day treatment on 30-day clinical failure rate in patients with uncomplicated gram-negative bacteremia: a randomized clinical trial. JAMA. 2020;323:2160–9.
Babady NE. Clinical metagenomics for bloodstream infections: is the juice worth the squeeze? Clin Infect Dis. 2021;72:246–8.
Xu T, Wang L, Wu S, Fenfen Z, Huang H. Utility of a simple scoring system in differentiating bacterial infections in cases of fever of unknown origin. Clin Infect Dis. 2020;71:409–15.
Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594.
Petersdorf RG, Beeson PB. Fever of unexplained origin: report on 100 cases. Medicine. 1961;40:1–30.
Durack DT, Street AC. Fever of unknown origin–reexamined and redefined. Curr Clin Top Infect Dis. 1991;11:35–51.
National Healthcare Safety Network. Centers for disease control and prevention web page. Central line-associated bloodstream infection (CLABSI) and non-central line-associated bloodstream infection. https://www.cdc.gov/nhsn/PDFs/pscManual/4PSC_CLABScurrent.pdf. Accessed 23 Jan 2020.
Sullivan LM, Massaro JM, D’Agostino RB. Presentation of multivariate data for clinical use: the Framingham study risk score functions. Stat Med. 2004;23:1631–60.
Fanshawe TR, Power M, Graziadio S, Ordóñez-Mena JM, Simpson J, Allen J. Interactive visualisation for interpreting diagnostic test accuracy study results. BMJ Evid Based Med. 2018;23:13–6.
Lee CC, Hong MY, Lee NY, Chen PL, Chang CM, Ko WC. Pitfalls in using serum C-reactive protein to predict bacteremia in febrile adults in the ED. Am J Emerg Med. 2012;30:562–9.
Debard AL, Vautrin C, Pariset C, Bienvenu J, Monneret G. High serum procalcitonin levels do not predict bacteremia in adult patients with acute fever. Clin Infect Dis. 2003;36:825–6.
Hoeboer SH, van der Geest PJ, Nieboer D, Groeneveld AB. The diagnostic accuracy of procalcitonin for bacteraemia: a systematic review and meta-analysis. Clin Microbiol Infect. 2015;21:474–81.
Marik PE, Stephenson E. The ability of Procalcitonin, lactate, white blood cell count and neutrophil-lymphocyte count ratio to predict blood stream infection. Analysis of a large database. J Crit Care. 2020;60:135–9.
Oussalah A, Ferrand J, Filhine-Tresarrieu P, et al. Diagnostic accuracy of procalcitonin for predicting blood culture results in patients with suspected bloodstream infection: an observational study of 35,343 consecutive patients (a STROBE-compliant article). Medicine. 2015;94:e1774.
Wu Q, Yang H, Kang Y. Comparison of diagnostic accuracy among procalcitonin, C-reactive protein, and interleukin 6 for blood culture positivity in general ICU patients. Crit Care. 2018;22:339.
Bassetti M, Russo A, Righi E, et al. Comparison between procalcitonin and C-reactive protein to predict blood culture results in ICU patients. Crit Care. 2018;22:252.
Liu CP, Liu ZY, Liu JP, Kang Y, Mao CS, Shang J. Diagnostic value of common inflammatory markers on fever of unknown origin. Jpn J Infect Dis. 2016;69:378–83.
Su CP, Chen TH, Chen SY, et al. Predictive model for bacteremia in adult patients with blood cultures performed at the emergency department: a preliminary report. J Microbiol Immunol Infect. 2011;44:449–55.
Ratzinger F, Haslacher H, Perkmann T, et al. Machine learning for fast identification of bacteraemia in SIRS patients treated on standard care wards: a cohort study. Sci Rep. 2018;8:12233.
Roimi M, Neuberger A, Shrot A, Paul M, Geffen Y, Bar-Lavie Y. Early diagnosis of bloodstream infections in the intensive care unit using machine-learning algorithms. Intensive Care Med. 2020;46:454–62.
Mert A, Ozaras R, Tabak F, et al. Fever of unknown origin: a review of 20 patients with adult-onset Still's disease. Clin Rheumatol. 2003;22:89–93.
Wright WF, Auwaerter PG. Fever and fever of unknown origin: review, recent advances, and lingering dogma. Open Forum Infect Dis. 2020;7:ofaa132.
The authors would like to thank Jiyuan Hu, the Division of Biostatistics, Department of Population Health at New York University School of Medicine for collaboration in the statistical analysis.
This work was supported by Shanghai Science and Technology Committee (Grant Nos. 16441900402, 21ZR1410800) and the National Natural Science Foundation of China (Grant No. 81973369).
All authors report no conflicts of interest relevant to this article. The funders had no role in study design, data collection and analysis, or decision to publish.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The organismsisolated from blood culture from 55 cases of bloodstream infection in the modeldevelopment cohort. Figure S1. Association betweencontinuous variables and BSI in FUO patients demonstrated by restricted cubicsplines (RCS). BSI bloodstream infection,ESR erythrocyte sedimentation rate, FUO fever of unknown origin, Nper neutrophil percentage, temp body temperature, WBC white blood cell count. FigureS2. Variables selectionusing the least absolute shrinkage and selection operator (LASSO) binarylogistic regression model. Panel A. LASSO coefficient profiles of the candidatevariables. Panel B. Tuning parameter (λ) selection in the LASSO model using tenfoldcross-validation via minimum criteria.
TRIPOD checklist:prediction model development and validation.
About this article
Cite this article
Xu, T., Wu, S., Li, J. et al. Development of a risk prediction model for bloodstream infection in patients with fever of unknown origin. J Transl Med 20, 575 (2022). https://doi.org/10.1186/s12967-022-03796-8
- Bloodstream infection
- Fever of unknown origin
- Serum inflammatory marker
- Scoring system
- Diagnostic performance