Evaluation of mortality prediction models for oncology patients in intensive care unit

1. Introduction

The Acute Physiology and Chronic Health Evaluation-2 (APACHE-2) score is a valid and widely recognized tool for predicting mortality in intensive care unit (ICU) settings [1]. It is calculated based on the worst physiological parameters recorded within the first 24 hours of admission. Similarly, the Simplified Acute Physiology Score-3 (SAPS-3), is another model designed to estimate in-hospital mortality with frequent use in cancer patient prognosis [2]. In addition to these models, the Sequential Organ Failure Assessment (SOFA) and National Early Warning Score (NEWS), are commonly employed for measuring disease severity organ dysfunction, and prognosis in ICU patients [1, 3]. SOFA and NEWS primarily focus on sepsis and include six clinical and laboratory parameters, while APACHE-2 and SAPS-3 evaluate 14 and 20 variables, respectively, for disease severity and mortality prediction [1].

Cancer represents a significant global health issue, with increasing new cancer cases and cancer-related deaths in recent years [4, 5, 6]. Advances in cancer treatment, including the use of immunotherapy, targeted therapies and screening tests have contributed to improved survival rates [7, 8]. However, the likelihood of ICU admission for oncology patients is also on the rise, as the prognosis for ICU care varies among cancer patients. By setting appropriate ICU we can ensure the efficient use of ICU beds [9, 10]. Scoring systems play a key role in determining prognosis [11, 12], but their performance in specific populations, such as cancer patients remains uncertain [13, 14, 15].

This study, aims to evaluate the 28-day mortality rates of cancer patients in ICU and assess the predictive value of scoring systems in this patient population.

2. Materials and methods

The study was conducted retrospective observational analysis of patients monitored in the 48-bed third-level ICU of the D Block at Başakşehir Cam and Sakura City Hospital. A total of 593 ICU patients hospitalized between April 2023 and October 2023 were examined. Patients were categorized based on their oncology and hematology follow-up status. If a patient had undergone surgery for cancer in the past five years, they were classified as a solid cancer patient. Planned ICU admissions for operations related to solid cancers were monitored in the Post-Anesthesia Care Unit. This group was not included in the study. Among the patients, 149 adult oncologic malignancy patients with ICU stays of more than 24 hours were included in the analysis. Patients who were not receiving active treatment due to poor prognosis or were under palliative care from the oncology or hematology departments were excluded resulting in 91 patients being included in the final analysis (Fig. 1).

Fig. 1.Flow chart.

The demographic data and treatment histories of the patients were recorded, along with information on cancer diagnosis and treatment. APACHE-2 scores were calculated based on the worst parameters within the first 24 hours of ICU admission. For patients with solid cancer who had undergone surgery, the APACHE-2 postoperative mortality rate was used; otherwise, the non-operative mortality rate was applied. SAPS-3, NEWS and SOFA scores were retrospectively calculated for each patient based on hospital records. SAPS-3 mortality rate was calculated using the mathematical algorithm as it was in the original study [16]. In SOFA, the mortality rates corresponding to the initial SOFA score were used [17]. In cases where laboratory data or comorbidities were missing, they were considered “normal” or “absent”. The primary outcome was defined as 28-day ICU mortality.

Statistical analysis, was performed using the Shapiro-Wilk test for normality. Independent samples t-tests and one-way analysis of variance (ANOVA) tests were used to compare numerical measurements across groups. Post-hoc analyses were conducted using the least significant difference test. Non-parametric tests, such as the Mann-Whitney U test, and Kruskal-Wallis test were employed for non-normally distributed data. Categorical variables were compared using chi-square and Fisher’s exact test. Receiver Operating Characteristic (ROC) analysis was used to determine cutoff values. The Area Under the ROC curve (AUROC) was used to evaluate the discriminative performance of the models. A model was classified as fair if the AUROC ranged from 0.7 to 0.8, good if between 0.8 and 0.9, and excellent if above 0.9. With p-values less than 0.05 considered statistically significant. The calibration of the prognostic models was evaluated using the Hosmer-Lemeshow C statistic. Lower Hosmer-Lemeshow χ² values and higher p-values (>0.05) indicate a good model fit. The standardized mortality ratio (SMR) was calculated for each model by dividing the observed mortality by the mortality expected according to the predictive model.

3. Result

Of the 91 patients included in the study, 53 were male (58%), with a mean age of 60.3 ± 14 years, and 38 were female (42%), with a mean age of 60.1 ± 14 years. The deceased and surviving patients had similar demographic characteristics (Table 1). Hematologic cancers, lung cancers, gynecologic cancers and gastrointestinal cancers were the most common cancer types observed. Among the primary cancer patients, 44 were admitted due to sepsis, and 49 were admitted due to acute renal failure. Invasive mechanical ventilation was administered to 37 patients. The overall 28-day ICU mortality rate was 59%.

Significant differences in APACHE-2, SAPS-3, NEWS and SOFA scores were observed between deceased and surviving patients (Table 2). Deceased patients had significantly higher scores in all four systems. The following values were noted for deceased and surviving patients, respectively: APACHE-2 (29.1 ± 6.7 vs. 26.4 ± 5, p = 0.042), SAPS-3 (90 ± 16.2 vs. 68.8 ± 11.2, p = 0.001), NEWS (9.7 ± 3 vs. 8.3 ± 2.5, p = 0.019), and SOFA (9 ± 4.1 vs. 5.1 ± 3, p = 0.001). The predicted mortality rates were calculated as follows: 57.5 ± 19.2 for APACHE-2, 68.3 ± 22.1 for SAPS-3, and 36.1 ± 30.5 for SOFA. In deceased patients, the mortality rates for SAPS-3 and SOFA were significantly higher, whereas the APACHE mortality rate did not show statistical significance. APACHE-2 and SOFA scores (with SMR value of 1.026 and 1.553, respectively) were underestimated the mortality, while SAPS-3 (with SMR value of 0.868) was overestimated it (Table 2). To determine the association between each scoring system and mortality, AUROC was calculated (Fig. 2). Mortality prediction using SAPS-3 and SOFA showed significant discriminatory power (AUROC = 0.857 and 0.783, respectively). APACHE-2, SOFA, NEWS and SAPS-3 exhibited accurate calibration performance (p > 0.05) (Table 3). Cutoff values were determined for each score to predict mortality as follows: >28.5 for APACHE-2, >76 for SAPS-3, >6.5 for SOFA, and >10.5 for NEWS. It was found that high SOFA and NEWS scores had high sensitivity (81.1% and 86.5%, respectively) in predicting mortality, whereas only SAPS-3 showed high specificity (83%) (Table 3).

Fig. 2.Receiver operating characteristic curve. APACHE: Acute Physiology and Chronic Health Evaluation; SAPS: Simplified Acute Physiology Score; NEWS: National Early Warning Score; SOFA: Sequential Organ Failure Assessment; ROC: Receiver Operating Characteristic.

4. Discussion

In our study, which evaluated the discrimination and calibration of scoring systems for cancer patients in ICU, we found that SAPS-3 had higher discriminatory power in predicting 28-day mortality compared to APACHE-2, SOFA and NEWS scores. SAPS-3 overestimated the mortality rates, but its calibration was deemed acceptable for all the systems analyzed.

Scoring systems are valuable tools that aid clinicians in predicting mortality and determining the need for ICU admission. An ideal scoring system should be easy to apply well-calibrated, have a high predictive value, be applicable across all populations, and provide insights into post-discharge functional status [11]. However, no current scoring system including APACHE, SAPS, SOFA or the Mortality Probability Model meets all of these criteria [1, 18].

Acute respiratory failure in cancer patients can result from a variety of causes, including the tumor’s local effects, pneumonia, acute respiratory distress syndrome (ARDS) or congestive heart failure [19, 20]. Including conditions like respiratory failure and sepsis in the SAPS-3 score may contribute to a more reliable prognosis prediction. In our study, SAPS-3 was found to have the highest discriminatory ability among cancer patients.

The heterogeneity of the cancer population, variation in cancer, types, oncological treatments, diverse reasons for ICU admission and differing end-of-life decisions have led to reported ICU mortality rates for cancer patients ranging from 20% to 50% [20, 21]. In our study, the mortality rate was 59%, with predicted mortality rates of 57.5 ± 19.2 for APACHE-2 and 68.3 ± 22.1 for SAPS-3. This higher mortality rate may link to the absence of end-of-life decisions and the allocation of limited ICU beds to cancer patients with poor prognoses in our country. Early ICU admission for high-risk hematologic patients without organ dysfunction has been shown to improve outcomes [22, 23].

In our study, SAPS-3 and SOFA demonstrated high accuracy in predicting 28-day ICU mortality in cancer patients (with AUROC values of 0.857 and 0.783, respectively). Discrimination was lower for APACHE-2 and NEWS (AUROC of 0.623 and 0.631, respectively). SOFA showed high accuracy in the early diagnosis of sepsis in the emergency department (AUROC = 0.866) and had high sensitivity and specificity (89% and 83%, respectively) in confirming sepsis with a cut-off value of ≥6 [24]. In our study, a cut-off of >6.5 for SOFA was found for predicting high mortality in the ICU (AUROC of 0.783). Compared to emergency department patients, the accuracy of scoring systems may vary in the case of critical illness in the ICU (SOFA median 4 vs. 6). In a study conducted in a surgery and trauma critical care unit, APACHE-4 (AUROC of 0.766) was found to predict mortality better than SAPS-3 and SOFA (AUROC of 0.716 and 0.734, respectively) [25]. The patients in the surgery and trauma critical care unit were younger on average compared to those in our study (mean age of 48.5 ± 19.8 vs. 60.3 ± 14.3). These findings emphasize the need to assess the discrimination and calibration of scoring systems in different patient groups. The NEWS scoring system identifies a score of ≥7 as high risk. However, in our study, the threshold value for NEWS in cancer patients was found to be 10.5 (sensitivity 86%). As our study, is retrospective has a limited number of patients, and does not represent a fully homogeneous population of cancer patients, it is not suited for recommending NEWS or SOFA-based ICU admission standards for this patient group. Admission criteria for cancer patients to the ICU remain uncertain [26]. Prospective studies are needed to evaluate scoring systems that can identify high-risk cancer patients.

A study by Sakr et al. [2], comparing the performance of APACHE-2 and SAPS-3 in postoperative patients yielded results similar to ours. The AUROC values showed weak correlation between SAPS-3 and APACHE-2 (0.84 and 0.78, respectively). While SAPS-3 exhibited comparable results in both studies, APACHE-2 (AUROC of 0.623) demonstrated lower discriminatory power for cancer patients in our analysis. In a large-scale study (n = 12,691), Zhu et al. [27] demonstrated that SAPS-3 had the best discrimination for 28-day mortality in sepsis patients (AUROC of 0.812).

In a study by Soares et al. [28], the SAPS-3 scoring system was evaluated for its prognostic ability in cancer patients. Their results showed that SAPS-3 underestimated mortality (SMR of 1.19), whereas in our study, SAPS-3 overestimated mortality (SMR <1). The average than the score in our study was 81.3 ± 17.7. Which is higher SAPS-3 scores reported by Soares et al. [28]. The prognostic accuracy of scoring systems is influenced by both patient-specific and center-specific factors.

The limitations of our study include its design, small sample size, and lack of representation of a fully complete and homogeneous cancer patient population. Additionally, the inclusion of a large number of oncology patients with poor prognoses due to the comprehensive and high-capacity nature of our hospital, may have influenced our results.

5. Conclusions

SAPS-3 score showed the best discrimination capacity for 28-day ICU mortality of cancer patients compared with the other models. However, SAPS-3 despite being more complex, offers a more comprehensive evaluation of comorbidities and appears to be a superior tool for predicting mortality in oncology and hematology.

The development of patient-centered scoring system is crucial for improving ICU care standards and optimizing resource use. The search for an ideal prognostic system should continue.

Availability of data and materials

The data presented in this study are available on reasonable request from the corresponding author.

Author contributions

FK and DG—designed the research study. DG and AK—analysed the data. FK, DG and GT—wrote the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Before the study, ethic approval was received from the Human Research Ethics Committee at the Basaksehir Cam and Sakura City Hospital, University of Health Sciences (Ethics Approval number: KAEK/2024.01.33). Our study is retrospective, and the need for informed consent was waived by the ethics committee, which approved our study.

Acknowledgment

Not applicable.

Funding

All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. The authors indicate no financial support or financial conflict of interest. The authors have indicated they have no financial relationships with any company and no external funding.

Conflict of interest

The authors declare no conflict of interest.