Comparing the predictive performance of in-hospital mortality of different frailty scales for elderly patients in the emergency department

1.1 Representative scales for frailty, disability and comorbidity

Frailty, disability and comorbidity are used to describe vulnerable older adults. Each of these terms implies different clinical relevance and therapeutic considerations. Frailty indicated a potential risk to adverse health outcomes, including falls, dependence of daily life activities, morbidity and mortality, etc. Disability is a physical or mental condition that limits one’s self-care tasks, and independence of daily life activities. Comorbidity is defined as the presence of two or more medically diagnosed diseases in the same patient [6].

Several tools, such as the Eastern Cooperative Oncology Group Performance Status Scale (ECOG), Charlson Comorbidity Index (CCI) and Clinical Frailty Scale (CFS), are used to evaluate frailty, disability and comorbidity in elderly patients [7, 8, 9]. The introduction and application of those scales in the ED setting were reviewed and summarized as below. Launched in 1982, ECOG mainly measures self-care independence, everyday activities and physical capacities. Originally intended to quantify the performance status of cancer patients [10], it has since been validated for predicting in-hospital mortality in ED patients, including those with various conditions such as neutropenic fever, pneumonia and other emergent medical problems [11, 12]. ECOG is also a practical tool for evaluating the level of disability in an ED setting.

The CCI, established by Charlson in 1987, estimates the risk of one-year mortality following hospitalization due to any cause [13, 14, 15]. Unlike the ECOG scale, this revised index encompasses a wide array of comorbidities, including solid tumors and malignancies, and is adjusted for patient age. As a respected and extensively utilized tool, the CCI is adept at assessing comorbidities in both chronic and acute conditions [8, 15]. Previous studies have successfully demonstrated the applicability of the CCI in ED settings for cases involving sepsis, brain injury and patients from aged care facilities [16, 17].

The CFS, another assessment tool, was conceptualized for the Canadian Study of Health and Aging. This scale measures multiple domains, such as comorbidity, cognition and function [9]. Prior findings have corroborated its predictive capability for mortality within an ED context [18]. It is necessary to emphasize that these tools were not initially created for emergency situations, so their validation for such applications may not be complete. Therefore, a clear consensus on which frailty scale is best for risk stratification in the ED is yet to be established [19].

1.2 Study goal

We aimed to improve ED care for the elderly by identifying the best tool to aid clinicians and streamline multidisciplinary diagnosis and treatment. We evaluated the predictive accuracy of the CFS, CCI and Eastern Cooperative Oncology Group (ECOG) in predicting frailty, comorbidity and disability in an ED setting.

2.1 Study design and setting

This prospective cohort study, utilizing convenience sampling, was carried out at the EDs of two hospitals in Taoyuan City, Taiwan, from October 2020 to June 2021. The hospitals were a university-affiliated medical center and a local teaching hospital, handling approximately 90,000 and 150,000 ED visits annually. Around 25% of these visits were made by senior patients, those 65 years or older. Meanwhile, critical conditions having triage levels II and I represented 20% and 17% of ED visits, respectively.

2.2 Patient enrollment

We included patients aged 65 or older who presented at the ED during our study period. We excluded terminally ill patients receiving palliative care, patients who suffered cardiac arrest upon arrival or during their ED stay, patients who declined to give consent and patients presenting with traumatic injuries.

2.3 Study protocol

Elderly patients who met the inclusion criteria were identified by research assistants during ED triage. Either the patients themselves or their legal representatives gave informed consent. Families and patients were interviewed and filled out questionnaires to provide information for the CFS, CCI and ECOG assessment tools. A standardized questionnaire was employed to gauge the patients’ daily activity and performance levels. Then, using a standardized, clearly defined reporting template, medical records were reviewed to gather data on patient characteristics and outcomes.

The ECOG can be conveniently used in EDs, with scores ranging from 0 (asymptomatic) to 5 (death). The CFS creates a frailty score that ranges from 1 (very fit) to 9 (terminally ill). In Taiwan, the Taiwan Triage and Acuity Scale (TTAS) was crafted by adapting the language of the Canadian Triage Acuity Scale (CTAS). A previous study affirmed its validity and showed it has a similar performance to CTAS [20]. The CCI consists of 16 elements, some weighted by severity and age, and has a scoring range from 0 to 33 points.

2.4 Study parameters

The following information about participants’ ED visits was collected from chart reviews: demographics (such as age, sex and triage category), comorbidities (including myocardial infarction, congestive heart failure, peripheral vascular disease, stroke, dementia, chronic obstructive pulmonary disease, connective tissue disease, peptic ulcer disease, liver disease, diabetes mellitus, hemiplegia, moderate to severe chronic kidney disease, high creatinine levels (>3 mg/dL), solid tumors, leukemia, lymphoma and acquired immune deficiency syndrome), patient disposition (like discharge, admission to a ward or intensive care unit, or surgery) and the length of the hospital stay. In addition, detailed questionnaires were developed to gather data about ECOG and CFS.

2.5 Outcomes

The primary objective of this study was to assess patient mortality during hospitalization. Secondary objectives included examining admission rates following ED visits and the average hospital length of stay (LOS). We then evaluated the scales’ effectiveness in predicting these primary and secondary outcomes.

2.6 Statistical analysis

We conducted data analysis via IBM SPSS Statistics (version 25; IBM Corp., Armonk, NY, USA). We made comparisons between patient outcomes and demographic features across various groups. We articulated categorical variables as quantities and percentages for every subgroup, while continuous variables were expressed as means and standard deviations. For comparing categorical variables, we employed either Pearson’s chi-square or Fisher’s exact test, as needed.

Continuous variables were compared using the Student’s t-test or Mann-Whitney U-test. Scale-predicted outcomes were plotted on receiver operating characteristic (ROC) curves to calculate the area under the curve (AUC). The AUCs for different scales were subsequently compared. A p-value < 0.05 was considered statistically significant.

3.1 Background and clinical characteristics

Of the 614 patients aged 65 years and older admitted to the ED during the study period, 593 were involved in the study. We excluded 20 patients because of their trauma history and one due to a terminal condition under palliative care (Fig.1). Females made up 48.9% of the study participants, with the average age being 75.8 years (Table 1).

Fig. 1.Participants enrolled. ED: emergency department.

Most participants (74.4%) fell into the TTAS level III category, while 114 (19.2%) were identified as critical under TTAS levels II or I. Diabetes was the most prevalent comorbidity at 43.7%, followed by peptic ulcer disease at 32.4% and heart disease at 29.2%. Autoimmune diseases were the least common comorbidity, appearing in only 3.5% of the patients. Participants’ average scores for the ECOG, CFS and CCI scales were 2.0, 3.9 and 4.3, respectively.

In terms of ED disposition, 383 (64.5%) patients required hospital admission and 26 (4.4%) did not survive the admission. On average, patients stayed in the hospital for 14.4 days.

3.2 Cut-off values for different scales

The CFS identified 274 patients as frail, each with a score greater than 4. The ECOG-PS identified 215 patients as disabled, each with a score greater than 2. The CCI identified 356 patients with high comorbidities, each with a score greater than 2. Patients categorized as frail, disabled or with high comorbidities generally had higher admission (ECOG >3: 71.6 vs. 60.6%, p = 0.007; CFS >4: 70.8 vs. 59.2%, p = 0.003; CCI >2: 69.7 vs. 57.0%, p = 0.002) and mortality rates (ECOG >3: 6.5% vs. 3.2%, p = 0.056; CFS >4: 7.3% vs. 1.9%,p = 0.001; CCI >2: 7.0% vs. 0.4%, p < 0.001). Admission and mortality rate remained significant associated with three scales above cut-off value after adjusted to age, gender and TTAs by logistic regression analysis, except for ECOG >3 group in predicting admission to hospital (p = 0.055)—as shown in Table 2. There was no significant difference of hospital length of stay in survivors among all scales above cut-off value (ECOG >3: 15.3 vs. 15.4 days, p = 0.935; CFS >4: 14.4 vs. 16.2 days, p = 0.315; CCI >2: 15.9 vs. 14.8 days, p = 0.547).

3.3 Predictability of patient outcomes for different scales

Fig.2 illustrates the ROC curves for various scales predicting mortality and hospital admission. The CCI significantly outperformed the others in mortality predictability, boasting an AUC of 0.810 (95% CI: 0.730–0.891). It was followed by the CFS and Eastern Cooperative Oncology Group Performance Status (ECOG-PS) with AUCs of 0.706 (95% CI: 0.614–0.799) and 0.660 (95% CI: 0.565–0.754) respectively. However, there was no significant difference between the scales in terms of predicting hospital admission, with AUCs for CCI, CFS and ECOG-PS being 0.600 (95% CI: 0.552–0.648), 0.583 (95% CI: 0.535–0.632) and 0.580 (95% CI: 0.531–0.628) respectively.

Fig. 2.The ROC curves for various scales predicting mortality and hospital admission. (A) AUC to predict patient mortality of different scales. (B) AUC to predict patient admission to hospital of different scales. ECOGPS, Eastern Cooperative Oncology Group Performance Status; ROC, receiver operating characteristic; CCI, Charlson Comorbidity Index; CFS, Clinical Frailty Scale.

4.1 CCI predicts in-hospital mortality

Taiwan is grappling with a pressing issue of a rapidly aging population. According to Taiwan’s Ministry of Interior, individuals aged 65 years and older represent 16.9% of the population, a rate expected to increase to 24.0% by 2030 [21]. This rate is nearly 1.5 times higher than the worldwide average [1]. This evolving demographic has led to the launch of a comprehensive strategy for elderly care. The ED is of critical importance in offering timely medical care for aged patients. Currently, there is no agreed-upon role for age in triage scales. A thorough evaluation of physiological reserves may be more suitable than just considering chronological age. This research compared the effectiveness of various frailty scales in the ED, including ECOG-PS, CCI and CFS. Our findings suggested that CCI was the most reliable scale for predicting in-hospital mortality for older patients at ED triage. The possibility of using CCI to guide patient management is worth further study.

4.2 Interpretation of the study results

Our study findings can be attributed to several factors. Firstly, the design and intended applications of the frailty scales differ significantly. ECOG, for instance, was originally created to gauge daily activity performance in cancer patients. Ahn et al. [12] utilized ECOG to formulate prognostic models that predicted mortality in specific patient types, such as those with febrile neutropenia and pneumonia-afflicted cancer patients, in ED. These studies validated ECOG’s effectiveness with these specific patient groups, but its efficacy may decrease when extended to a wider ED patient demographic. Some studies also included abnormal vital signs and physiological data with ECOG to improve prediction accuracy.

Junhasavasdikul et al. [11] discovered that the delta Modified Early Warning Score bore a more significant correlation with ED fatality than ECOG. Our study, with a larger sample size of 593, demonstrated a lower mortality rate of 4.4% compared to the 8.9% in Junhasavasdikul et al.’s [11] research. In our results, ECOG solely indicated an AUC of 0.660 (95% CI: 0.565–0.754) in predicting ED mortality. Consequently, both our investigation and past studies have cast doubts about the effectiveness of ECOG as a predictor of mortality in ED.

In contrast, the CFS unifies performance indicators and comorbidity categories into one rating scale. An exploratory review of 183 original studies on the CFS, obtained via online databases, showed its correlation with comorbidity and complications in 73% and 100% of cases, respectively, across various environments. However, less than 10% of the studies in this review pertained to ED settings [9]. Elliot et al. [18] conducted a forward-looking observational study in the ED, investigating 138,328 patients over 2 years to assess the CFS’s predictive capability for mortality and hospitalization. The adjusted hazard ratio for mortality was 3.6 for CFS 7 to 8 compared with score 1 to 3.

This study also showed an acceptable predictive performance for the CFS in assessing hospital mortality, with an AUC of 0.706 (95% CI: 0.61–0.80).

The CCI, incorporating a thorough range of comorbidity categories for analysis, outperformed both ECOG and CFS in predicting mortality (AUC: 0.810; 95% CI: 0.730–0.891). Quan et al. [14] further refined and validated the CCI score using data from various countries, such as Canada, Australia, New Zealand, Switzerland and France. Their results proved promising in discriminating in-hospital mortality, with AUCs consistently above 0.8. Past research has confirmed CCI’s validity in predicting mortality among in-hospital cardiac arrest, ischemic stroke and dialysis patients [22, 23, 24]. The precise categorization of comorbidity severity, determined by a point system from 0 to 33, makes CCI a more accurate and objective tool. It, therefore, fits better for describing patient frailty across diverse disease categories and predicting mortality in ED settings.

4.3 Limitations

Our study bears several limitations. It was limited to two hospitals in the same area of Taoyuan City, Taiwan, from October 2020 to June 2021, which may limit the applicability of the results to other locations. Also, potential selection bias may have occurred since all data were collected by a single assistant during office hours using convenience sampling. Uncontrolled variables such as lab findings, imaging results, treatment options, familial support and willingness for invasive procedures could have affected the results. We have evaluated several factors relevant to our study, yet some, correlating with patient health and eventual results, were overlooked. In addition, all data is based on patient or family reporting this would be a source of bias. It is crucial to recognize that collecting data from patients and their families is frequently the most practical method in clinical research, especially considering the emphasis of this study on patient experiences and perceptions.

Lastly, the exclusion of cardiac arrest and patients under palliative care may have resulted in underestimating the performance of the various scales. Regardless, the wider scope of elderly ED visits validated the performance of these scales.