Key process and outcome indicators on quality and safety of care for critically ill pediatric patients according to international standard organization protocols: a four-year follow-up study

2.1 Setting

An eight-bed tertiary PICU at Hippokration General Hospital of Thessaloniki, Greece, founded in 1999, to serve all critically ill children aged 35 days to 16 years old, from the wider region of Northern Greece. All diagnostic categories are admitted, apart from cardiac surgery patients and patients with severe burns (>10% Body Surface Area), due to lack of support of the corresponding surgical specialties, serviced by one director and seven consultants, four of whom are certified in intensive care, with 24/7 consultant presence. All PICU modalities are available in the unit, apart from Extra Corporeal Membrane Oxygenation. Nurse to patient ratio ranges from 1:2 in morning shifts to 1:3 or 1:4 in rest shifts, according to PICU occupancy rate which fluctuates around 70%.

2.2 Key Process and Outcome Indicators (KPOIs)

On occasion of PICU ISO implementation, ESICM indicators were adopted as part of PICU performance evaluation. Seven out of nine ESICM indicators were chosen. Two structure indicators—that ICUs fulfil national requirements to provide intensive care and 24 h consultant availability—were requirements for the creation of the PICU and considered standards of care. As a result, we focused our study on five KPOIs; one process indicator—Standardized Handover procedure for discharging patients, and four outcome indicators; namely (a) reporting and analysis of Standardized Mortality Ratio (SMR), e.g., the ratio of the observed to predicted mortality rate, (b) ICU readmission rate within 48 h of ICU discharge, (c) the rate of central venous Catheter Related Blood Stream Infection (CRBSI), and (d) the rate of unplanned endotracheal extubations.

2.3 Definitions

2.4 Study design

The study was designed as an observational four-year (01 January 2017—31 December 2020) cohort study. Three indicators; namely SMR, PICU readmission rate within 48 h of discharge, and the rates of CRBSI/CLABSI were examined during the first two years (2017–2018) to set baseline values, and their performance were continually evaluated (2019–2020) as well. Two more indicators, the standardized electronic handover procedure for discharging patients and the rate of unplanned extubations were prospectively added in 2019 and the more detailed index of number of unplanned extubations:1000 days of mechanical ventilation was further assessed for 2020.

2.5 Exclusion criteria

Readmissions were excluded as happened in SMR evaluation of PRISM ΙΙΙ-24 and ESICM indicators. However, early readmissions only, were included for the evaluation of readmission rate within 48 h of discharge, as the ESICM indicator.

2.6 Data collected

Demographic data, reasons for admissions, PRISMIII-24, the presence of comorbidities, the need for mechanical ventilation and mechanical ventilation days, the need for inotropic support, LOS, CRBSI and CLABSI rate, readmission <48 h rate, unplanned extubations, the outcome at discharge and the SMR were collected for the above time periods.

2.7 Statistical analysis

Categorical variables were described as percentages (%), while continuous variables as mean ± SD or median (IQR), as appropriate, according to normal Kolmogorov-Smirnoff testing. Differences between patients who died or survived were sought with t-test for independent samples and/or Mann Whitney test for continuous variables, whereas for the univariate analysis between categorical variables and the dependent variable (death), the χ² test was applied. The 2-tailed significance level was set at 0.05. Data were analyzed using IBM SPSS Statistics 22.0 (Armonk, NY, USA).

3.1 Overall evaluation of the population studied

Among 441 consecutive admissions during the study period, 63 were excluded as readmissions, and 378 children remained for evaluation. There were 208 boys (55%) and 170 girls (45%) with a median (IQR) age of 46 (10–108) months (Table 1). The main reasons for admission were: respiratory failure (21.2%), postoperative care (20.9%), neurologic failure (12.7%), trauma (11.1%), Status Epilepticus (10.8%), sepsis (7.4%), cardiac arrest—Return of Spontaneous Circulation (ROSC) (6.3%), metabolic diseases (3.4%), other diseases (3.7%) and cardiac diseases (2.4%). The median PRISM III-24 score was 9 (6–13). More than half (51.9%) suffered from complex chronic conditions. A big proportion (40.5%) received inotropic support. The MV rate (2019–2020) was 93.4% and the median MV duration (2021) was 7 (3–21) days. The median LOS was 9 (4–23) days. Overall, 66 patients died, given a discharge mortality rate of 17.5%. Patients who died had a greater (mean ± SD) PRISM III-24 score (20.07 ± 11.61 vs. 9.14 ± 5.09, p < 0.001), were more likely to receive inotropic support (38.6% vs. 3.1%, p < 0.001), to be girls (22.4% vs. 13.5%, p = 0.023) and suffered from different reasons at admission (p < 0.001). On the contrary, they did not show significant differences in age (p = 0.726), comorbidities (p = 0.306), MV days (p = 0.482) and LOS (p = 0.777), compared to patients who survived. The worst mortality rate within diagnosis was recorded in sepsis (42.9%) and in cardiac arrest patients resuscitated with ROSC (41.7%), followed by cardiac (33.3%), respiratory (23.8%), neurologic (22.9%), other (21.4%), metabolic (7.7%), postoperative (6.3%), and trauma (4.8%). Interestingly, no deaths were recorded in Status Epilepticus patients.

3.2 Standardized Electronic Handover Procedure (SEHOP) for discharging patients

The previous handwritten discharge instructions were transformed to a standardized electronic preformed document (SEHOP) that accompanied each patient at PICU discharge. This enabled the electronic record keeping of each patient as well. We recorded a 100% compliance rate since the beginning of the evaluation period from 01 January 2019 to 31 December 2020.

3.3 Reporting of Standardized Mortality Ratio (SMR)

96, 85, 103 and 94 patients were admitted in 2017, 2018, 2019 and 2020 with median (IQR) PRISM III-24 h scores of 10 (6.25–17), 10 (6–13), 8 (5–13) and 8 (6.75–12) respectively. Mortality rates were 24%, 11.8%, 17.5%, 16%, and SMR were 1.42, 0.92, 1.56 and 1.33 correspondingly (Table 1).

3.4 ICU readmission rate within 48 of ICU discharge

No early readmissions were recorded for 2017 and 2018, only 1 in 2019 and none in 2020. Readmission rate for 2019 was 0.8%.

3.5 CRBSI and CLABSI rates

424 temporary CVCs for 3870 CD and 85 semi-permanent Hickman catheters for 2063 Hickman CD, were evaluated. Overall DUR was 0.89, fluctuating between 0.868 and 0.929. Eight CRBSIs were documented in 5933 CD, given an overall CRBSI rate of 1.34:1000 CD. 57 CLABSIs were recorded in 5933 CD, setting an overall CLABSI rate of 9.60:1000 CD. Analytically, CLABSI and CRBSI rates per year are exposed in Table 1. LOS were significantly higher in patients with CLABSI (37.44 ± 32.93 vs. 14.88 ± 19.39 days, p < 0.001) and CRBSI (47.25 ± 54.11 vs. 17.04 ± 21.29 days, p < 0.001). Mortality, although not significantly, was higher both in CLABSI (25.50% vs. 16.30%, p = 0.119) and CRBSI patients (25% vs. 17.30%, p = 0.570).

3.6 The rate of unplanned endotracheal extubations

Ten unplanned extubations were recorded in 97 MV patients in 2019 (10.30%) and 5 in 87 MV patients in 2020 (5.72%). The total duration of MV in 2020 was 1278 days, given an unplanned 2020 extubation rate of 3.91:1000 MV days. All unplanned extubations were self-extubations during the weaning period, and only one patient needed re-intubation.

4.1 Standardized Electronic Handover Procedure (SEHOP) for discharging patients

SEHOP is an effectiveness and efficiency indicator that should accompany every PICU patient on discharge including standardized documentation of the reasons for admission and the forthcoming diagnosis made, the on-going problems, the full list of the recent medications and alterations of previous if any, and the issues that need to be resolved [23, 24, 25, 26]. The diagnosis should follow a recognized system of classifying disease and the information should be available to all clinical teams caring for the patient post discharge. In our case, handwritten discharge instructions were given to all patients since the origin of the PICU. The improvement made after ISO implementation was the commencement of the SHEOP following a predetermined form, beginning in 01 January 2019. The International Classification of Diseases, Tenth Revision (ICD 10) system was used for disease classification [27, 28]. The standardized form helped avoiding errors due to bad handwriting while the predetermined pattern allowed all the critical information to be included without omissions. According to the OPDSA proposal, we observed the discharging process, planned a SEHOP form and applied it for the years 2019 and 2020. Studying our actions, we found a 100% compliance rate. Our future efforts will be focused on maintaining this performance level.

4.2 Analysis of Standardized Mortality Ratio (SMR)

Mortality at PICU discharge (observed mortality) is a raw indicator of effectiveness and quality of care as it doesn’t take into consideration the severity of critical illness and the case mix. SMR evaluation allows internal audit and external PICU benchmarking, and enables the performance of reflective practice and quality improvement [12]. As depicted in Table 2, according to the OPDSA cycle, firstly, we observed the SMR for the years 2017 (1.42) and 2018 (0.92) compared to ESICM ICU standards of 0.75–1.25 [10], the international PICU SMR levels of 0.92–1.39 [13, 29, 30, 31], and the national SMR for Greek PICUs of 1.54 [32]. In the literature, better outcomes are reported in populations with more surgical patients, less severity of illness and lower proportion of MV [13, 31, 33, 34, 35, 36, 37]. Although the characteristics of our cohort could justify our results, a better outcome was sought. As understaffing, especially nursing, is related to poor outcome, an initiative for better medical and nurse staffing was carried out [21, 22, 38, 39, 40, 41]. Two more MD doctors and five nurses were added to PICU workforce. Better internal allocation of nurses’ shifts was planned as well, moving from the minimum 1:4 nurse to patient ratio during night shifts when it was sometimes necessary, to the least acceptable ratio of 1:2.66 (minimum three nurses in each shift to maximum of eight patient beds). In the parallel, the better education of medical stuff was pursued; four doctors started post graduate studies and three more consultants were certified in intensive care. Furthermore, under continuing educational programs, we updated all medical protocols. The prospective 2019 (SMR 1.56) and 2020 (SMR 1.33) evaluations were still relatively high, showing that more actions are required. The fact that we managed to contain our SMR values during the first year of the severe acute respiratory syndrome coronavirus 2 SARS-CoV-2 pandemic could serve as an indication that our efforts started to flourish.

4.3 ICU readmission rate within 48 h of ICU discharge

Early (<48 h) readmissions could be related to poor discharge decision making, e.g., earlier than anticipated discharge, gaps in the communication regarding patient needs to the step down facility, deficient handover procedure, poor ward patient care, and serve as an ICU safety and quality indicator worldwide [29, 42, 43, 44, 45]. Patients with early readmissions are associated with increased hospital stay, cost, morbidity and mortality. Lower early readmission rate is typically reported for discharged pediatric PICU patients (1.2–3.7%) [46, 47, 48, 49, 50] compared to the target of 4% set by ESICM for adults [10]. Ensuing the OPDSA cycle, we observed readmission rates for the years 2017 and 2018 which were zero (0), as demonstrated in Table 2.

Our low numbers were not realistic compared to the literature and an action was started to examine and determine the reasons behind the data. We decided to check the hypothesis that the patients stayed too long in the PICU, until they had better health than is normal for discharge. This course of action, increased, on the one hand, the patient’s safety, but, on the other hand, keeping patients too long in the PICU is inefficient [42, 51]. The prolonged need for PICU care is intertwined to illness severity, case mix, discharge policies and existence of step-down facilities. As mentioned, the majority of our population suffered from comorbidities (51.7%) and had all the characteristics that inevitably, mandated a prolonged PICU stay. The lack of step-down pediatric units in our country complicates further the equation on LOS. An initiative was started in 2019 to examine whether an earlier discharge policy is feasible with the concept of having a shorter LOS with acceptable anticipated readmission rates; protocols for less sedation and faster weaning from MV and increased use of Non-invasive Ventilation (NIV) were incorporated as well [52]. A shorter, although not significantly, LOS was recorded for 2019 (17.08 ± 24.06) and 2020 (16.84 ± 17.70) days compared to 2017 (17.12 ± 24.36) and 2018 (19.98 ± 24.99). Despite that, early readmission rates remained very low, 0.8% for 2019 and 0% for 2020, suggesting that probably other reasons behind PICU management are to be sought.

4.4 CRBSI and CLABSI rates

Bloodstream infections are the first Hospital Acquired Infections (HAI) infections in pediatrics [53] and CRBSI and CLABSI are considered device related preventable HAI that increase LOS and cost whereas their role in mortality remains controversial [18, 54, 55, 56]. The last decade an effort has started worldwide to keep their rate as low as possible, and their control is considered not only a safety and a quality indicator, but a reimbursement one as well [57]. DUR is inseparably related to CVC infection rates and reduction in CVC use is among the first bundles of care to be taken under their control [9, 58]. DUR in our occasion was 0.89, very high compared to international ICU values of around 0.70 [59] and PICU values which are even lower, about 0.50 [60, 61], and are another confirming factor regarding high illness severity in our population. Higher patient’s acuity necessitates CVC presence, which inevitably expose them to higher infection rates. As exposed in Table 2, CRBSIs baseline rates for 2017 and 2018 found them within ESICM limits (<4:1000 CD) [10], and our plan was to maintain those high result standards for the following years, which we succeeded in doing. Conversely, baseline CLABSI rates were higher than reported USA pediatric values which are below 3.1:1000 CD [60, 62, 63]. Our results approached other international values ranging from 3.7–18.8:1000 CD [61, 64, 65, 66, 67, 68] and were found within national PICU CLABSI rates of 6.09 to 16.67:1000 CD [69, 69]. Actions were indicated with focused efforts in increasing compliance with the insertion and prevention CLABSI bundles of care [18, 71, 72]. A stabilization was recorded in 2019, with an unfortunate relapse in 2020, despite our efforts, probably due to the burden of the 2020 pandemic, where less experienced personnel had to deal with increased PICU workload. In accordance with the literature, CRBSIs and CLABSIs increased LOS, whereas they did not play a role in the mortality.

4.5 The rate of unplanned endotracheal extubations

Unplanned extubation (UEX) rate is an indicator of poor patient care and related to patient safety as it is associated with the need for re-intubation, and increased risk for nosocomial pneumonia and death [10, 73, 74]. UEX are defined as accidental, due to accidental removal of endotracheal or tracheostomy tube during nursing or medical handling, or as self-extubations, due to patient movements, happening usually during awakening or weaning periods. Reference values are recorded as percentage of mechanically ventilated patients of 2–16% for adults [74, 75] and 3.4–14.7% for children [29, 76, 77, 78] and/or with the more accurate index of unplanned extubations:1000 days of MV with standards lower than 10:1000 MV days, both for adults and children [10, 79]. As small steps to our OPDSA cycles, UEX were added in 2019 and their rate per 1000 MV days in 2020 (Table 2). The first recorded rate in our study was 9.70% for 2019, and subsequently 5.37% and 3.91:1000 MV days for 2020, all values being well within international references. Interestingly, all UEX were self-extubations during the weaning period and only one patient in 2020 needed re-intubation, which could indicate an unnecessary prolonged weaning period. Our plans are to maintain these good standards into the future with the use of bundles of care, including better fixation of the endotracheal and/or tracheostomy tubes, use of anatomical and arithmetic reference points, which are to be checked at least every shift, and protocols for high risk situations [79, 80]. Our goals will be enhanced by less use of invasive MV, increasing the use of noninvasive ventilation and shorten, as much as possible, the weaning period.