Predicting delays in antibiotic administration in the emergency department: a machine learning approach incorporating nursing workload and crowding factors

2.1 Study setting

We conducted a retrospective data analysis at a tertiary academic hospital in Seoul, South Korea. The study site was a 67-bed ED with an annual average of 75,000 visits. Patients who visited the ED between 01 January and 31 December 2022, were included in the study.

To provide a comprehensive overview of ED performance, we included all patient populations, including adults, children, and deceased patients. This broad inclusion was essential for accurately evaluating the total nursing workload, ensuring a complete assessment of nursing activities and their associated workload within the ED setting. However, we excluded patients who did not enter the treatment area, those who visited the ED for non-medical purposes (e.g., obtaining medical documents), and those who were immediately transferred to other departments (e.g., delivery room).

2.2 Input variables

2.3 Outcome: delay in antibiotic administration

The outcome variable was defined as the hourly frequency of cases in which antibiotic administration for suspected infections exceeded 1 hour after prescription. This metric serves as a key quality indicator for ED performance and reflects the burden on medical staff caused by crowding, which affects both the preparation and administration of prescribed antibiotics. Given that the input and output variables were measured as frequencies, no missing values were present in the dataset.

2.4 Model development and evaluation process

2.4.1 Data splitting and feature selection

The preprocessed dataset was randomly divided into training and test sets using a 7:3 ratio (Fig. 1). Poisson regression was conducted to identify key variables among the 46 nursing procedure, as the outcome variable represents the number of events occurring within a specific time and space [23]. The Poisson regression model was developed in two steps. First, we performed univariate Poisson regression on the training set for each predictor. Second, we developed a multivariate Poisson regression model using the variables identified as statistically significant in the univariate analysis. Predictors that remained statistically significant in the multivariate model (p < 0.05) were designated as “workload-only variables”.

Fig. 1.Model development process. NEDOCS: national emergency department overcrowding score; XGBoost: eXtreme gradient boosting.

3.1 Data characteristics

Fig. 2 illustrates the eligibility process used in this study. This study included 45,896 patients with a total of 63,831 ED visits (Table 1). Patient acuity was assessed using the Korean Triage and Acuity Scale (KTAS), the official triage system employed in South Korean EDs [31, 32, 33]. The KTAS categorizes patients into five levels, with Level 1 representing the highest urgency and Level 5 indicating the lowest. The frequency of delayed antibiotic administration observed in the ED was 0.83 occurrences per hour.

Fig. 2.Eligibility process. ED: emergency department; NEDOCS: national emergency department overcrowding scale.

Of the 8760 total records, 6132 (70%) were allocated to the training dataset, whereas 2628 (30%) were assigned to the test dataset. No statistically significant differences were observed between the two datasets. During the 1-year study period, 42 types of nursing procedures were performed 1,977,268 times, averaging 31.6 procedures per patient visit. The three most frequently performed nursing procedures were nursing record entry (26.6%), verification of physicians’ orders (16.2%) and vital sign assessment (13.8%) (Supplementary Table 1).

3.2 Poisson regression analysis for significant variable selection

Univariate Poisson regression analysis identified 28 nursing procedures as statistically significant predictors. Subsequent multivariate Poisson regression analysis refined this selection to nine, which were determined as workload-only group variables: (1) verification of physicians’ orders, (2) intravenous medication administration, (3) explanation of the emergency treatment process, (4) Foley catheterization, (5) simple dressing, (6) application of low-flow oxygen therapy, (7) subcutaneous and intradermal injection, (8) application of high-flow nasal cannula, and (9) intramuscular medication administration (Supplementary Table 2).

3.3 Machine learning models

Model performance varied based on the inclusion of nursing workload data (Table 2). Models incorporating workload-related variable groups (workload-only and NEDOCS/workload groups) consistently outperformed those incorporating the NEDOCS-only variable group across all algorithms. Evaluation on the test dataset demonstrated that both the RF and XGBoost models using the NEDOCS/workload variable group achieved identical performance (RMSE = 0.907, adjusted R² = 0.120 and MAE = 0.712). XGBoost was selected as the best model owing to its computational efficiency and interpretability, as confirmed through SHAP analysis. Among all predictors, verification of physicians’ orders had the greatest influence (SHAP value = 0.165), indicating its strong positive effect on model performance. Other significant contributors included the total number of patients (Total pt) and maximum waiting time (max waiting), with SHAP values of 0.095 and 0.039, respectively.

Beyond relative importance, the SHAP plot (Fig. 3) illustrated the directional effect of various factors on antibiotic administration delays. Among all predictors, verification of physicians’ orders had the strongest influence. A consistent pattern was observed across all major variables—including verification of orders, total patient count, maximum waiting time, and number of boarding patients. Higher values (shown in red) were associated with increased delay predictions, whereas lower values (shown in blue) corresponded with decreased predicted delays. Notably, an increase in the verification of physicians’ orders was associated with greater delays in antibiotic administration.

Fig. 3.SHAP plot for XGBoost model. Total pt: Total number of patients present in the emergency department (ED); Boarding pt: Number of patients boarding in the ED while awaiting inpatient bed assignment; Vent pt adjusted: Number of patients receiving mechanical ventilation (up to 2); Max boarding: Longest boarding time (in hours) after admission decision, while awaiting inpatient bed assignment; Max waiting: Longest waiting time (in hours) in the waiting room before initial physician assessment; SHAP: SHapley Additive exPlanations.

4.1 Principal findings

This study successfully developed and validated an ML model to predict delays in antibiotic administration in the ED. First, although numerous studies have applied ML to predict sepsis in the ED—primarily focusing on prognosis and diagnosis [34], we concentrated on the operational aspects of care delivery. Specifically, we identified which nursing interventions and ED crowding factors contribute to delayed antibiotic administration using nursing data. Second, although the relationship between ED crowding and delays in antibiotic administration has been extensively studied [9], to the best of our knowledge, our study is the first to highlight the pivotal role of nursing workload in this process. The integration of nursing-specific workload data in our model provided a unique perspective on the influence of nursing demands on timely antibiotic delivery, an aspect previously overlooked in crowding metrics. Finally, we developed a predictive model for timely care delivery that applies to all patients requiring antibiotic administration, rather than being restricted to patients with severe conditions, such as sepsis or septic shock. By including a broad range of clinical conditions (e.g., open fractures, neutropenic fever, infective endocarditis, intra-abdominal infections and bacterial meningitis), this study demonstrated that nursing workload significantly influences care delivery across diverse clinical presentations. These findings provide a valuable foundation for future research aimed at improving nurses’ care delivery processes and optimizing timely antibiotic administration in the ED.

4.2 Importance of nursing workload in antibiotic administration

Among the models evaluated, those incorporating only nursing workload variables outperformed the NEDOCS-only model, with the combined NEDOCS/workload model demonstrating the best predictive performance. The most frequent and workload-intensive nursing procedure was nursing record entry, followed by the verification of physician orders. Key factors most frequently associated with delays in antibiotic administration included verifying orders, total number of patients, maximum waiting time, administration of intravenous medication and explanation of emergency treatment processes.

Although crowding metrics typically highlight patient volume and waiting times, they rarely account for the strain on nursing resources, despite the crucial role nurses play in antibiotic administration [35]. Furthermore, patient volume and waiting times alone do not fully reflect the added complexities introduced by high patient acuity and hospital capacity constraints [36]. For instance, a critically ill patient awaiting a bed at the intensive care unit demands significantly more nursing attention than a stable patient, yet both count equally in total volume metrics. Moreover, when ED beds are unavailable, critically ill patients may be boarded in waiting rooms, sometimes requiring care to begin in non-designated areas. These hidden challenges and systemic bottlenecks can dramatically increase nursing workload, underscoring the need for a more comprehensive approach to assessing the true effect of “crowding” on patient outcomes.

Our findings support the need to incorporate nursing-specific workload indicators when predicting delays in antibiotic administration. Previous research aligns with this conclusion—Fee et al. [37] found that ED overcrowding disproportionately affects nurses administering antibiotics. Additionally, Roberts et al. [38] identified excessive patient workload as a key barrier for nurses to initiate timely antibiotic therapy in patients with septic shock. By integrating these nursing-specific factors, our study highlights the importance of moving beyond conventional crowding metrics to gain more accurate and actionable insights into ED operational challenges.

4.3 Interpreting the machine learning model results

Our methodology aligns with feature engineering in ML, where domain expertise informs the selection of robust and context-relevant variables. Traditional ED crowding metrics rarely incorporate nursing workload indicators; however, we explicitly included these indicators based on clinical knowledge. The integration of nursing procedures and their frequencies facilitated the improvement of the model’s ability to capture the real-world complexity of ED operations. This approach contributed to the superior performance of the combined NEDOCS/workload model compared to models relying solely on traditional crowding metrics.

The interpretability of our final XGBoost model was examined using SHAP analysis, which quantified the magnitude and direction of each factor’s effect on antibiotic administration delays. Verification of physicians’ orders had the highest effect (SHAP value = 0.165), followed by total patient count (0.095) and maximum waiting time (0.039). The SHAP plot consistently showed that higher values for these variables were associated with increased delays, underscoring the substantial role of nursing workload in ensuring timely antibiotic administration.

Notably, the “verification of physicians’ orders” variable did more than simply count prescriptions; it captured the time and cognitive effort nurses devote to interpreting and confirming newly generated orders in the fast-paced ED environment. This workload had a greater effect on predicting delays than traditional crowding indicators, such as total patient volume and maximum waiting times, further emphasizing the critical role of nursing-specific tasks in determining the timeliness of care.

4.4 Practical approaches to ED nursing workload management

In the present study, delays in antibiotic administration occurred approximately once every 50 minutes (0.83 times per hour). Reducing nursing workload is pivotal in mitigating these delays, particularly in overcrowded EDs. With the growing number of boarding patients awaiting admission, crowding has become a widespread challenge in EDs [36]. To mitigate delays in antibiotic administration in this setting, various strategies have been proposed, including implementing sepsis screening tools at triage [39, 40, 41], establishing hospital-wide timely care protocols [42], and improving team communication [41]. Although these interventions address common workflow inefficiencies in EDs, they alone are insufficient if hospital capacity bottlenecks remain unaddressed. Improving patient flow must extend beyond ED bed management and incorporate a hospital-wide approach to optimizing bed availability.

Beyond these broader strategies, our study highlights actionable short-term improvements within the ED. Our results showed that verifying physician orders had the greatest effect on delayed care. Given that these essential nursing tasks can not be eliminated, efforts should focus on minimizing the workload associated with each task. Unnecessary workload is imposed on nurses when laboratory tests that could otherwise be performed concurrently are ordered separately or when prescriptions require additional clarification. For instance, a routine test or medication may unexpectedly disappear from a prescription, the criteria for conducting a prescribed test may be unclear, or an antibiotic may be changed without an explanation. Although physicians typically make these decisions based on sound clinical reasoning, failing to communicate the rationale at the time of prescription lead to additional inquiries via phone calls or text messages, ultimately increasing the workload for both nurses and physicians.

To address this issue, leveraging the EHR system to provide a comprehensive overview of relevant changes—such as recent prescription alterations and associated laboratory tests—can help reduce nurses’ cognitive burden. Furthermore, requiring physicians to provide a brief rationale for prescription changes can minimize unnecessary communication, thereby alleviating workloads for both nurses and physicians. Ultimately, these improvements should be pursued through a team-based approach, with nurses and physicians collaborating continually to develop more efficient strategies in managing care in crowded ED environments.

Furthermore, the findings of this study can drive real-world operational improvements. Real-time workload monitoring could enable healthcare organizations allocate additional staff to areas where ED crowding threatens patient care quality. Currently, most EDs lack dynamic staffing adjustments for sudden, unpredictable workload surges. However, implementing a monitoring system could enable flexible, data-driven responses to these challenges. At the study-site ED, charge nurses currently identify high-intensity zones and allocate additional staff as needed. Implementing an automated system to track workload-related metrics, as identified in this study, would enable proactive decision-making, allowing for optimized staffing in response to surges in nursing workload.

4.5 Limitations

Although this study offers valuable insights, it had several limitations. First, its single-center design may limit the generalizability of the findings, potentially introducing geographic and demographic biases in the models. Expanding future studies to multiple centers could enhance external validity and broader applicability. Second, certain nursing procedures known to contribute to high workload such as interruptions and coordination activities were not included, as they are essential but not recorded. Incorporating these factors into future models would provide a more comprehensive understanding of ED care delivery. Finally, relying on retrospective records of nursing activities limited the ability to capture real-time changes in workload, potentially introducing temporal bias. Despite thorough clinical validation, the quality and completeness of historical medical records may have influenced the reliability of our ML models. For instance, although some nurses documented skin tests before antibiotic administration, others did not. Additionally, shift changes occasionally led to incomplete documentation, particularly when nurses had to prioritize emergency patient care, potentially resulting in an underestimation of workload data. Although fall prevention activities were recorded, we could not verify whether all nurses consistently implemented these precautions. Future research should incorporate observational studies to assess the alignment between actual nursing workloads and EHR-extracted data. Furthermore, establishing a standardized method for recording nursing activities could enhance the accuracy of real-time workload assessments in nursing practice.