Introduction. Simulation training with an integrated simulator is appropriate for achieving educational goals in airway management. Thus, we designed this study to evaluate the effectiveness of a simulation based emergency airway management program (SBEAMP) in actual practice.
Method. This is a retrospective sub-group analysis of the Korean Emergency Airway Management Registry from 2006 to 2010. We categorized all hospitals into two groups. Six hospitals that actively attended SBEAMP were defined as the ‘participant group’, and the others as the ‘non-participant group’. The types of medicines administered, the use of pre-oxygenation, and the rate of first pass success were compared.
Result. The ratio of patients with no medicine received during intubation showed a decrease in both groups but was more rapid in the participant group (p<0.001). The ratio of intubation with sedatives alone was high in the non-participant group (P<0.001). The ratio of intubation with paralytics alone was high in the non-participant group (p<0.001). In the participant group, a combination of both agents was used more frequently (P<0.001). Cases of intubation with both agents and preoxygenation were more prevalent in the participant group (P<0.001).
Conclusion. We concluded in this study that SBEAMP had a positive influence on actual clinical outcomes in emergency airway management.
Key words: airway management, registry, simulation training
Airway management is one of the most important steps in emergency patient care, and it is part of the core content of any emergency medicine training program. An essential part of airway management involves rapid sequence intubation (RSI), which emphasizes proper medication and procedure selection. Emergency airway education is a necessary part of an organized program and should include the recognition of the necessity for intubation, predictions of difficult airways, the selection of proper intubation methods and medications, and the building of skills related to the use of alternative airway devices and rescue ventilation methods. (1) Nevertheless, traditional training methods, involving the use of a cadaver, an artificial airway model, or actual patients are not suitable for a comprehensive airway management training program due to the limitations such as cadaver supply, ethical problems or patient safety. (2,3)
Simulation training using a high technology simulator is appropriate for achieving complex educational goals and has therefore been utilized with trainees to teach them how to handle difficult and dynamic airway conditions. (2-4) In Korea, the Korean Emergency Airway Management Society (KEAMS, http://www.keams.or.kr/english) has played an important role in the development of a simulation-based emergency airway management education program (SBEAMP), and has educated healthcare professionals since 2006.
In general, the effectiveness of SBEAMP is usually evaluated using pre- and post-education questionnaires or tests, but these types of evaluation methods are inadequate for evaluating the effect of SBEAMP on actual clinical practice. Thus, using a web-based airway registry system, we evaluated the effectiveness and limitations of SBEAMP in actual practice by comparing airway management trends in hospitals that use SBEAMP program with other hospitals that do not use the program.
The present study was designed as a retrospective sub-group analysis of the registry of the Korean Emergency Airway Management Society (KEAMS). We collected data in accordance with our protocol, which included information about patient airway management aspects, such as general patient information, patient evaluation results, and selected medications and devices. This study was reviewed by the corresponding research ethics committees. The Korean Emergency Airway Management Registry (KEAMR) is utilized in 20 hospitals for the purpose of identifying the current status of airway management in domestic emergency medical centers. After performing endotracheal intubation, all physicians are required to record data on a standard form. The data are then examined by the supervising investigator and entered into a web-based registry.
All instructors of SBEAMP are board-certified emergency physicians and resident directors at their respective hospital. The course was developed through multiple intensive consensus meetings. The goals and objectives, content of the program, essential skills set, and simulation scenarios were developed by referencing current literature, well established airways courses, and clinical experiences of the instructors. The course consisted of three parts. A didactic session encompassed the concept of difficulty airways, the RSI, and emergency airway algorithms. The skills workshop, which provided opportunities to become skilled with airway devices and to become familiar with RSI pharmacology, was the second part. Small-group immersive simulation training using a high-technology mannequin simulator constituted the third part of SBEAMP. After the training program, surveys and peer reviews were conducted to improve the program.
In all, there were 10,978 registered patients from 2006 to 2010. Among these cases, patients in cardiac arrest and intubations which were not performed by an emergency physician were excluded from this study. Twenty hospitals utilized the KEAMR during the study period. We categorized these twenty hospitals into two groups. At six hospitals, all junior emergency residents regularly participated in SBEAMP developed by KEAMS. These were defined as the ‘participant’ group, and the other fourteen hospitals were defined as the ‘non-participant group’. We collected general information on the patients, including their age, gender, initial vital signs, cause of endotracheal intubation, and level of consciousness. The types of medications which were used for intubation, the use of pre-oxygenation, and the rate of first pass success (FPS) were compared between the two groups for each year.
Statistical analysis was performed using SAS version 9.2 (SAS Institute Inc., Cary, NC, USA). Continuous variables were presented as means and standard deviations, and the frequency was presented as numbers and percentages. To compare the differences between the two groups by year, a logistic regression model was used. In this model, the year and group were the main effect. In addition, the interaction term of ‘group by year’ was considered. p-values of less than 0.05 were defined as statistically significant in this logistic regression model. To evaluate the fitness of this logistic regression model, Hosmer-Lemeshow goodness-of-fit test was used. Harrell’s c-statistics was utilized to evaluate the discrimination ability. A p-value lager than 0.10 means ‘good fitness’ in Hosmer-Lemeshow goodness-of-fit test. If the result of Harrell’s c-statistics was close to 1.00, that could be interpreted as ‘high discrimination ability’.
In total, as mentioned above, there were 10,978 registered patients from 2006 to 2010. Excluding cardiac arrest patients and airway management conducted by non-emergency physicians, 5,783 patients were finally included (figure 1), 3,790 patients in the participant group, and 1,993 patients in the non-participant group. The data pertaining to the non-participant group in 2006 could not be collected because that group initially joined the KEAMR from 2007. The average age of the patients in the non-participant group was slightly higher than that of the participant group. The weight of the participants was greater in this group as well, compared to the other group. Male patients were larger than female patients in both groups. However, there were no differences in terms of the gender ratio. The blood pressure and pulse rate did not show any differences between the two groups, but the respiration rate was higher in the participant group. Initial oxygen saturation was higher in the participant group as well (table 1).
The use of medication and the conduction of preoxygenation
The ratio of patients for which no medicine was administered during endotracheal intubation showed a decrease year over year in both groups. However, it decreased more rapidly in the participant group (figure 2) (p<0.001). The ratio of intubation performed only with sedative agents was relatively high in the non-participant group, and tended to increase (figure 3) (P<0.001). The ratio of intubation performed only with a paralytic agent was relatively high in the non-participant group as well (figure 4) (p<0.001). In the participant group, a combination of both types of agents was used more frequently. In addition, the ratio tended to increase year after year (figure 5) (P<0.001). Cases of intubation with preoxygenation were more prevalent in the participant group. Furthermore, the tendency was prominent (figure 6) (P<0.001).
The ratio of first pass success (FPS)
We defined FPS as successful intubation on only the first attempt. The ratio of FPS in the participant group showed an increase every year (p=0.02), but differences between the two groups were not readily apparent (figure 7) (P=0.08).
The ratio of failed airways
A failed airway was defined as a case in which oxygen saturation greater than 90% could not be maintained and/or a case involving three failed intubation attempts by an experienced provider. The incidence of a failed airway was not low in both groups. It was lower in the participant group, but a statistical difference was not observed (figure 8) (p=0.08).
Airway management in an emergency department (ED) is usually performed in very unstable and dynamic conditions. It is very difficult to ensure safe airway management, and the incidence of complications can be relatively high. (5) Therefore, it is essential for an emergency physician, who takes primary responsibility for airway management in an ED, to have the skills necessary to deal with various airway conditions. For this purpose, well-designed education and training programs are essential. (1) However, most airway management education programs conducted up to the early 2000s consisted of didactic lectures and/or skill training with partial-airway mannequins. (6) To ensure safe and proper airway management, healthcare providers need to be trained not only in technical skills, but also be trained in their cognitive competency, as well as the non-technical aspect of care.
RSI is still one of the most important parts of emergency airway management. To perform proper airway management with RSI, it is necessary to understand not only the intubation technique itself but also to have proper knowledge of the medications used. Complex capabilities, including early prediction of emergency conditions, skills pertaining to rescue and the use of alternative devices, and the ability to make rapid decisions and proper responses are essential in cases involving difficult or failed airway management. (5)
Simulation-based healthcare training using mannequin simulators is extensively used to enhance both technical and non-technical competencies. It also provides a safe learning environment for trainees, providing the opportunity to experience uncommon, dangerous and difficult conditions, in the hope of contributing to a reduction in medical errors and perhaps expenses, as well. These training programs have been adopted in various fields of medicine, including anesthesiology and critical care medicine. Furthermore, the use of simulation education methods in emergency medicine started to increase from 2000’s. (7,8)
In Korea, interest in patient safety has accelerated in the development and dissemination of simulation-based training programs. KEAMS, which was founded in 2006 by emergency physicians and nurses, developed SBEAMP for the first time in Korea. They tried to play key roles in the growth of simulation-based education programs for emergency airway management in Korea by conducting regular airway management workshops and managing KEAMR, the first multicenter emergency airway management registry in Korea, starting in 2006.
With the growth of these medical simulations, various types of studies over many years have attempted to estimate their usefulness. (9) Mayo et al. reported that education with a patient simulator could promote clinical skills in a prospective randomized controlled trial with 50 advanced cardiac life support trained residents. (10) Hall et al. showed that simulation-based education was as effective as education with actual patients in a randomized trial with 36 emergency medicine technician students. (11) However, these studies simply estimated the differences before and after a one-time education event. Therefore, Vincent et al. designed a study with 27 anesthesiology residents to evaluate the effects of simulation-based difficult airway management training. They showed a long-term effect of simulation education in their study, which evaluated attendants’ clinical skills immediately after the course, after one month, after six months and after 12 months. (12) Nevertheless, there are some inevitable limitations in these studies when attempting to prove that clinical actions with actual patients were actually improved, as changes were estimated only with a patient simulator and a test.
It is essential to evaluate the effectiveness of simulation-based training by estimating changes in actual clinical performance levels after a simulation-based education program, as the final goal of all medical education is a clinical improvement in actual patient care. We designed our study to evaluate the effect of SBEAMP indirectly by comparing the clinical performance levels of the participant group with that of a non-participant group from 2006 to 2010 using KEAMR. In the participant group, the ratio of intubations conducted with sedatives, paralytics and preoxygenation was relatively high, showing an increase year after year. Nevertheless, there were no significant statistical differences between the two groups in terms of the first pass success rate and the rate of failed airways. Given this result, we could conclude that the participant group showed more application and implementation in real practice of educational content learned from the SBEAMP. However, additional research is necessary to determine long-term outcomes and the incidence of complications, in more detail.
In some indices, similar tendencies were observed between the two groups, with yearly differences. The ratio of intubation without medication in the non-participant group showed a decrease (figure 2). An analogous tendency was reported in the ratio of intubation with both sedatives and paralytics as well (figure 5). These tendencies could be regarded as accidents and as unrelated events; however, they could also be a result of a positive ‘collateral effect’ between the two groups, as doctors in both groups could communicate with each other in various ways directly or indirectly. Jeffrey et al. reported that, after simulation-based central venous catheterization training for senior residents, the skills of uneducated junior residents were advanced, too. They defined the phenomenon, when one educated group promoted the ability of another group through secondary education, as the ‘collateral effect’. (13) We surmised that collateral effects may explain such tendencies in our study. However, long-term follow-up of airway management should be done as part of additional work.
There are several limitations to this study. First, we cannot be certain that SBEAMP was the sole factor creating the noted differences between the two groups. For example, the total number of patients in the participant group was higher than that in the other, despite the fact that there were fewer hospitals in the participant group (6 versus 14). Therefore, doctors in the participant group may have had more opportunities to conduct emergency airway management on actual patients. Such a difference in clinical experience could be another factor which influenced the two groups. Furthermore, it was possible that other airway management training programs had been taken by one or both groups. However, we confirmed that there were no simulation-based airway management programs taught to each group since our program during the study period. The non-participant group were taught with didactic lectures and some skills training using part-task trainers. Second, it is difficult to conclude that SBEAMP influenced clinical airway management performance overall, as not all emergency residents had an opportunity to participate in the program. Nonetheless, we consider that it may have sufficiently motivated a change in clinical performance, as an accumulation of educational experience over years can have a positive collateral effect. Third, other important factors related to airway management, such as the adequacy of the medication dosage, predictions of a difficult airway, and the selection of adequate devices, were not evaluated. Thus, only partial changes in clinical performance were evaluated in this study. This is one limitation of a retrospective study which should be offset in a future study with the design of a more convenient registry to be quantitatively evaluated. Fourth, in the registry, clinical conditions such as the severity of patients could not be calibrated. Fifth, the fact that there were no differences in the first pass success rate and the incidence of a failed airway between two groups, despite several differences found in the selection of medications and the conduction of preoxygenation, is difficult to explain. Another limitation is that long-term complications could not be recorded because the registry was filled in immediately after intubation. In future studies, we want to design a better study protocol that can compensate for these limitations.
Despite these limitations, we conclude that SBEAMP had a positive influence on actual clinical outcomes in emergency airway management. One additional important aspect of this study was that it represents an effort to prove the effect of SBEAMP in actual clinical practice.
- Kovacs G, Law JA, Ross J, Tallon J, MacQuarrie K, Petrie D, et al. Acute airway management in the emergency department by non-anesthesiologists. Can J Anaesth 2004;51(2):174-80.
- Goldmann K, Ferson DZ. Education and training in airway management. Best Pract Res Clin Anaesthesiol 2005;19(4):717-32.
- Schaefer JJ, 3rd. Simulators and difficult airway management skills. Paediatr Anaesth 2004;14(1):28-37.
- Amin MR, Friedmann DR. Simulation-based training in advanced airway skills in an otolaryngology residency program. Laryngoscope 2013;123(3):629-34.
- Orebaugh SL. Difficult airway management in the emergency department. J Em Med 2002;22(1):31-48.
- Reeder TJ, Brown CK, Norris DL. Managing the difficult airway: a survey of residency directors and a call for change. J Em Med 2005;28(4):473-8.
- McFetrich J. A structured literature review on the use of high fidelity patient simulators for teaching in emergency medicine. EMJ 2006;23(7):509-11.
- Grenvik A, Schaefer JJ, 3rd, DeVita MA, Rogers P. New aspects on critical care medicine training. Curr Opin Crit Care 2004;10(4):233-7.
- Kennedy CC, Cannon EK, Warner DO, Cook DA. Advanced airway management simulation training in medical education: a systematic review and meta-analysis. Crit Care Med 2014;42(1):169-78.
- Mayo PH, Hackney JE, Mueck JT, Ribaudo V, Schneider RF. Achieving house staff competence in emergency airway management: results of a teaching program using a computerized patient simulator. Crit Care Med 2004;32(12):2422-7.
- Hall RE, Plant JR, Bands CJ, Wall AR, Kang J, Hall CA. Human patient simulation is effective for teaching paramedic students endotracheal intubation. Academic emergency medicine: official journal of the Society for Academic Emergency Medicine. 2005;12(9):850-5.
- Hubert V, Duwat A, Deransy R, Mahjoub Y, Dupont H. Effect of simulation training on compliance with difficult airway management algorithms, technical ability, and skills retention for emergency cricothyrotomy. Anesthesiology 2014;120(4):999-1008.
- Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Unexpected collateral effects of simulation-based medical education. Acad Med 2011;86(12):1513-7.
|Table 1. General characteristics|
|Non-Participant Group||Participant Group||P value|
|Systolic blood pressure (mmHg)||130.67||±45.69||133.19||±46.02||0.062|
|Diastolic Blood pressure (mmHg)||76.55||±24.36||77.67||±27.08||0.149|
|Pulse rate (per min)||99.41||±27.93||97.89||±27.78||0.06|
|Respiratory rate (per min)||23.29||±7.53||22.18||±7.71||<0.001|
|Oxygen saturation (%)||86.26||±15.66||88.09||±14.92||<0.001|
|Non trauma patients (n)||1448||2644||<0.001|
|Trauma patients (n)||500||1146|
|Causes of Intubation (n)|
|Airway protection failure||741||719||<0.001|
|Oxygenation and ventilation failure||586||692|
|Anticipated oxygenation or airway protection failure||621||1714|
|Initial mental status (n)|
|Response to verbal stimulation||183||284|
|Response to Pain||511||759|
Figure 1. Flow chart of the research protocol
Figure 2. Ratio of endotracheal intubation without any medicine (p<0.001)
Hosmer-Lemeshow goodness-of-fit test p >0.99; Harrell’s c-statistics = 0.68
Figure 3. Ratio of endotracheal intubation with sedative agents only (p<0.001)
Hosmer-Lemeshow goodness-of-fit test p >0.99; Harrell’s c-statistics = 0.75
Figure 4. Ratio of endotracheal intubation with paralytic agents only (p<0.001)
Hosmer-Lemeshow goodness-of-fit test p >0.99; Harrell’s c-statistics = 0.79
Figure 5. Ratio of endotracheal intubation with sedative agents and paralytic agents (p<0.001);
Hosmer-Lemeshow goodness-of-fit test p >0.99; Harrell’s c-statistics = 0.75
Figure 6. Ratio of endotracheal intubation with sedative agents, paralytic agents and preoxygenation (p<0.001)
Hosmer-Lemeshow goodness-of-fit test p >0.99; Harrell’s c-statistics = 0.74
Figure 7. Ratio of first pass success (p=0.08)
Hosmer-Lemeshow goodness-of-fit test p >0.99; Harrell’s c-statistics = 0.55
Figure 8. Ratio of failed airway (p=0.08)
Hosmer-Lemeshow goodness-of-fit test p >0.99; Harrell’s c-statistics = 0.63
This work is licensed under a Creative Commons Attribution 4.0 International License