Title
Author
DOI
Article Type
Special Issue
Volume
Issue
Assessment of pediatric emergency physicians’ interpretation of brain non-contrast computed tomography: a prospective study in a tertiary care center
1Department of Emergency Medicine, College of Medicine, King Saud University, 11461 Riyadh, Saudi Arabia
2College of Medicine, Dar Al Uloom University, 13314 Riyadh, Saudi Arabia
3Department of Emergency Medicine and Critical Care, College of Medicine, King Saud University, 11461 Riyadh, Saudi Arabia
4Department of Clinical Sciences, College of Medicine and Riyadh Hospital, Dar Al Uloom University, 13314 Riyadh, Saudi Arabia
5Therapeutic Deputyship, Ministry of Health, 12382 Riyadh, Saudi Arabia
*Corresponding Author(s): hbinsalleeh@ksu.edu.sa (Hashim M. Bin Salleeh)
| History | Submitted: 28 December 2023 | Accepted: 12 March 2024 | Published: 08 August 2024 |
| Copyright: | ©2024 The Author(s). Published by MRE Press. |
This study aims to prospectively assess the proficiency of pediatric emergency physicians (PEPs) in interpreting non-contrast computed tomography (NCCT) brain images. A prospective investigation was conducted at the pediatric emergency unit of King Saud University Medical City (KSUMC), Saudi Arabia, over a one-year period. All patients undergoing plain brain NCCT during this period were enrolled. An independent attending neuroradiologist and two certified pediatric emergency consultants compared the interpretations of PEPs with the official final reports issued by the on-call radiologist. A total of 202 pediatric patients were examined, all under 14 years of age, with a mean age of 4.8 ± 3.6 years. Trauma was the predominant presenting complaint (127 patients, 62.9%), followed by seizures (28 patients, 13.9%). The primary indication for brain NCCT was to detect intracranial bleeding, identified in 134 patients (66.3%), followed by a space-occupying lesion in 22 patients (10.9%). Additionally, hydrocephalus with elevated intracranial pressure (ICP) was observed in 20 patients (9.9%). The overall agreement between PEPs and radiologists, based on Landis and Koch benchmark classification, was moderate (Kappa = 0.578), with an accuracy of 82.18%. The overall accuracy of brain NCCT interpretation by PEPs compared to radiologists was found to be moderate. Further multicenter studies in pediatric emergency settings with larger sample sizes are warranted to validate these findings.
Cite this article
Hashim M. Bin Salleeh, Mohammed A. Alrowayshed, Anas A. Althenayan, Ahmed Mohammed A Alkhars, Rakan S. Shaheen, Zohair Al Aseri. Assessment of pediatric emergency physicians' interpretation of brain non-contrast computed tomography: a prospective study in a tertiary care center. Signa Vitae. 2024; 20(8): 55-60. doi: 10.22514/sv.2024.077
1. Introduction
Brain non-contrast computed tomography (NCCT) is an integral component of emergency patient evaluation owing to its ability to assess emergent neurological conditions; therefore, accurate interpretation is paramount for timely and appropriate interventions [1]. Moreover, brain NCCT is incorporated into various head injury algorithms commonly utilized in pediatric emergency care settings [2]. Requests for brain NCCT in the emergency department can trigger a series of actions affecting the patient’s length of stay, including the wait for the radiologist’s report [3], thereby prompting inquiries into the accuracy of pediatric emergency physicians (PEPs) in interpreting brain NCCT compared to radiologists.
Although current medical literature lacks pediatric emergency-specific studies addressing this question, some studies in adult emergency medicine settings have been reported. For instance, Khan et al. [4] reported a Kappa value of 0.64 with 87.14% concordance between emergency physicians and radiologists in interpreting brain NCCT, while Zohair Al Aseri et al. [5] found a Kappa value of 0.672 with 91.6% concordance, both indicating substantial agreement [6]. In this present study, we aimed to prospectively assess the interpretive abilities of PEPs to interpret brain NCCT in a blinded manner.
2. Methods
This prospective study was conducted at the pediatric emergency medicine (PEM) unit, which receives approximately 450,000 annual visits of varying acuity levels, situated within King Saud University Medical City, a prominent tertiary care and level I trauma center in Riyadh, Saudi Arabia. The study was conducted over a one-year period, from May 2014 to May 2015, and included all pediatric patients (aged 14 years and younger) undergoing brain NCCT using a General Electric (GE) model High Definition (HD) 128 slice computed tomography (CT) machine. Patients whose brain NCCT interpretations were conducted by physicians other than PEPs, such as on-call radiologists or admitting medical services like neurologists or neurosurgeons, were excluded. A total of 202 patients were recruited and included in the analysis.
Treating physicians, comprising certified PEPs with 10–15 years of experience or certified general pediatricians with over 15 years of practice in PEM, were responsible for completing the study’s data collection sheet, which included patient identification and the indication for brain NCCT request. Following CT imaging, the initial interpretation and the final disposition plan for each patient were documented. No additional training or instructions were provided to physicians regarding brain NCCT interpretation. Radiological reports were subsequently traced through the radiology system.
An independent attending neuroradiologist, possessing 8 years of experience, along with two certified PEPs with over 10 years of experience, compared the interpretations made by PEPs with the official final reports dictated by the on-call radiologist. Each report was reviewed and categorized as either positive with no discrepancies or discrepant. Positive findings on brain NCCT included acute hemorrhage, skull fracture, space-occupying lesion, cerebral edema, acute hydrocephalus, facial bone fractures, or evidence of encephalitis. A positive brain NCCT with no discrepancies was noted when an agreement between the PEP and the on-call radiologist was reached regarding the etiology.
Data analysis was performed using the Statistical Package for Social Science software (IBM SPSS Statistics Grad Pack 28.0, Chicago, IL, USA). Sensitivity, specificity, concordance, and kappa coefficient were calculated with reference to the radiologist’s report, serving as the reference standard for evaluating inter-rater reliability between radiologists and PEPs. Universal Kappa agreement analysis [6] was conducted for this study. Furthermore, the McNemar test was utilized for conservative inference between binomial distributions, focusing on inter-rater disagreements, with a significance level set at p < 0.05.
3. Results
During the study duration, a total of 202 pediatric patients were assessed. All patients were under the age of 14, with a mean age of 4.8 ± 3.6 years. Among them, trauma was the predominant presenting complaint in 127 patients (62.9%), followed by seizures in 28 patients (13.9%). The other complaints are presented in Table 1.
| Characteristics | Descriptions | N (%) |
| Age (mon) | ||
| Min–Max | 1–168 | |
| Mean ± SD | 57.8 ± 43.1 | |
| Median (P25, P75) | 48 (21, 96) | |
| Complications | ||
| Trauma | 127 (62.9%) | |
| Headache | 21 (10.4%) | |
| Seizure | 28 (13.9%) | |
| Vomiting | 22 (10.9%) | |
| Soft tissue swelling of the eye, ear, scalp | 9 (4.5%) | |
| Known craniopathy for assessment Hx of brain tumor, VP shunt, etc. | 21 (10.4%) | |
| Cranial nerves concerning loss of vision, facial palsy, etc. | 4 (2.0%) | |
| Decreased level/loss of consciousness | 21 (10.4%) | |
| Dizziness or ataxia | 7 (3.5%) | |
| Others: (i.e., neck complaints, nasal bleeding, foreign bodies, isolated fever) | 21 (10.4%) | |
| Indication for brain NCCT is to look for possibility of the followings | ||
| Intra cranial Bleeding/injury (i.e., EDH, SAH, SDH, intraparenchymal cranial bleeding, hemorrhagic stroke, etc.) | 134 (66.3%) | |
| Skull fractures | 18 (8.9%) | |
| Parenchymal mass or space-occupying lesion | 22 (10.9%) | |
| Brain edema, midline shift | 3 (1.5%) | |
| Hydrocephalus with raised ICP | 20 (9.9%) | |
| Facial bone fracture | 4 (2.0%) | |
| Cervical spine injuries | 2 (1.0%) | |
| Others (encephalitis) | 8 (4.0%) | |
| Not documented | 13 (6.4%) | |
| ED interpretation | ||
| EDH | 9 (4.5%) | |
| SDH | 9 (4.5%) | |
| SAH | 3 (1.5%) | |
| Intraparenchymal cranial bleeding | 4 (2.0%) | |
| Skull fractures | 20 (9.9%) | |
| Ischemic stroke/changes | 0 (0.0%) | |
| Space occupying lesion | 0 (0.0%) | |
| Brain edema, shift | 1 (0.5%) | |
| Hydrocephalus with raised ICP | 17 (8.4%) | |
| Facial bone fracture | 0 (0.0%) | |
| Soft tissue edema/swelling | 9 (4.5%) | |
| Normal, chronic, nil acute | 146 (72.3%) | |
| Others: (Basal ganglia abnormality, thrombosis) | 0 (0.0%) | |
| Radiology report | ||
| EDH | 7 (3.5%) | |
| SDH | 7 (3.5%) | |
| SAH | 1 (0.5%) | |
| Intraparenchymal cranial bleeding | 3 (1.5%) | |
| Skull fractures | 29 (14.4%) | |
| Ischemic stroke/changes | 2 (1.0%) | |
| Space occupying lesion | 0 (0.0%) | |
| Brain edema, shift | 7 (3.5%) | |
| Hydrocephalus with raised ICP | 8 (4.0%) | |
| Facial bone fracture | 6 (3.0%) | |
| Soft tissue edema/swelling | 36 (17.8%) | |
| Normal, chronic, nil acute | 137 (67.8%) | |
| Others: (Basal ganglia abnormality, thrombosis) | 2 (1.0%) | |
| ED findings per patient | ||
| One | 43 (21.3%) | |
| Two | 10 (5.0%) | |
| Three | 3 (1.5%) | |
| None | 146 (72.3%) | |
| RAD findings per patient | ||
| One | 38 (18.8%) | |
| Two | 16 (7.9%) | |
| Three | 10 (5.0%) | |
| Four | 2 (1.0%) | |
| None | 136 (67.3%) | |
| ED vs. RAD | ||
| Agreement | 148 (73.3%) | |
| Disagreement | 54 (26.7%) | |
| EDH: epidural hematoma; SDH: subdural hemorrhage; SAH: subarachnoid hemorrhage; ICP: intracranial pressure; NCCT: non-contrast computed tomography; SD: standard deviation; VP: ventriculoperitoneal shunt; ED: emergency department; RAD: radiologist. |
The most common indication for brain NCCT requests by PEPs was to assess for intracranial bleeding, observed in 134 patients (66.3%), followed by the evaluation of space-occupying lesions in 22 patients (10.9%), and the assessment of hydrocephalus patients for potential raised intracranial pressure (ICP) in 20 patients (9.9%). Other indications, along with emergency department (ED) interpretations and radiology reports (Table 1).
PEPs reported 1–3 findings per patient, documenting a total of 72 lesions in 56 (27.8%) patients. The majority of PEP reports (n = 146; 72.3%) were classified as normal, chronic, or showing no acute abnormalities and were thus deemed normal. Similarly, radiologists observed 1 to 4 findings per patient, documenting 108 lesions in 66 (32.7%) patients. Most of the radiology reports (n = 137; 67.3%) were classified as normal.
Table 2 provides a summary of the inter-rater agreement between PEPs and radiologists regarding specific findings. The PEP interpretation was compared with the confirmed radiologist’s report for each specified finding. Inter-rater agreement is indicated by the sensitivity percentage score, with the association measure between PEPs and radiologists determined using kappa test results. The overall accuracy between PEP and radiologist findings was 82.18%, suggesting a moderate level of agreement [7] (Kappa = 0.578).
| Characteristic (N = 202) | Cell frequency | Measures of association | Agreement | p-value | |||||||||
| Site | ED Interpretation | Radiology Report | ED+/RAD+ | ED+/RAD− | ED−/RAD+ | ED−/RAD− | Sensitivity | Specificity | PPV | NPV | Accuracy | Kappa | McNemar |
| EDH | 9 | 7 | 3 | 6 | 4 | 189 | 42.86 | 96.92 | 33.33 | 97.93 | 95.05 | 0.350 | 0.754 |
| SDH | 9 | 7 | 4 | 5 | 3 | 190 | 57.14 | 97.44 | 44.44 | 98.45 | 96.04 | 0.480 | 0.727 |
| SAH | 3 | 1 | 1 | 2 | 0 | 199 | 100.00 | 99.00 | 33.33 | 100.00 | 99.01 | 0.496 | 0.500 |
| Intraparenchymal cranial bleeding | 4 | 3 | 2 | 2 | 1 | 197 | 66.67 | 98.99 | 50.00 | 99.49 | 98.51 | 0.564 | >0.999 |
| Skull fractures | 20 | 29 | 19 | 1 | 10 | 172 | 65.52 | 99.42 | 95.00 | 94.51 | 94.55 | 0.746 | 0.012 |
| Ischemic stroke/changes | 0 | 2 | 0 | 0 | 2 | 200 | 20.00 | 99.75 | 50.00 | 99.01 | 99.26 | … | … |
| Space occupying lesion | 0 | 0 | 0 | 0 | 0 | 202 | 50.00 | 99.75 | 50.00 | 99.75 | 100.00 | … | … |
| Brain edema, shift | 1 | 7 | 1 | 0 | 6 | 195 | 14.29 | 100.00 | 100.00 | 97.01 | 97.03 | 0.243 | 0.031 |
| Hydrocephalus with raised ICP | 17 | 8 | 7 | 10 | 1 | 184 | 87.50 | 94.85 | 41.18 | 99.46 | 94.55 | 0.535 | 0.012 |
| Facial bone fracture | 0 | 6 | 0 | 0 | 6 | 196 | 7.69 | 99.75 | 50.00 | 97.03 | 97.28 | … | … |
| Soft tissue edema/swelling | 9 | 36 | 7 | 2 | 29 | 164 | 19.44 | 98.80 | 77.78 | 84.97 | 84.65 | 0.258 | <0.001 |
| Normal, chronic, nil acute | 146 | 136 | 123 | 23 | 13 | 43 | 90.44 | 65.15 | 84.25 | 76.79 | 82.18 | 0.565 | 0.188 |
| Others: (Basal ganglia abnormality, thrombosis) | 0 | 2 | 0 | 0 | 2 | 200 | 20.00 | 99.75 | 50.00 | 99.01 | 99.26 | … | … |
| Overall | 56 | 66 | 43 | 13 | 23 | 123 | 65.15 | 90.44 | 76.79 | 84.25 | 82.18 | 0.578 | 0.132 |
| EDH: epidural hematoma; SDH: subdural hemorrhage; SAH: subarachnoid hemorrhage; ICP: intracranial pressure; ED: emergency department; RAD: radiologist; PPV: positive predictive value; NPV: negative predictive value. |
The agreement analysis presented in Table 2 indicates varying levels of agreement across specific findings. Epidural hemorrhage (EDH) showed fair agreement (kappa = 0.350), while subdural hemorrhage (SDH), subarachnoid hemorrhage (SAH), intraparenchymal cranial bleeding, and findings categorized as normal, chronic, or non-acute exhibited moderate agreement. Skull fractures demonstrated substantial kappa agreement (0.746), attributed to occurrences of both false positives and false negatives in the confusion matrix, with a significant difference (p = 0.012). Additionally, brain edema and shift, hydrocephalus with raised ICP, and soft tissue edema/swelling revealed significant confusion.
4. Discussion
Our study revealed moderate agreement between the interpretation of brain NCCT scans conducted by PEPs and radiologists, with a kappa value of 0.578 and an accuracy of 82.18%. Studies specifically focusing on the pediatric population are scarce, with most either including adults exclusively or mixed populations without subgroup analysis for pediatric subjects. Nonetheless, our findings are comparable to prior studies conducted in adult populations. For instance, A. Khan et al. [4] demonstrated substantial agreement between emergency physicians (EPs) and radiologists in interpreting NCCT scans, with a kappa value of 0.64 and an accuracy of 87.14% in their adult patients. Similarly, Zohair Al Aseri et al. [5] also reported substantial agreement between EPs and radiologists, albeit with a higher accuracy rate of 91.5%. Discrepancies between our results and theirs could stem from variations in sample size, the heterogeneity of their data encompassing all age groups compared to our focus on children only, and differences in the ability of PEPs to discern abnormal or normal brain NCCT in children versus emergency physicians’ proficiency in adult cases. Given the moderate overall agreement observed in our study, we advocate for the necessity of radiologist interpretation of brain NCCT scans. Moreover, we posit that further training for PEPs in brain NCCT interpretation may enhance agreement between PEPs and radiologists.
Subdural hemorrhage (SDH), epidural hematoma (EDH) and subarachnoid hemorrhage (SAH) have been associated with substantial neuro-disability in affected children. Therefore, rapid and accurate decision-making guided by brain NCCT findings is imperative for optimal patient management [8, 9]. In this present study, we observed accuracy rates of 96.04%, 95.05% and 99.01% for diagnosing SDH, EDH and SAH, respectively, which are comparable to findings from other studies conducted exclusively in adult populations [4, 5].
This study had some limitations worth mentioning. First it was conducted few years ago. However, given the absence of published reports specifically focused on pediatrics, our data remains valuable for researchers interested in further exploration of this topic and for comparison with more recent findings if they become available. Another limitation is the small sample size from a single center, which may hinder the generalizability of the results. To address this, future studies could involve multicenter collaborations with larger sample sizes and the analysis of additional variables that could potentially influence the study outcomes.
5. Conclusions
In conclusion, our study demonstrates a moderate level of accuracy in brain NCCT interpretation between PEPs and radiologists. However, further evaluation is warranted through multicenter pediatric emergency-based studies with larger sample sizes to validate our findings more comprehensively.
Availability of data and materials
The data presented in this study are available on reasonable request from the corresponding author.
Author contributions
ZAA—designed the research study. HMBS—performed the research. MAA, AMAA and RSS—analyzed the data. AAA and HMBS—wrote the manuscript. All authors read and approved the final manuscript.
Ethics approval and consent to participate
This study was approved by the Institutional Review Board (IRB) for Health Sciences Colleges Research on Human Subjects of King Saud University, the informed consent for participants was waived, with project number E-216106.
Acknowledgment
Not applicable.
Funding
This research received no external funding.
Conflict of interest
The authors declare no conflict of interest.
