A real-world intelligent system for the diagnosis and triage of COVID-19 in the emergency department
1Department of Software Engineering, DiCITS Lab, iMUDS, DaSci, University of Granada, 18071 Granada, Spain
2Department of Computer Science and Artificial Intelligence, DiCITS Lab, iMUDS, DaSci, University of Granada, 18071 Granada, Spain
3Department of Information and Communication Technologies, Andalusian Health Service, 41001 Andalucia, Spain
4Department of Radiology, Hospital Universitario Virgen de las Nieves, 18014 Granada, Spain
5Biosanitary Institute of Granada (ibs.GRANADA), 18014 Granada, Spain
6Rural Engineering Department, DiCITS Lab, University of Cordoba, 14005 Cordoba, Spain
Submitted: 18 February 2022 Accepted: 07 May 2022
Online publish date: 13 October 2022
The Coronavirus Disease 2019 (COVID-19) pandemic has had an unprecedented impact on healthcare systems, prompting the need to improve the triaging of patients in the Emergency Department (ED). This could be achieved by automatic analysis of chest X-rays (CXR) using Artificial Intelligence (AI). We conducted a research project to generate and thoroughly document the development process of an intelligent system for COVID-19 diagnosis. This work aims at explaining the problem formulation, data collection and pre-processing, use of base convolutional neural networks to approach our diagnostic problem, the process of network building and how our model was validated to reach the final diagnostic system. Using publicly available datasets and a locally obtained dataset with more than 100,000 potentially eligible CXR images, we developed an intelligent diagnostic system that achieves an average performance of 93% success. Then, we implemented a web-based interface that will allow its use in real-world medical practice, with an average response time of less than 1 second. There were some limitations in the application of the diagnostic system to our local dataset which precluded obtaining high diagnostic performance. Although not all these limitations are straightforward, the most relevant ones are discussed, along with potential solutions. Further research is warranted to overcome the limitations of state-of-the-art AI systems used for the imaging diagnosis of COVID-19 in the ED.
COVID-19; Diagnosis artificial intelligence; Emergency care; Epidemiology; SARS-CoV-2
Miguel Lastra Leidinger,Francisco Aragón Royón,Oier Etxeberria,Luis Balderas,Antonio Jesús Láinez Ramos-Bossini,Genaro López Milena,Liz Alfonso,Rosario Moreno,Antonio Arauzo,José M. Benítez. A real-world intelligent system for the diagnosis and triage of COVID-19 in the emergency department. Signa Vitae. 2022.doi:10.22514/sv.2022.070.
 Rivera-Izquierdo M, Láinez-Ramos-Bossini AJ, de Alba IG, Ortiz-González-Serna R, Serrano-Ortiz Á, Fernández-Martínez NF, et al. Long COVID 12 months after discharge: persistent symptoms in patients hospitalised due to COVID-19 and patients hospitalised due to other causes—a multicentre cohort study. BMC Medicine. 2022; 20: 92.
 Hakim M, Khattak FA, Muhammad S, Ismail M, Ullah N, Atiq Orakzai M, et al. Access and use experience of personal protective equipment among frontline healthcare workers in Pakistan during the COVID-19 emergency: a cross-sectional study. Health Security. 2021; 19: 140–149.
 Rivera-Izquierdo M, Valero-Ubierna MDC, R-delAmo JL, Fernández-García MÁ, Martínez-Diz S, Tahery-Mahmoud A, et al. Therapeutic agents tested in 238 COVID-19 hospitalized patients and their relation-ship with mortality. Medicina Clinica. 2020; 155: 375–381.
 Rivera López E, Abal F, Rekers R, Holzer F, Melamed I, Salmún D, et al. Proposal for the elaboration of a triage guideline in the context of the COVID-19 pandemic. Revista de bioetica y derecho & perspectivas bioeticas. 2020; 2020: 37–61.
 Alhalaseh YN, Elshabrawy HA, Erashdi M, Shahait M, Abu-Humdan AM, Al-Hussaini M. Allocation of the ‘already’ limited medical resources amid the COVID-19 pandemic, an iterative ethical encounter including suggested solutions from a real life encounter. Frontiers in Medicine. 2021; 7: 616277.
 Rivera-Olivero IA, Henríquez-Trujillo AR, Kyriakidis NC, Ortiz-Prado E, Laglaguano JC, Vallejo-Janeta AP, et al. Diagnostic performance of seven commercial COVID-19 serology tests available in South America. Frontiers in Cellular and Infection Microbiology. 2022; 12: 787987.
 Chowdhury MEH, Rahman T, Khandakar A, Mazhar R, Kadir MA, Mahbub ZB, et al. Can AI help in screening viral and COVID-19 pneumonia? IEEE Access. 2020; 8: 132665–132676.
 Kim CK, Choi JW, Jiao Z, Wang D, Wu J, Yi TY, et al. An automated COVID-19 triage pipeline using artificial intelligence based on chest radiographs and clinical data. Npj Digital Medicine. 2022; 5: 5.
 Mei X, Lee H, Diao K, Huang M, Lin B, Liu C, et al. Artificial intelligence—enabled rapid diagnosis of patients with COVID-19. Nature Medicine. 2020; 26: 1224–1228.
 Ozturk T, Talo M, Yildirim EA, Baloglu UB, Yildirim O, Rajendra Acharya U. Automated detection of COVID-19 cases using deep neural networks with X-ray images. Computers in Biology and Medicine. 2020; 121: 103792.
 Puppe F. Systematic introduction to expert systems: knowledge represen-tations and problem-solving methods. 1st. Springer-Verlag: Berlin. 1993.
 Kiseleva TV, Toropchina EV. Expert systems in medical diagnostics: analysis of functioning. Automation and Remote Control. 2014; 75: 1316–1322.
 Rajkomar A, Dean J, Kohane I. Machine learning in medicine. New England Journal of Medicine. 2019; 380: 1347–1358.
 Sajda P. Machine learning for detection and diagnosis of disease. Annual Review of Biomedical Engineering. 2006; 8: 537–565.
 Schulz H, Behnke S. Deep learning. German Journal of Artificial Intelligence. 2012; 26: 357–363.
 Bakator M, Radosav D. Deep learning and medical diagnosis: a review of literature. Multimodal Technologies and Interaction. 2018; 2: 47.
 Bustos N, Tello M, Droppelmann G, García N, Feijoo F, Leiva V. Machine learning techniques as an efficient alternative diagnostic tool for COVID-19 cases. Signa Vitae. 2022; 18: 23–33.
 Zhang J, Xie Y, Wu Q, Xia Y. Medical image classification using synergic deep learning. Medical Image Analysis. 2019; 54: 10–19.
 Lu D, Weng Q. A survey of image classification methods and techniques for improving classification performance. International Journal of Remote Sensing. 2007; 28: 823–870.
 Castiglioni I, Rundo L, Codari M, Di Leo G, Salvatore C, Interlenghi M, et al. AI applications to medical images: from machine learning to deep learning. European Journal of Medical Physics. 2021; 83: 9–24.
 Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017; 542: 115–118.
 Tartaglione E, Barbano CA, Berzovini C, Calandri M, Grangetto M. Unveiling COVID-19 from chest X-Ray with deep learning: a hurdles race with small data. International Journal of Environmental Research and Public Health. 2020; 17: 6933.
 Roberts M, Driggs D, Thorpe M, Gilbey J, Yeung M, Ursprung S, et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nature Machine Intelligence. 2021; 3: 199–217.
 Goodfellow I, Bengio Y, Courville A. Deep learning. Edition. The MIT Press: Cambrige. 2016.
 Deng J, Dong W, Socher R, Li L, Kai Li, Li Fei-Fei. ImageNet: a large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009; 248–255.
 Simonyan K, Zisserman A. ‘Very deep convolutional networks for large-scale image recognition’, The 3rd International Conference on Learning Representations. San Diego, California, United States. 2015.
 Howard A, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, et al. MobileNets: efficient convolutional neural networks for mobile vision applications. 2017. Available at: https://arxiv.org/abs/ 1704.04861 (Accessed: Day 13 February 2022).
 He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016; 770–778.
 Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016; 2818–2826.
 Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017; 4700–4708.
 Chollet F. Xception: deep learning with depthwise separable convo-lutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017; 1251–1258.
 Wang L, Lin ZQ, Wong A. COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images. Scientific Reports. 2020; 10: 19549.
 Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L. MobileNetV2: inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018; 4510–4520.
 Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017; 2097–2106.
 Ronneberger O, Fischer P, Brox T. ‘U-net: convolutional networks for biomedical image segmentation’, International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany, 2015. Springer International Publishing: Switzerland. 2015.
 Wang L, Zhang Y, Whang D, Tong X, Liu T, Zhang S, et al. Artificial intelligence for COVID-19: a systematic review. Frontiers in Medicine. 2021; 8: 704256.
Science Citation Index Expanded (SciSearch) Created as SCI in 1964, Science Citation Index Expanded now indexes over 9,200 of the world’s most impactful journals across 178 scientific disciplines. More than 53 million records and 1.18 billion cited references date back from 1900 to present.
Journal Citation Reports/Science Edition Journal Citation Reports/Science Edition aims to evaluate a journal’s value from multiple perspectives including the journal impact factor, descriptive data about a journal’s open access content as well as contributing authors, and provide readers a transparent and publisher-neutral data & statistics information about the journal.
Chemical Abstracts Service Source Index The CAS Source Index (CASSI) Search Tool is an online resource that can quickly identify or confirm journal titles and abbreviations for publications indexed by CAS since 1907, including serial and non-serial scientific and technical publications.
IndexCopernicus The Index Copernicus International (ICI) Journals database’s is an international indexation database of scientific journals. It covered international scientific journals which divided into general information, contents of individual issues, detailed bibliography (references) sections for every publication, as well as full texts of publications in the form of attached files (optional). For now, there are more than 58,000 scientific journals registered at ICI.
Geneva Foundation for Medical Education and Research The Geneva Foundation for Medical Education and Research (GFMER) is a non-profit organization established in 2002 and it works in close collaboration with the World Health Organization (WHO). The overall objectives of the Foundation are to promote and develop health education and research programs.
Scopus: CiteScore 0.5(2021) Scopus is Elsevier's abstract and citation database launched in 2004. Scopus covers nearly 36,377 titles (22,794 active titles and 13,583 Inactive titles) from approximately 11,678 publishers, of which 34,346 are peer-reviewed journals in top-level subject fields: life sciences, social sciences, physical sciences and health sciences.
Embase Embase (often styled EMBASE for Excerpta Medica dataBASE), produced by Elsevier, is a biomedical and pharmacological database of published literature designed to support information managers and pharmacovigilance in complying with the regulatory requirements of a licensed drug.