Tom Dyer1, Sanjiv Chawda2, Raed Alkilani2, Tom Naunton Morgan3, Mike Hughes3, Simon Rasalingham3. 1. Behold.ai, 180 Borough high St, London, SE1 1LB, UK. tomd@behold.ai. 2. Department of Radiology, Barking, Havering and Redbridge University Hospitals NHS Trust, Romford, RM7 0AG, UK. 3. Behold.ai, 180 Borough high St, London, SE1 1LB, UK.
Abstract
PURPOSE: Non-contrast CT head scans provide rapid and accurate diagnosis of acute head injury; however, increased utilisation of CT head scans makes it difficult to prioritise acutely unwell patients and places pressure on busy emergency departments (EDs). This study validates an AI algorithm to triage patients presenting with Intracranial Haemorrhage (ICH) or Acute Infarct whilst also identifying a subset of patients as Normal, with the potential to function as a rule-out test. METHODS: In total, 390 CT head scans were collected from 3 institutions in the UK, US and India. Ground-truth labels were assigned by 3 FRCR consultant radiologists. AI performance, as well as the performance of 3 independent radiologists, was measured against ground-truth labels. RESULTS: The algorithm showed AUC values of 0.988 (0.978-0.994), 0.933 (0.901-0.961) and 0.939 (0.919-0.958) for ICH, Acute Infarct and Normal, respectively. Sensitivity/specificity for ICH and Acute Infarct were 0.988/0.925 and 0.833/0.927, respectively, compared to 0.907/0.991 and 0.618/0.977 for radiologists. AI rule-out of Normal scans achieved 0.93% negative predictive value (NPV) for the removal of 54.3% of Normal cases, compared to 86.8% NPV for radiologists. CONCLUSION: We show our algorithm can provide effective triage of ICH and Acute Infarct to prioritise acutely unwell patients. AI can also benefit clinical accuracy, with the algorithm identifying 91.3% of radiologist false negatives for ICH and 69.1% for Acute Infarct. Rule-out of Normal scans has huge potential for workload management in busy EDs, in this case removing 27.4% of all scans with no acute findings missed.
PURPOSE: Non-contrast CT head scans provide rapid and accurate diagnosis of acute head injury; however, increased utilisation of CT head scans makes it difficult to prioritise acutely unwell patients and places pressure on busy emergency departments (EDs). This study validates an AI algorithm to triage patients presenting with Intracranial Haemorrhage (ICH) or Acute Infarct whilst also identifying a subset of patients as Normal, with the potential to function as a rule-out test. METHODS: In total, 390 CT head scans were collected from 3 institutions in the UK, US and India. Ground-truth labels were assigned by 3 FRCR consultant radiologists. AI performance, as well as the performance of 3 independent radiologists, was measured against ground-truth labels. RESULTS: The algorithm showed AUC values of 0.988 (0.978-0.994), 0.933 (0.901-0.961) and 0.939 (0.919-0.958) for ICH, Acute Infarct and Normal, respectively. Sensitivity/specificity for ICH and Acute Infarct were 0.988/0.925 and 0.833/0.927, respectively, compared to 0.907/0.991 and 0.618/0.977 for radiologists. AI rule-out of Normal scans achieved 0.93% negative predictive value (NPV) for the removal of 54.3% of Normal cases, compared to 86.8% NPV for radiologists. CONCLUSION: We show our algorithm can provide effective triage of ICH and Acute Infarct to prioritise acutely unwell patients. AI can also benefit clinical accuracy, with the algorithm identifying 91.3% of radiologist false negatives for ICH and 69.1% for Acute Infarct. Rule-out of Normal scans has huge potential for workload management in busy EDs, in this case removing 27.4% of all scans with no acute findings missed.
Authors: Hyunkwang Lee; Sehyo Yune; Mohammad Mansouri; Myeongchan Kim; Shahein H Tajmir; Claude E Guerrier; Sarah A Ebert; Stuart R Pomerantz; Javier M Romero; Shahmir Kamalian; Ramon G Gonzalez; Michael H Lev; Synho Do Journal: Nat Biomed Eng Date: 2018-12-17 Impact factor: 25.671
Authors: Adam E Flanders; Luciano M Prevedello; George Shih; Safwan S Halabi; Jayashree Kalpathy-Cramer; Robyn Ball; John T Mongan; Anouk Stein; Felipe C Kitamura; Matthew P Lungren; Gagandeep Choudhary; Lesley Cala; Luiz Coelho; Monique Mogensen; Fanny Morón; Elka Miller; Ichiro Ikuta; Vahe Zohrabian; Olivia McDonnell; Christie Lincoln; Lubdha Shah; David Joyner; Amit Agarwal; Ryan K Lee; Jaya Nath Journal: Radiol Artif Intell Date: 2020-07-29
Authors: Tudor G Jovin; Angel Chamorro; Erik Cobo; María A de Miquel; Carlos A Molina; Alex Rovira; Luis San Román; Joaquín Serena; Sonia Abilleira; Marc Ribó; Mònica Millán; Xabier Urra; Pere Cardona; Elena López-Cancio; Alejandro Tomasello; Carlos Castaño; Jordi Blasco; Lucía Aja; Laura Dorado; Helena Quesada; Marta Rubiera; María Hernandez-Pérez; Mayank Goyal; Andrew M Demchuk; Rüdiger von Kummer; Miquel Gallofré; Antoni Dávalos Journal: N Engl J Med Date: 2015-04-17 Impact factor: 91.245
Authors: Weicheng Kuo; Christian Hӓne; Pratik Mukherjee; Jitendra Malik; Esther L Yuh Journal: Proc Natl Acad Sci U S A Date: 2019-10-21 Impact factor: 11.205