Anish Bhuva1,2, Wenjia Bai1, Clement Lau1, Rhodri Davies1,2, Yang Ye1,2, Heeraj Bulluck1,2, Elisa McAlindon1,2, Veronica Culotta1, Peter Swoboda1,2, Gabriella Captur1,2, Thomas Treibel2,3, Joao Augusto2, Kristopher Knott2,4, Andreas Seraphim2, Graham Cole2,3, Steffen Petersen5, Nicola Edwards6, John Greenwood7,8, Chiara Bucciarelli-Ducci7, Alun Hughes9, Daniel Rueckert9, James Moon10, Charlotte Manisty11. 1. Institute for Cardiovascular Science, University College London, United Kingdom 2. Department of Cardiovascular Imaging, Barts Heart Centre, Barts Health NHS Trust, London, United Kingdom 3. Imperial College London, South Kensington Campus, United Kingdom. William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, United Kingdom 4. Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University, Hangzhou, People's Republic of China 5. Data Science Institute and Department of Medicine (W.B.),zzm321990Department of Computing 6. Data Science Institute and Department of Medicine 7. Bristol Heart Institute, Bristol NIHR Biomedical Research Centre, University Hospitals Bristol NHS Trust and Universityzzm321990of Bristol, United Kingdom 8. Heart and Lung Centre, New Cross Hospital, Wolverhampton, United Kingdom 9. Multidisciplinary Cardiovascular Research Centre and Division of Biomedical Imaging, Leeds Institute of Cardiovascularzzm321990and Metabolic Medicine, University of Leeds, United Kingdom 10. Imperial College London, National Heart and Lung Institute, Hammersmith Hospital, United Kingdom 11. Auckland City Hospital, New Zealand and Institute of Cardiovascular Science, University of Birmingham
Abstract
BACKGROUND: Automated analysis of cardiac structure and function using machine learning (ML) has great potential, but is currently hindered by poor generalizability. Comparison is traditionally against clinicians as a reference, ignoring inherent human inter- and intraobserver error, and ensuring that ML cannot demonstrate superiority. Measuring precision (scan:rescan reproducibility) addresses this. We compared precision of ML and humans using a multicenter, multi-disease, scan:rescan cardiovascular magnetic resonance data set. METHODS: One hundred ten patients (5 disease categories, 5 institutions, 2 scanner manufacturers, and 2 field strengths) underwent scan:rescan cardiovascular magnetic resonance (96% within one week). After identification of the most precise human technique, left ventricular chamber volumes, mass, and ejection fraction were measured by an expert, a trained junior clinician, and a fully automated convolutional neural network trained on 599 independent multicenter disease cases. Scan:rescan coefficient of variation and 1000 bootstrapped 95% CIs were calculated and compared using mixed linear effects models. RESULTS: Clinicians can be confident in detecting a 9% change in left ventricular ejection fraction, with greater than half of coefficient of variation attributable to intraobserver variation. Expert, trained junior, and automated scan:rescan precision were similar (for left ventricular ejection fraction, coefficient of variation 6.1 [5.2%-7.1%], P=0.2581; 8.3 [5.6%-10.3%], P=0.3653; 8.8 [6.1%-11.1%], P=0.8620). Automated analysis was 186× faster than humans (0.07 versus 13 minutes). CONCLUSIONS: Automated ML analysis is faster with similar precision to the most precise human techniques, even when challenged with real-world scan:rescan data. Assessment of multicenter, multi-vendor, multi-field strength scan:rescan data (available at www.thevolumesresource.com) permits a generalizable assessment of ML precision and may facilitate direct translation of ML to clinical practice.
BACKGROUND: Automated analysis of cardiac structure and function using machine learning (ML) has great potential, but is currently hindered by poor generalizability. Comparison is traditionally against clinicians as a reference, ignoring inherent human inter- and intraobserver error, and ensuring that ML cannot demonstrate superiority. Measuring precision (scan:rescan reproducibility) addresses this. We compared precision of ML and humans using a multicenter, multi-disease, scan:rescan cardiovascular magnetic resonance data set. METHODS: One hundred ten patients (5 disease categories, 5 institutions, 2 scanner manufacturers, and 2 field strengths) underwent scan:rescan cardiovascular magnetic resonance (96% within one week). After identification of the most precise human technique, left ventricular chamber volumes, mass, and ejection fraction were measured by an expert, a trained junior clinician, and a fully automated convolutional neural network trained on 599 independent multicenter disease cases. Scan:rescan coefficient of variation and 1000 bootstrapped 95% CIs were calculated and compared using mixed linear effects models. RESULTS: Clinicians can be confident in detecting a 9% change in left ventricular ejection fraction, with greater than half of coefficient of variation attributable to intraobserver variation. Expert, trained junior, and automated scan:rescan precision were similar (for left ventricular ejection fraction, coefficient of variation 6.1 [5.2%-7.1%], P=0.2581; 8.3 [5.6%-10.3%], P=0.3653; 8.8 [6.1%-11.1%], P=0.8620). Automated analysis was 186× faster than humans (0.07 versus 13 minutes). CONCLUSIONS: Automated ML analysis is faster with similar precision to the most precise human techniques, even when challenged with real-world scan:rescan data. Assessment of multicenter, multi-vendor, multi-field strength scan:rescan data (available at www.thevolumesresource.com) permits a generalizable assessment of ML precision and may facilitate direct translation of ML to clinical practice.
Entities:
Keywords:
artificial intelligence; image processing; left ventricular remodeling; magnetic resonance imaging, cine; ventricular function
Authors: Nadine Kawel-Boehm; Scott J Hetzel; Bharath Ambale-Venkatesh; Gabriella Captur; Christopher J Francois; Michael Jerosch-Herold; Michael Salerno; Shawn D Teague; Emanuela Valsangiacomo-Buechel; Rob J van der Geest; David A Bluemke Journal: J Cardiovasc Magn Reson Date: 2020-12-14 Impact factor: 5.364
Authors: Andreas M Rauschecker; Tyler J Gleason; Pierre Nedelec; Michael Tran Duong; David A Weiss; Evan Calabrese; John B Colby; Leo P Sugrue; Jeffrey D Rudie; Christopher P Hess Journal: Radiol Artif Intell Date: 2021-11-10
Authors: Nickolas Forsch; Sachin Govil; James C Perry; Sanjeet Hegde; Alistair A Young; Jeffrey H Omens; Andrew D McCulloch Journal: J Comput Sci Date: 2020-09-19
Authors: Nicola C Edwards; Anna M Price; Samir Mehta; Thomas F Hiemstra; Amreen Kaur; Peter J Greasley; David J Webb; Neeraj Dhaun; Iain M MacIntyre; Tariq Farrah; Vanessa Melville; Anna S Herrey; Gemma Slinn; Rebekah Wale; Natalie Ives; David C Wheeler; Ian Wilkinson; Richard P Steeds; Charles J Ferro; Jonathan N Townend Journal: Clin J Am Soc Nephrol Date: 2021-08-30 Impact factor: 10.614
Authors: Anke Busse; Rengarajan Rajagopal; Seyrani Yücel; Ebba Beller; Alper Öner; Felix Streckenbach; Daniel Cantré; Hüseyin Ince; Marc-André Weber; Felix G Meinel Journal: Radiologe Date: 2020-11 Impact factor: 0.635
Authors: Wenjia Bai; Hideaki Suzuki; Jian Huang; Catherine Francis; Shuo Wang; Giacomo Tarroni; Florian Guitton; Nay Aung; Kenneth Fung; Steffen E Petersen; Stefan K Piechnik; Stefan Neubauer; Evangelos Evangelou; Abbas Dehghan; Declan P O'Regan; Martin R Wilkins; Yike Guo; Paul M Matthews; Daniel Rueckert Journal: Nat Med Date: 2020-08-24 Impact factor: 87.241