Hussein Ibrahim1,2,3, Xiaoxuan Liu1,2,3,4,5, Samantha Cruz Rivera3,6,7, David Moher8,9, An-Wen Chan10, Matthew R Sydes4,11, Melanie J Calvert3,5,6,7,12,13,14, Alastair K Denniston15,16,17,18,19,20. 1. Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, B15 2TT, UK. 2. University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK. 3. Centre for Regulatory Science and Innovation, Birmingham Health Partners, Birmingham, UK. 4. Moorfields Eye Hospital NHS Foundation Trust, London, UK. 5. Health Data Research UK, London, UK. 6. Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK. 7. Institute of Applied Health Research, University of Birmingham, Birmingham, UK. 8. Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. 9. School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada. 10. Department of Medicine, Women's College Research Institute, Women's College Hospital, University of Toronto, Toronto, Ontario, Canada. 11. Medical Research Council Clinical Trials Unit at University College London, Institute of Clinical Trials and Methodology, University College London, London, UK. 12. National Institute for Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK. 13. National Institute for Health Research Applied Research Collaborative West Midlands, Coventry, UK. 14. National Institute for Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK. 15. Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, B15 2TT, UK. a.denniston@bham.ac.uk. 16. University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK. a.denniston@bham.ac.uk. 17. Centre for Regulatory Science and Innovation, Birmingham Health Partners, Birmingham, UK. a.denniston@bham.ac.uk. 18. Health Data Research UK, London, UK. a.denniston@bham.ac.uk. 19. Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK. a.denniston@bham.ac.uk. 20. National Institute for Health Research Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust and University College London Institute of Ophthalmology, London, UK. a.denniston@bham.ac.uk.
Abstract
BACKGROUND: The application of artificial intelligence (AI) in healthcare is an area of immense interest. The high profile of 'AI in health' means that there are unusually strong drivers to accelerate the introduction and implementation of innovative AI interventions, which may not be supported by the available evidence, and for which the usual systems of appraisal may not yet be sufficient. MAIN TEXT: We are beginning to see the emergence of randomised clinical trials evaluating AI interventions in real-world settings. It is imperative that these studies are conducted and reported to the highest standards to enable effective evaluation because they will potentially be a key part of the evidence that is used when deciding whether an AI intervention is sufficiently safe and effective to be approved and commissioned. Minimum reporting guidelines for clinical trial protocols and reports have been instrumental in improving the quality of clinical trials and promoting completeness and transparency of reporting for the evaluation of new health interventions. The current guidelines-SPIRIT and CONSORT-are suited to traditional health interventions but research has revealed that they do not adequately address potential sources of bias specific to AI systems. Examples of elements that require specific reporting include algorithm version and the procedure for acquiring input data. In response, the SPIRIT-AI and CONSORT-AI guidelines were developed by a multidisciplinary group of international experts using a consensus building methodological process. The extensions include a number of new items that should be reported in addition to the core items. Each item, where possible, was informed by challenges identified in existing studies of AI systems in health settings. CONCLUSION: The SPIRIT-AI and CONSORT-AI guidelines provide the first international standards for clinical trials of AI systems. The guidelines are designed to ensure complete and transparent reporting of clinical trial protocols and reports involving AI interventions and have the potential to improve the quality of these clinical trials through improvements in their design and delivery. Their use will help to efficiently identify the safest and most effective AI interventions and commission them with confidence for the benefit of patients and the public.
BACKGROUND: The application of artificial intelligence (AI) in healthcare is an area of immense interest. The high profile of 'AI in health' means that there are unusually strong drivers to accelerate the introduction and implementation of innovative AI interventions, which may not be supported by the available evidence, and for which the usual systems of appraisal may not yet be sufficient. MAIN TEXT: We are beginning to see the emergence of randomised clinical trials evaluating AI interventions in real-world settings. It is imperative that these studies are conducted and reported to the highest standards to enable effective evaluation because they will potentially be a key part of the evidence that is used when deciding whether an AI intervention is sufficiently safe and effective to be approved and commissioned. Minimum reporting guidelines for clinical trial protocols and reports have been instrumental in improving the quality of clinical trials and promoting completeness and transparency of reporting for the evaluation of new health interventions. The current guidelines-SPIRIT and CONSORT-are suited to traditional health interventions but research has revealed that they do not adequately address potential sources of bias specific to AI systems. Examples of elements that require specific reporting include algorithm version and the procedure for acquiring input data. In response, the SPIRIT-AI and CONSORT-AI guidelines were developed by a multidisciplinary group of international experts using a consensus building methodological process. The extensions include a number of new items that should be reported in addition to the core items. Each item, where possible, was informed by challenges identified in existing studies of AI systems in health settings. CONCLUSION: The SPIRIT-AI and CONSORT-AI guidelines provide the first international standards for clinical trials of AI systems. The guidelines are designed to ensure complete and transparent reporting of clinical trial protocols and reports involving AI interventions and have the potential to improve the quality of these clinical trials through improvements in their design and delivery. Their use will help to efficiently identify the safest and most effective AI interventions and commission them with confidence for the benefit of patients and the public.
Entities:
Keywords:
Artificial intelligence; Checklist; Clinical trials; Guidelines; Machine learning; Randomised controlled trials; Research design; Research report
Authors: David Moher; Sally Hopewell; Kenneth F Schulz; Victor Montori; Peter C Gøtzsche; P J Devereaux; Diana Elbourne; Matthias Egger; Douglas G Altman Journal: BMJ Date: 2010-03-23
Authors: Michael David Abràmoff; Yiyue Lou; Ali Erginay; Warren Clarida; Ryan Amelon; James C Folk; Meindert Niemeijer Journal: Invest Ophthalmol Vis Sci Date: 2016-10-01 Impact factor: 4.799
Authors: Pu Wang; Tyler M Berzin; Jeremy Romek Glissen Brown; Shishira Bharadwaj; Aymeric Becq; Xun Xiao; Peixi Liu; Liangping Li; Yan Song; Di Zhang; Yi Li; Guangre Xu; Mengtian Tu; Xiaogang Liu Journal: Gut Date: 2019-02-27 Impact factor: 23.059
Authors: Nichole S Tyler; Clara M Mosquera-Lopez; Leah M Wilson; Robert H Dodier; Deborah L Branigan; Virginia B Gabo; Florian H Guillot; Wade W Hilts; Joseph El Youssef; Jessica R Castle; Peter G Jacobs Journal: Nat Metab Date: 2020-06-01
Authors: Michael Yu; An Tang; Kip Brown; Rima Bouchakri; Pascal St-Onge; Sheng Wu; John Reeder; Louis Mullie; Michaël Chassé Journal: BMJ Date: 2021-12-31
Authors: Simon Williams; Hugo Layard Horsfall; Jonathan P Funnell; John G Hanrahan; Danyal Z Khan; William Muirhead; Danail Stoyanov; Hani J Marcus Journal: Cancers (Basel) Date: 2021-10-07 Impact factor: 6.639
Authors: Andrej Thurzo; Wanda Urbanová; Bohuslav Novák; Ladislav Czako; Tomáš Siebert; Peter Stano; Simona Mareková; Georgia Fountoulaki; Helena Kosnáčová; Ivan Varga Journal: Healthcare (Basel) Date: 2022-07-08