Ji Eun Park1, Donghyun Kim2, Ho Sung Kim3, Seo Young Park4, Jung Youn Kim5, Se Jin Cho1, Jae Ho Shin6, Jeong Hoon Kim7. 1. Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 43 Olympic-ro 88, Songpa-Gu, Seoul, 05505, South Korea. 2. Department of Radiology, Inje University Busan Paik Hospital, Busan, South Korea. 3. Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 43 Olympic-ro 88, Songpa-Gu, Seoul, 05505, South Korea. radhskim@gmail.com. 4. Department of Clinical Epidemiology and Biostatistics, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea. 5. Department of Radiology, Kangbuk Samsung Medical Center, Seoul, South Korea. 6. St. Vincent Hospital, College of Medicine, The Catholic University of Korea, Suwon, South Korea. 7. Department of Neurosurgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea.
Abstract
OBJECTIVES: To evaluate radiomics studies according to radiomics quality score (RQS) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) to provide objective measurement of radiomics research. MATERIALS AND METHODS: PubMed and Embase were searched for studies published in high clinical imaging journals until December 2018 using the terms "radiomics" and "radiogenomics." Studies were scored against the items in the RQS and TRIPOD guidelines. Subgroup analyses were performed for journal type (clinical vs. imaging), intended use (diagnostic vs. prognostic), and imaging modality (CT vs. MRI), and articles were compared using Fisher's exact test and Mann-Whitney analysis. RESULTS: Seventy-seven articles were included. The mean RQS score was 26.1% of the maximum (9.4 out of 36). The RQS was low in demonstration of clinical utility (19.5%), test-retest analysis (6.5%), prospective study (3.9%), and open science (3.9%). None of the studies conducted a phantom or cost-effectiveness analysis. The adherence rate for TRIPOD was 57.8% (mean) and was particularly low in reporting title (2.6%), stating study objective in abstract and introduction (7.8% and 16.9%), blind assessment of outcome (14.3%), sample size (6.5%), and missing data (11.7%) categories. Studies in clinical journals scored higher and more frequently adopted external validation than imaging journals. CONCLUSIONS: The overall scientific quality and reporting of radiomics studies is insufficient. Scientific improvements need to be made to feature reproducibility, analysis of clinical utility, and open science categories. Reporting of study objectives, blind assessment, sample size, and missing data is deemed to be necessary. KEY POINTS: • The overall scientific quality and reporting of radiomics studies is insufficient. • The RQS was low in demonstration of clinical utility, test-retest analysis, prospective study, and open science. • Room for improvement was shown in TRIPOD in stating study objective in abstract and introduction, blind assessment of outcome, sample size, and missing data categories.
OBJECTIVES: To evaluate radiomics studies according to radiomics quality score (RQS) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) to provide objective measurement of radiomics research. MATERIALS AND METHODS: PubMed and Embase were searched for studies published in high clinical imaging journals until December 2018 using the terms "radiomics" and "radiogenomics." Studies were scored against the items in the RQS and TRIPOD guidelines. Subgroup analyses were performed for journal type (clinical vs. imaging), intended use (diagnostic vs. prognostic), and imaging modality (CT vs. MRI), and articles were compared using Fisher's exact test and Mann-Whitney analysis. RESULTS: Seventy-seven articles were included. The mean RQS score was 26.1% of the maximum (9.4 out of 36). The RQS was low in demonstration of clinical utility (19.5%), test-retest analysis (6.5%), prospective study (3.9%), and open science (3.9%). None of the studies conducted a phantom or cost-effectiveness analysis. The adherence rate for TRIPOD was 57.8% (mean) and was particularly low in reporting title (2.6%), stating study objective in abstract and introduction (7.8% and 16.9%), blind assessment of outcome (14.3%), sample size (6.5%), and missing data (11.7%) categories. Studies in clinical journals scored higher and more frequently adopted external validation than imaging journals. CONCLUSIONS: The overall scientific quality and reporting of radiomics studies is insufficient. Scientific improvements need to be made to feature reproducibility, analysis of clinical utility, and open science categories. Reporting of study objectives, blind assessment, sample size, and missing data is deemed to be necessary. KEY POINTS: • The overall scientific quality and reporting of radiomics studies is insufficient. • The RQS was low in demonstration of clinical utility, test-retest analysis, prospective study, and open science. • Room for improvement was shown in TRIPOD in stating study objective in abstract and introduction, blind assessment of outcome, sample size, and missing data categories.
Authors: Philipp Kickingereder; David Bonekamp; Martha Nowosielski; Annekathrin Kratz; Martin Sill; Sina Burth; Antje Wick; Oliver Eidel; Heinz-Peter Schlemmer; Alexander Radbruch; Jürgen Debus; Christel Herold-Mende; Andreas Unterberg; David Jones; Stefan Pfister; Wolfgang Wick; Andreas von Deimling; Martin Bendszus; David Capper Journal: Radiology Date: 2016-09-16 Impact factor: 11.105
Authors: Hamed Akbari; Spyridon Bakas; Jared M Pisapia; MacLean P Nasrallah; Martin Rozycki; Maria Martinez-Lage; Jennifer J D Morrissette; Nadia Dahmane; Donald M O'Rourke; Christos Davatzikos Journal: Neuro Oncol Date: 2018-07-05 Impact factor: 12.300
Authors: Penny F Whiting; Anne W S Rutjes; Marie E Westwood; Susan Mallett; Jonathan J Deeks; Johannes B Reitsma; Mariska M G Leeflang; Jonathan A C Sterne; Patrick M M Bossuyt Journal: Ann Intern Med Date: 2011-10-18 Impact factor: 25.391
Authors: G I Cassinelli Petersen; J Shatalov; T Verma; W R Brim; H Subramanian; A Brackett; R C Bahar; S Merkaj; T Zeevi; L H Staib; J Cui; A Omuro; R A Bronen; A Malhotra; M S Aboian Journal: AJNR Am J Neuroradiol Date: 2022-03-31 Impact factor: 3.825
Authors: Wei Mu; Ilke Tunali; Jhanelle E Gray; Jin Qi; Matthew B Schabath; Robert J Gillies Journal: Eur J Nucl Med Mol Imaging Date: 2019-12-05 Impact factor: 9.236
Authors: Wei Mu; Evangelia Katsoulakis; Christopher J Whelan; Kenneth L Gage; Matthew B Schabath; Robert J Gillies Journal: Br J Cancer Date: 2021-04-07 Impact factor: 7.640
Authors: Radouane El Ayachy; Nicolas Giraud; Paul Giraud; Catherine Durdux; Philippe Giraud; Anita Burgun; Jean Emmanuel Bibault Journal: Front Oncol Date: 2021-05-05 Impact factor: 6.244
Authors: Usman Mahmood; Aditya Apte; Christopher Kanan; David D B Bates; Giuseppe Corrias; Lorenzo Manneli; Jung Hun Oh; Yusuf Emre Erdi; John Nguyen; Joseph O'Deasy; Amita Shukla-Dave Journal: J Med Imaging (Bellingham) Date: 2021-06-29