Jingyu Zhong1, Yangfan Hu2, Liping Si1, Geng Jia2, Yue Xing2, Huan Zhang3, Weiwu Yao4. 1. Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, No. 1111 Xianxia Road, Changning District, Shanghai, 200050, China. 2. Department of Radiology, Shanghai Jiao Tong University Affiliated Sixth People's Hospital, No. 600 Yishan Road, Xuhui District, Shanghai, 200233, China. 3. Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, No. 197 Ruijin 2nd Road, Huangpu District, Shanghai, 200025, China. huanzhangy@163.com. 4. Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, No. 1111 Xianxia Road, Changning District, Shanghai, 200050, China. yaoweiwuhuan@163.com.
Abstract
OBJECTIVES: To assess the methodological quality and risk of bias in radiomics studies investigating diagnosis, therapy response, and survival of patients with osteosarcoma. METHODS: In this systematic review, literatures on radiomics in osteosarcoma were included and assessed for methodological quality through the radiomics quality score (RQS). The risk of bias and concern of application was assessed using the Quality Assessment of Diagnostic Accuracy Studies tool. A meta-analysis of studies focusing on predicting osteosarcoma response to neoadjuvant chemotherapy was performed. RESULTS: Twelve radiomics studies exploring osteosarcoma were identified, and five were included in meta-analysis. The RQS reached an average of 20.4% (6.92 of 36) with good inter-rater agreement (ICC 0.95, 95% CI 0.85-0.99). Four studies validated results with an internal dataset, none of which used external dataset; one study was prospectively designed, and another one shared part of the dataset. The risk of bias and concern of application were mainly related to index test aspect. The meta-analysis showed a diagnostic odds ratio of 43.68 (95%CI 13.5-141.31) for predicting response to neoadjuvant chemotherapy with high heterogeneity and low methodological quality. CONCLUSIONS: The overall scientific quality of included studies is insufficient; however, radiomics remains a promising technology for predicting treatment response, which might guide therapeutic decision-making and related to prognosis. Improvements in study design, validation, and open science needs to be made to demonstrate the generalizability of findings and to achieve clinical applications. Widespread application of RQS, pre-trained RQS scoring procedure, and modification of RQS in response to clinical needs are necessary. KEY POINTS: • Limited radiomics studies were established in osteosarcoma with mean RQS of 20.4%, commonly due to unvalidated results, retrospective study design, and absence of open science. • Meta-analysis of radiomics studies predicting osteosarcoma response to neoadjuvant chemotherapy showed high diagnostic odds ratio 43.68, while high heterogeneity and low methodological quality were the main concerns. • A previously trained data extraction instrument allowed reaching moderate inter-rater agreement in RQS applications, while RQS still needs improvement to become a wide adaptive tool in reviews of radiomics studies, in routine self-check before manuscript submitting and in study design.
OBJECTIVES: To assess the methodological quality and risk of bias in radiomics studies investigating diagnosis, therapy response, and survival of patients with osteosarcoma. METHODS: In this systematic review, literatures on radiomics in osteosarcoma were included and assessed for methodological quality through the radiomics quality score (RQS). The risk of bias and concern of application was assessed using the Quality Assessment of Diagnostic Accuracy Studies tool. A meta-analysis of studies focusing on predicting osteosarcoma response to neoadjuvant chemotherapy was performed. RESULTS: Twelve radiomics studies exploring osteosarcoma were identified, and five were included in meta-analysis. The RQS reached an average of 20.4% (6.92 of 36) with good inter-rater agreement (ICC 0.95, 95% CI 0.85-0.99). Four studies validated results with an internal dataset, none of which used external dataset; one study was prospectively designed, and another one shared part of the dataset. The risk of bias and concern of application were mainly related to index test aspect. The meta-analysis showed a diagnostic odds ratio of 43.68 (95%CI 13.5-141.31) for predicting response to neoadjuvant chemotherapy with high heterogeneity and low methodological quality. CONCLUSIONS: The overall scientific quality of included studies is insufficient; however, radiomics remains a promising technology for predicting treatment response, which might guide therapeutic decision-making and related to prognosis. Improvements in study design, validation, and open science needs to be made to demonstrate the generalizability of findings and to achieve clinical applications. Widespread application of RQS, pre-trained RQS scoring procedure, and modification of RQS in response to clinical needs are necessary. KEY POINTS: • Limited radiomics studies were established in osteosarcoma with mean RQS of 20.4%, commonly due to unvalidated results, retrospective study design, and absence of open science. • Meta-analysis of radiomics studies predicting osteosarcoma response to neoadjuvant chemotherapy showed high diagnostic odds ratio 43.68, while high heterogeneity and low methodological quality were the main concerns. • A previously trained data extraction instrument allowed reaching moderate inter-rater agreement in RQS applications, while RQS still needs improvement to become a wide adaptive tool in reviews of radiomics studies, in routine self-check before manuscript submitting and in study design.
Authors: P G Casali; S Bielack; N Abecassis; H T Aro; S Bauer; R Biagini; S Bonvalot; I Boukovinas; J V M G Bovee; B Brennan; T Brodowicz; J M Broto; L Brugières; A Buonadonna; E De Álava; A P Dei Tos; X G Del Muro; P Dileo; C Dhooge; M Eriksson; F Fagioli; A Fedenko; V Ferraresi; A Ferrari; S Ferrari; A M Frezza; N Gaspar; S Gasperoni; H Gelderblom; T Gil; G Grignani; A Gronchi; R L Haas; B Hassan; S Hecker-Nolting; P Hohenberger; R Issels; H Joensuu; R L Jones; I Judson; P Jutte; S Kaal; L Kager; B Kasper; K Kopeckova; D A Krákorová; R Ladenstein; A Le Cesne; I Lugowska; O Merimsky; M Montemurro; B Morland; M A Pantaleo; R Piana; P Picci; S Piperno-Neumann; A L Pousa; P Reichardt; M H Robinson; P Rutkowski; A A Safwat; P Schöffski; S Sleijfer; S Stacchiotti; S J Strauss; K Sundby Hall; M Unk; F Van Coevorden; W T A van der Graaf; J Whelan; E Wardelmann; O Zaikova; J Y Blay Journal: Ann Oncol Date: 2018-10-01 Impact factor: 32.976
Authors: M P Link; A M Goorin; A W Miser; A A Green; C B Pratt; J B Belasco; J Pritchard; J S Malpas; A R Baker; J A Kirkpatrick Journal: N Engl J Med Date: 1986-06-19 Impact factor: 91.245
Authors: Philippe Lambin; Ralph T H Leijenaar; Timo M Deist; Jurgen Peerlings; Evelyn E C de Jong; Janita van Timmeren; Sebastian Sanduleanu; Ruben T H M Larue; Aniek J G Even; Arthur Jochems; Yvonka van Wijk; Henry Woodruff; Johan van Soest; Tim Lustberg; Erik Roelofs; Wouter van Elmpt; Andre Dekker; Felix M Mottaghy; Joachim E Wildberger; Sean Walsh Journal: Nat Rev Clin Oncol Date: 2017-10-04 Impact factor: 66.675
Authors: Philippe Lambin; Emmanuel Rios-Velazquez; Ralph Leijenaar; Sara Carvalho; Ruud G P M van Stiphout; Patrick Granton; Catharina M L Zegers; Robert Gillies; Ronald Boellard; André Dekker; Hugo J W L Aerts Journal: Eur J Cancer Date: 2012-01-16 Impact factor: 9.162
Authors: G Rosen; B Caparros; A G Huvos; C Kosloff; A Nirenberg; A Cacavio; R C Marcove; J M Lane; B Mehta; C Urban Journal: Cancer Date: 1982-03-15 Impact factor: 6.860