Nancy A Obuchowski1. 1. Quantitative Health Sciences/JJN3, Cleveland Clinic Foundation, 9500 Euclid Ave, Cleveland, OH 44195-5196. Electronic address: obuchon@ccf.org.
Abstract
RATIONALE AND OBJECTIVES: Quantitative imaging biomarkers (QIBs) are becoming increasingly adopted into clinical practice to monitor changes in patients' conditions. The repeatability coefficient (RC) is the clinical cut-point used to discern between changes in a biomarker's measurements due to measurement error and changes that exceed measurement error, thus indicating real change in the patient. Imaging biomarkers have characteristics that make them difficult for estimating the repeatability coefficient, including nonconstant error, non-Gaussian distributions, and measurement error that must be estimated from small studies. METHODS: We conducted a Monte Carlo simulation study to investigate how well three statistical methods for estimating the repeatability coefficient perform under five settings common for QIBs. RESULTS: When the measurement error is constant and replicates are normally distributed, all of the statistical methods perform well. When the measurement error is proportional to the true value, approaches that use the log transformation or coefficient of variation perform similarly. For other common settings, none of the methods for estimating the repeatability coefficient perform adequately. CONCLUSION: Many of the common approaches to estimating the repeatability coefficient perform well for only limited scenarios. The optimal approach depends strongly on the pattern of the within-subject variability; thus, a precision profile is critical in evaluating the technical performance of QIBs. Asymmetric bounds for detecting regression vs progression can be implemented and should be used when clinically appropriate.
RATIONALE AND OBJECTIVES: Quantitative imaging biomarkers (QIBs) are becoming increasingly adopted into clinical practice to monitor changes in patients' conditions. The repeatability coefficient (RC) is the clinical cut-point used to discern between changes in a biomarker's measurements due to measurement error and changes that exceed measurement error, thus indicating real change in the patient. Imaging biomarkers have characteristics that make them difficult for estimating the repeatability coefficient, including nonconstant error, non-Gaussian distributions, and measurement error that must be estimated from small studies. METHODS: We conducted a Monte Carlo simulation study to investigate how well three statistical methods for estimating the repeatability coefficient perform under five settings common for QIBs. RESULTS: When the measurement error is constant and replicates are normally distributed, all of the statistical methods perform well. When the measurement error is proportional to the true value, approaches that use the log transformation or coefficient of variation perform similarly. For other common settings, none of the methods for estimating the repeatability coefficient perform adequately. CONCLUSION: Many of the common approaches to estimating the repeatability coefficient perform well for only limited scenarios. The optimal approach depends strongly on the pattern of the within-subject variability; thus, a precision profile is critical in evaluating the technical performance of QIBs. Asymmetric bounds for detecting regression vs progression can be implemented and should be used when clinically appropriate.
Authors: Daniel C Sullivan; Nancy A Obuchowski; Larry G Kessler; David L Raunig; Constantine Gatsonis; Erich P Huang; Marina Kondratovich; Lisa M McShane; Anthony P Reeves; Daniel P Barboriak; Alexander R Guimaraes; Richard L Wahl Journal: Radiology Date: 2015-08-12 Impact factor: 11.105
Authors: Wolfgang A Weber; Constantine A Gatsonis; P David Mozley; Lucy G Hanna; Anthony F Shields; Denise R Aberle; Ramaswamy Govindan; Drew A Torigian; Joel S Karp; Jian Q Michael Yu; Rathan M Subramaniam; Robert A Halvorsen; Barry A Siegel Journal: J Nucl Med Date: 2015-04-23 Impact factor: 10.057
Authors: Jessica M Winfield; Nina Tunariu; Mihaela Rata; Keiko Miyazaki; Neil P Jerome; Michael Germuska; Matthew D Blackledge; David J Collins; Johann S de Bono; Timothy A Yap; Nandita M deSouza; Simon J Doran; Dow-Mu Koh; Martin O Leach; Christina Messiou; Matthew R Orton Journal: Radiology Date: 2017-03-16 Impact factor: 11.105
Authors: David L Raunig; Lisa M McShane; Gene Pennello; Constantine Gatsonis; Paul L Carson; James T Voyvodic; Richard L Wahl; Brenda F Kurland; Adam J Schwarz; Mithat Gönen; Gudrun Zahlmann; Marina V Kondratovich; Kevin O'Donnell; Nicholas Petrick; Patricia E Cole; Brian Garra; Daniel C Sullivan Journal: Stat Methods Med Res Date: 2014-06-11 Impact factor: 3.021
Authors: Nancy A Obuchowski; Anthony P Reeves; Erich P Huang; Xiao-Feng Wang; Andrew J Buckler; Hyun J Grace Kim; Huiman X Barnhart; Edward F Jackson; Maryellen L Giger; Gene Pennello; Alicia Y Toledano; Jayashree Kalpathy-Cramer; Tatiyana V Apanasovich; Paul E Kinahan; Kyle J Myers; Dmitry B Goldgof; Daniel P Barboriak; Robert J Gillies; Lawrence H Schwartz; Daniel C Sullivan Journal: Stat Methods Med Res Date: 2014-06-11 Impact factor: 3.021
Authors: Nancy A Obuchowski; Huiman X Barnhart; Andrew J Buckler; Gene Pennello; Xiao-Feng Wang; Jayashree Kalpathy-Cramer; Hyun J Grace Kim; Anthony P Reeves Journal: Stat Methods Med Res Date: 2014-06-11 Impact factor: 3.021
Authors: Robert Seifert; Patrick Sandach; David Kersting; Wolfgang P Fendler; Boris Hadaschik; Ken Herrmann; John J Sunderland; Janet H Pollard Journal: J Nucl Med Date: 2021-08-26 Impact factor: 11.082
Authors: Charles R Hatt; Andrea S Oh; Nancy A Obuchowski; Jean-Paul Charbonnier; David A Lynch; Stephen M Humphries Journal: Radiol Cardiothorac Imaging Date: 2021-04-22
Authors: Samantha J Paulsen; Trevor M Mitcham; Charlene S Pan; James Long; Bagrat Grigoryan; Daniel W Sazer; Collin J Harlan; Kevin D Janson; Mark D Pagel; Jordan S Miller; Richard R Bouchard Journal: PLoS One Date: 2021-12-09 Impact factor: 3.240
Authors: Ernst S Kooreman; Petra J van Houdt; Rick Keesman; Vivian W J van Pelt; Marlies E Nowee; Floris Pos; Karolina Sikorska; Andreas Wetscherek; Arndt-Christian Müller; Daniela Thorwarth; Alison C Tree; Uulke A van der Heide Journal: Front Oncol Date: 2021-08-13 Impact factor: 6.244