Daniel L Schwartz1,2, Ian Tagge1, Katherine Powers1, Sinyeob Ahn3, Rohit Bakshi4, Peter A Calabresi5, R Todd Constable6, John Grinstead1,3, Roland G Henry7, Govind Nair8, Nico Papinutto7, Daniel Pelletier9, Russell Shinohara10, Jiwon Oh5,11, Daniel S Reich8, Nancy L Sicotte12, William D Rooney1,2. 1. Advanced Imaging Research Center, Oregon Health & Science University, Portland, Oregon, USA. 2. Neurology, Oregon Health & Science University, Portland, Oregon, USA. 3. Siemens Healthineers, Malvern, Pennsylvania, USA. 4. Brigham & Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA. 5. Johns Hopkins University, Baltimore, Maryland, USA. 6. Yale University, New Haven, Connecticut, USA. 7. University of California San Francisco, San Francisco, California, USA. 8. National Institute of Neurological Disease and Stroke, National Institute of Health, Bethesda, Maryland, USA. 9. University of Southern California, Keck School of Medicine, Los Angeles, California, USA. 10. University of Pennsylvania, Philadelphia, Pennsylvania, USA. 11. University of Toronto, Ontario, Canada. 12. Cedars-Sinai Medical Center, Los Angeles, California, USA.
Abstract
BACKGROUND: MRI is the imaging modality of choice for diagnosis and intervention assessment in neurological disease. Its full potential has not been realized due in part to challenges in harmonizing advanced techniques across multiple sites. PURPOSE: To develop a method for the assessment of reliability and repeatability of advanced multisite-multisession neuroimaging studies and specifically to assess the reliability of an advanced MRI protocol, including multiband fMRI and diffusion tensor MRI, in a multisite setting. STUDY TYPE: Prospective. POPULATION: Twice repeated measurement of a single subject with stable relapsing-remitting multiple sclerosis (MS) at seven institutions. FIELD STRENGTH/SEQUENCE: A 3 T MRI protocol included higher spatial resolution anatomical scans, a variable flip-angle longitudinal relaxation rate constant (R1 ≡ 1/T1 ) measurement, quantitative magnetization transfer imaging, diffusion tensor imaging, and a resting-state fMRI (rsFMRI) series. ASSESSMENT: Multiple methods of assessing intrasite repeatability and intersite reliability were evaluated for imaging metrics derived from each sequence. STATISTICAL TESTS: Student's t-test, Pearson's r, and intraclass correlation coefficient (ICC) (2,1) were employed to assess repeatability and reliability. Two new statistical metrics are introduced that frame reliability and repeatability in the respective units of the measurements themselves. RESULTS: Intrasite repeatability was excellent for quantitative R1 , magnetization transfer ratio (MTR), and diffusion-weighted imaging (DWI) based metrics (r > 0.95). rsFMRI metrics were less repeatable (r = 0.8). Intersite reliability was excellent for R1 , MTR, and DWI (ICC >0.9), and moderate for rsFMRI metrics (ICC∼0.4). DATA CONCLUSION: From most reliable to least, using a new reliability metric introduced here, MTR > R1 > DWI > rsFMRI; for repeatability, MTR > DWI > R1 > rsFMRI. A graphical method for at-a-glance assessment of reliability and repeatability, effect sizes, and outlier identification in multisite-multisession neuroimaging studies is introduced. LEVEL OF EVIDENCE: 1 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;50:878-888.
BACKGROUND: MRI is the imaging modality of choice for diagnosis and intervention assessment in neurological disease. Its full potential has not been realized due in part to challenges in harmonizing advanced techniques across multiple sites. PURPOSE: To develop a method for the assessment of reliability and repeatability of advanced multisite-multisession neuroimaging studies and specifically to assess the reliability of an advanced MRI protocol, including multiband fMRI and diffusion tensor MRI, in a multisite setting. STUDY TYPE: Prospective. POPULATION: Twice repeated measurement of a single subject with stable relapsing-remitting multiple sclerosis (MS) at seven institutions. FIELD STRENGTH/SEQUENCE: A 3 T MRI protocol included higher spatial resolution anatomical scans, a variable flip-angle longitudinal relaxation rate constant (R1 ≡ 1/T1 ) measurement, quantitative magnetization transfer imaging, diffusion tensor imaging, and a resting-state fMRI (rsFMRI) series. ASSESSMENT: Multiple methods of assessing intrasite repeatability and intersite reliability were evaluated for imaging metrics derived from each sequence. STATISTICAL TESTS: Student's t-test, Pearson's r, and intraclass correlation coefficient (ICC) (2,1) were employed to assess repeatability and reliability. Two new statistical metrics are introduced that frame reliability and repeatability in the respective units of the measurements themselves. RESULTS: Intrasite repeatability was excellent for quantitative R1 , magnetization transfer ratio (MTR), and diffusion-weighted imaging (DWI) based metrics (r > 0.95). rsFMRI metrics were less repeatable (r = 0.8). Intersite reliability was excellent for R1 , MTR, and DWI (ICC >0.9), and moderate for rsFMRI metrics (ICC∼0.4). DATA CONCLUSION: From most reliable to least, using a new reliability metric introduced here, MTR > R1 > DWI > rsFMRI; for repeatability, MTR > DWI > R1 > rsFMRI. A graphical method for at-a-glance assessment of reliability and repeatability, effect sizes, and outlier identification in multisite-multisession neuroimaging studies is introduced. LEVEL OF EVIDENCE: 1 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;50:878-888.
Authors: Alejandro Caceres; Deanna L Hall; Fernando O Zelaya; Steven C R Williams; Mitul A Mehta Journal: Neuroimage Date: 2008-12-31 Impact factor: 6.556
Authors: William D Rooney; Glyn Johnson; Xin Li; Eric R Cohen; Seong-Gi Kim; Kamil Ugurbil; Charles S Springer Journal: Magn Reson Med Date: 2007-02 Impact factor: 4.668
Authors: Navid Shiee; Pierre-Louis Bazin; Arzu Ozturk; Daniel S Reich; Peter A Calabresi; Dzung L Pham Journal: Neuroimage Date: 2009-09-17 Impact factor: 6.556
Authors: Mark J Lowe; Micheal D Phillips; Joseph T Lurito; David Mattson; Mario Dzemidzic; Vincent P Mathews Journal: Radiology Date: 2002-07 Impact factor: 11.105
Authors: J S Wonderlick; D A Ziegler; P Hosseini-Varnamkhasti; J J Locascio; A Bakkour; A van der Kouwe; C Triantafyllou; S Corkin; B C Dickerson Journal: Neuroimage Date: 2008-11-07 Impact factor: 6.556
Authors: K T M Fernando; D J Tozer; K A Miszkiel; R M Gordon; J K Swanton; C M Dalton; G J Barker; G T Plant; A J Thompson; D H Miller Journal: Brain Date: 2005-10-11 Impact factor: 13.501
Authors: Pietro Maggi; Martina Absinta; Pascal Sati; Gaetano Perrotta; Luca Massacesi; Bernard Dachy; Caroline Pot; Reto Meuli; Daniel S Reich; Massimo Filippi; Renaud Du Pasquier; Marie Théaudin Journal: Mult Scler Date: 2019-09-19 Impact factor: 6.312
Authors: John Virostko; Richard C Craddock; Jonathan M Williams; Taylor M Triolo; Melissa A Hilmes; Hakmook Kang; Liping Du; Jordan J Wright; Mara Kinney; Jeffrey H Maki; Milica Medved; Michaela Waibel; Thomas W H Kay; Helen E Thomas; Siri Atma W Greeley; Andrea K Steck; Daniel J Moore; Alvin C Powers Journal: PLoS One Date: 2021-08-24 Impact factor: 3.240
Authors: Mahbaneh Eshaghzadeh Torbati; Davneet S Minhas; Ghasan Ahmad; Erin E O'Connor; John Muschelli; Charles M Laymon; Zixi Yang; Ann D Cohen; Howard J Aizenstein; William E Klunk; Bradley T Christian; Seong Jae Hwang; Ciprian M Crainiceanu; Dana L Tudorascu Journal: Neuroimage Date: 2021-11-01 Impact factor: 6.556
Authors: Sirisha Tadimalla; Daniel J Wilson; David Shelley; Gavin Bainbridge; Margaret Saysell; Iosif A Mendichovszky; Martin J Graves; J Ashley Guthrie; John C Waterton; Geoffrey J M Parker; Steven P Sourbron Journal: J Magn Reson Imaging Date: 2022-02-27 Impact factor: 5.119
Authors: Christopher Kurz; Giulia Buizza; Guillaume Landry; Florian Kamp; Moritz Rabe; Chiara Paganelli; Guido Baroni; Michael Reiner; Paul J Keall; Cornelis A T van den Berg; Marco Riboldi Journal: Radiat Oncol Date: 2020-05-05 Impact factor: 3.481