Jessica Burggraaff1, Jonas Dorn2, Marcus D'Souza3, Cecily Morrison4, Christian P Kamm5, Peter Kontschieder4, Prejaas Tewarie6, Saskia Steinheimer7, Abigail Sellen4, Frank Dahlke2, Ludwig Kappos8, Bernard Uitdehaag6. 1. Department of Neurology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands. Electronic address: j.burggraaff@vumc.nl. 2. Novartis Pharma AG, Basel, Switzerland. 3. Neurologic Clinic and Policlinic, Departments of Medicine, Clinical Research, Biomedicine, and Biomedical Engineering, University Hospital of Basel, Basel, Switzerland; Neurocure Clinical Research Center, Charité Universitaetsmedizin, Berlin, Germany. 4. Microsoft Research, Cambridge, United Kingdom. 5. Department of Neurology, Inselspital, Bern University Hospital, and University of Bern, Bern, Switzerland; Neurology and Neurorehabilitation Center, Luzerner Kantonsspital Lucerne, Switzerland. 6. Department of Neurology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands. 7. Department of Neurology, Inselspital, Bern University Hospital, and University of Bern, Bern, Switzerland. 8. Neurologic Clinic and Policlinic, Departments of Medicine, Clinical Research, Biomedicine, and Biomedical Engineering, University Hospital of Basel, Basel, Switzerland.
Abstract
OBJECTIVES: To examine the feasibility, reliability, granularity, and convergent validity of a video-based pairwise comparison technique that uses algorithmic support to enable automated rating of motor dysfunction in patients with multiple sclerosis (MS). DESIGN: Feasibility and larger cross-sectional cohort study. SETTING: The outpatient clinic of 2 specialist university medical centers. PARTICIPANTS: Selected sample from a cohort of patients with MS participating in the Assess MS study (N=42). Videos were randomly drawn from each strata of the ataxia severity-degrees as defined in the Expanded Disability Status Scale (EDSS). In Basel: 19 videos of 17 patients (mean age, 43.4±11.6y; 10 women). In Amsterdam: 50 videos of 25 patients (mean age, 50.0±10.0y; 15 women). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: In each center, neurologists (n=13; n=10) viewed pairs of videos of patients performing standardized movements (eg, finger-to-nose test) to assess relative performance. A comparative assessment score was calculated for each video using the TrueSkill algorithm and analyzed for intrarater (test-retest; ratio of agreement) and interrater reliability (intraclass correlation coefficient [ICC] for absolute agreement) and convergent validity (Spearman ρ). Granularity was estimated from the average difference in comparative assessment scores at which 80% of neurologists considered performance to be different. RESULTS: Intrarater reliability was excellent (median ratio of agreement≥0.87). The comparative assessment scores calculated from individual neurologists demonstrated good-excellent ICCs for interrater reliability (0.89; 0.71). The comparative assessment scores correlated (very) highly with their Neurostatus-EDSS equivalent (ρ=0.78, P<.001; ρ=0.91, P<.05), suggesting a more fine-grained rating. CONCLUSIONS: Video-based pairwise comparison of motor dysfunction allows for reliable and fine-grained capturing of clinical judgment about neurologic performance, which can contribute to the development of a consistent quantified metric of motor ability in MS.
OBJECTIVES: To examine the feasibility, reliability, granularity, and convergent validity of a video-based pairwise comparison technique that uses algorithmic support to enable automated rating of motor dysfunction in patients with multiple sclerosis (MS). DESIGN: Feasibility and larger cross-sectional cohort study. SETTING: The outpatient clinic of 2 specialist university medical centers. PARTICIPANTS: Selected sample from a cohort of patients with MS participating in the Assess MS study (N=42). Videos were randomly drawn from each strata of the ataxia severity-degrees as defined in the Expanded Disability Status Scale (EDSS). In Basel: 19 videos of 17 patients (mean age, 43.4±11.6y; 10 women). In Amsterdam: 50 videos of 25 patients (mean age, 50.0±10.0y; 15 women). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: In each center, neurologists (n=13; n=10) viewed pairs of videos of patients performing standardized movements (eg, finger-to-nose test) to assess relative performance. A comparative assessment score was calculated for each video using the TrueSkill algorithm and analyzed for intrarater (test-retest; ratio of agreement) and interrater reliability (intraclass correlation coefficient [ICC] for absolute agreement) and convergent validity (Spearman ρ). Granularity was estimated from the average difference in comparative assessment scores at which 80% of neurologists considered performance to be different. RESULTS: Intrarater reliability was excellent (median ratio of agreement≥0.87). The comparative assessment scores calculated from individual neurologists demonstrated good-excellent ICCs for interrater reliability (0.89; 0.71). The comparative assessment scores correlated (very) highly with their Neurostatus-EDSS equivalent (ρ=0.78, P<.001; ρ=0.91, P<.05), suggesting a more fine-grained rating. CONCLUSIONS: Video-based pairwise comparison of motor dysfunction allows for reliable and fine-grained capturing of clinical judgment about neurologic performance, which can contribute to the development of a consistent quantified metric of motor ability in MS.
Authors: Ka-Hoo Lam; Caspar E P van Munster; Marcus D'Souza; Saskia Steinheimer; Christian P Kamm; Jessica Burggraaff; Matthew Johnson; Yordan Zaykov; Jonas Dorn; Frank Dahlke; Ludwig Kappos; Joep Killestein; Bernard Uitdehaag Journal: Int J MS Care Date: 2022-06-20
Authors: Marcus D'Souza; Caspar E P Van Munster; Jonas F Dorn; Alexis Dorier; Christian P Kamm; Saskia Steinheimer; Frank Dahlke; Bernard M J Uitdehaag; Ludwig Kappos; Matthew Johnson Journal: J Med Internet Res Date: 2020-05-08 Impact factor: 5.428