Susan V Duff1, Jiaxiu He2, Monica A Nelsen3, Christianne J Lane3, Veronica T Rowe4, Steve L Wolf5, Alexander W Dromerick6, Carolee J Winstein7. 1. Thomas Jefferson University, Philadelphia, PA, USA. 2. Northwestern University, Chicago, IL, USA. 3. University of Southern California, Los Angeles, CA, USA. 4. University of Central Arkansas, Conway, AR, USA Texas Woman's University, Dallas, TX, USA. 5. Emory School of Medicine, Atlanta, GA, USA. 6. Georgetown University and MedStar National Rehabilitation Hospital, Washington, DC, USA. 7. University of Southern California, Los Angeles, CA, USA winstein@usc.edu.
Abstract
BACKGROUND: One important objective for clinical trialists in rehabilitation is determining efficacy of interventions to enhance motor behavior. In part, limitation in the precision of measurement presents a challenge. The few valid, low-cost observational tools available to assess motor behavior cannot escape the variability inherent in test administration and scoring. This is especially true when there are multiple evaluators and raters, as in the case of multisite randomized controlled trials (RCTs). One way to enhance reliability and reduce variability is to implement rigorous quality control (QC) procedures. OBJECTIVE: This article describes a systematic QC process used to refine the administration and scoring procedures for the Wolf Motor Function Test (WMFT)-Functional Ability Scale (FAS). METHODS: The QC process, a systematic focus-group collaboration, was developed and used for a phase III RCT, which enlisted multiple evaluators and an experienced WMFT-FAS rater panel. RESULTS: After 3 staged refinements to the administration and scoring instructions, we achieved a sufficiently high interrater reliability (weighted κ = 0.8). CONCLUSIONS AND IMPLICATIONS: A systematic focus-group process was shown to be an effective method to improve reliability of observational assessment tools for motor behavior in neurorehabilitation. A reduction in noise-related variability in performance assessments will increase power and potentially lower the number needed to treat. Improved precision of measurement can lead to more cost-effective and efficient clinical trials. Finally, we suggest that improved precision in measures of motor behavior may provide more insight into recovery mechanisms than a single measure of movement time alone.
BACKGROUND: One important objective for clinical trialists in rehabilitation is determining efficacy of interventions to enhance motor behavior. In part, limitation in the precision of measurement presents a challenge. The few valid, low-cost observational tools available to assess motor behavior cannot escape the variability inherent in test administration and scoring. This is especially true when there are multiple evaluators and raters, as in the case of multisite randomized controlled trials (RCTs). One way to enhance reliability and reduce variability is to implement rigorous quality control (QC) procedures. OBJECTIVE: This article describes a systematic QC process used to refine the administration and scoring procedures for the Wolf Motor Function Test (WMFT)-Functional Ability Scale (FAS). METHODS: The QC process, a systematic focus-group collaboration, was developed and used for a phase III RCT, which enlisted multiple evaluators and an experienced WMFT-FAS rater panel. RESULTS: After 3 staged refinements to the administration and scoring instructions, we achieved a sufficiently high interrater reliability (weighted κ = 0.8). CONCLUSIONS AND IMPLICATIONS: A systematic focus-group process was shown to be an effective method to improve reliability of observational assessment tools for motor behavior in neurorehabilitation. A reduction in noise-related variability in performance assessments will increase power and potentially lower the number needed to treat. Improved precision of measurement can lead to more cost-effective and efficient clinical trials. Finally, we suggest that improved precision in measures of motor behavior may provide more insight into recovery mechanisms than a single measure of movement time alone.
Authors: B Dobkin; H Barbeau; D Deforge; J Ditunno; R Elashoff; D Apple; M Basso; A Behrman; S Harkema; M Saulino; M Scott Journal: Neurorehabil Neural Repair Date: 2007 Jan-Feb Impact factor: 3.919
Authors: Margaret Duff; Yinpeng Chen; Long Cheng; Sheng-Min Liu; Paul Blake; Steven L Wolf; Thanassis Rikakis Journal: Neurorehabil Neural Repair Date: 2012-12-03 Impact factor: 3.919
Authors: Tomoko Kitago; Johnny Liang; Vincent S Huang; Sheila Hayes; Phyllis Simon; Laura Tenteromano; Ronald M Lazar; Randolph S Marshall; Pietro Mazzoni; Laura Lennihan; John W Krakauer Journal: Neurorehabil Neural Repair Date: 2012-07-13 Impact factor: 3.919
Authors: Steven C Cramer; Steven L Wolf; Harold P Adams; Daofen Chen; Alexander W Dromerick; Kari Dunning; Caitlyn Ellerbe; Andrew Grande; Scott Janis; Maarten G Lansberg; Ronald M Lazar; Yuko Y Palesch; Lorie Richards; Elliot Roth; Sean I Savitz; Lawrence R Wechsler; Max Wintermark; Joseph P Broderick Journal: Stroke Date: 2017-02-07 Impact factor: 7.914
Authors: Sunghoon I Lee; Catherine P Adans-Dester; Anne T OBrien; Gloria P Vergara-Diaz; Randie Black-Schaffer; Ross Zafonte; Jennifer G Dy; Paolo Bonato Journal: IEEE Trans Biomed Eng Date: 2021-05-21 Impact factor: 4.538