BACKGROUND AND PURPOSE: Use of the modified Rankin scale (mRS) in multicenter trials may be limited by interobserver variability. We assessed the effect of this on trial power and developed a novel group adjudication approach. METHODS: We generated power and sample size estimates from simulated trials modeled with varying mRS reliability. We conducted a virtual acute stroke trial across 14 UK sites to develop a group adjudication approach. Traditional mRS interviews, performed at local sites, were digitally recorded and scored by adjudication committee. We assessed the effect of translation by comparing scores in translated mRS interviews, originally conducted in English and Mandarin. Agreement was measured using κ and weighted κ (κw) statistics and intraclass correlation coefficient. RESULTS: Statistical simulations suggest that improving mRS reliability from κ=0.25 to κ=0.5 or 0.7 may allow reductions in sample size of n=386 or 490 in a typical n=2000 study. Our virtual acute stroke trial included 370 participants and 563 mRS video assessments. We adjudicated mRS in 538 of 563 (96%) study visits. At 30 and 90 days, 161 of 280 (57.5%) and 131 of 258 (50.8%) clips showed interobserver disagreement. Agreement within the adjudication committee was good (30-day κw=0.85 [95% confidence interval, 0.81-0.86]; 90-day κw=0.86 [95% confidence interval, 0.82-0.88]) without significant or systematic bias in mRS scoring compared with the local mRS. Interobserver reliability of translated mRS assessments was similar to native language clips (native [n=69] κw=0.91 [95% confidence interval, 0.94-0.99]; translated [n=89] κw=0.90 [95% confidence interval, 0.83-0.96]). CONCLUSIONS: Achievable improvements in interobserver reliability may substantially reduce study sample size, with associated financial benefits. Central adjudication of mRS assessments is feasible (including across international centers), valid and reliable despite the challenges of mRS assessment in large clinical trials.
BACKGROUND AND PURPOSE: Use of the modified Rankin scale (mRS) in multicenter trials may be limited by interobserver variability. We assessed the effect of this on trial power and developed a novel group adjudication approach. METHODS: We generated power and sample size estimates from simulated trials modeled with varying mRS reliability. We conducted a virtual acute stroke trial across 14 UK sites to develop a group adjudication approach. Traditional mRS interviews, performed at local sites, were digitally recorded and scored by adjudication committee. We assessed the effect of translation by comparing scores in translated mRS interviews, originally conducted in English and Mandarin. Agreement was measured using κ and weighted κ (κw) statistics and intraclass correlation coefficient. RESULTS: Statistical simulations suggest that improving mRS reliability from κ=0.25 to κ=0.5 or 0.7 may allow reductions in sample size of n=386 or 490 in a typical n=2000 study. Our virtual acute stroke trial included 370 participants and 563 mRS video assessments. We adjudicated mRS in 538 of 563 (96%) study visits. At 30 and 90 days, 161 of 280 (57.5%) and 131 of 258 (50.8%) clips showed interobserver disagreement. Agreement within the adjudication committee was good (30-day κw=0.85 [95% confidence interval, 0.81-0.86]; 90-day κw=0.86 [95% confidence interval, 0.82-0.88]) without significant or systematic bias in mRS scoring compared with the local mRS. Interobserver reliability of translated mRS assessments was similar to native language clips (native [n=69] κw=0.91 [95% confidence interval, 0.94-0.99]; translated [n=89] κw=0.90 [95% confidence interval, 0.83-0.96]). CONCLUSIONS: Achievable improvements in interobserver reliability may substantially reduce study sample size, with associated financial benefits. Central adjudication of mRS assessments is feasible (including across international centers), valid and reliable despite the challenges of mRS assessment in large clinical trials.
Authors: Daniel F Hanley; Richard E Thompson; Michael Rosenblum; Gayane Yenokyan; Karen Lane; Nichol McBee; Steven W Mayo; Amanda J Bistran-Hall; Dheeraj Gandhi; W Andrew Mould; Natalie Ullman; Hasan Ali; J Ricardo Carhuapoma; Carlos S Kase; Kennedy R Lees; Jesse Dawson; Alastair Wilson; Joshua F Betz; Elizabeth A Sugar; Yi Hao; Radhika Avadhani; Jean-Louis Caron; Mark R Harrigan; Andrew P Carlson; Diederik Bulters; David LeDoux; Judy Huang; Cully Cobb; Gaurav Gupta; Ryan Kitagawa; Michael R Chicoine; Hiren Patel; Robert Dodd; Paul J Camarata; Stacey Wolfe; Agnieszka Stadnik; P Lynn Money; Patrick Mitchell; Rosario Sarabia; Sagi Harnof; Pal Barzo; Andreas Unterberg; Jeanne S Teitelbaum; Weimin Wang; Craig S Anderson; A David Mendelow; Barbara Gregson; Scott Janis; Paul Vespa; Wendy Ziai; Mario Zuccarello; Issam A Awad Journal: Lancet Date: 2019-02-07 Impact factor: 79.321
Authors: Azmil H Abdul-Rahim; David Alexander Dickie; Johann R Selvarajah; Kennedy R Lees; Terence J Quinn Journal: Trials Date: 2019-02-08 Impact factor: 2.279
Authors: H Bart van der Worp; Malcolm R Macleod; Philip Mw Bath; Raj Bathula; Hanne Christensen; Bridget Colam; Charlotte Cordonnier; Jacques Demotes-Mainard; Isabelle Durand-Zaleski; Christian Gluud; Janus Christian Jakobsen; Bernd Kallmünzer; Rainer Kollmar; Derk W Krieger; Kennedy R Lees; Dominik Michalski; Carlos Molina; Joan Montaner; Risto O Roine; Jesper Petersson; Richard Perry; Nikola Sprigg; Dimitre Staykov; Istvan Szabo; Geert Vanhooren; Joanna M Wardlaw; Per Winkel; Stefan Schwab Journal: Eur Stroke J Date: 2019-04-20
Authors: Peter J Mitchell; Bernard Yan; Leonid Churilov; Richard J Dowling; Steven Bush; Thang Nguyen; Bruce C V Campbell; Geoffrey A Donnan; Zhongrong Miao; Stephen M Davis Journal: J Stroke Date: 2022-01-31 Impact factor: 6.967
Authors: Aisling McFall; Tuuli M Hietamies; Ashton Bernard; Margaux Aimable; Stuart M Allan; Philip M Bath; Gaia Brezzo; Roxana O Carare; Hilary V Carswell; Andrew N Clarkson; Gillian Currie; Tracy D Farr; Jill H Fowler; Mark Good; Atticus H Hainsworth; Catherine Hall; Karen Horsburgh; Rajesh Kalaria; Patrick Kehoe; Catherine Lawrence; Malcolm Macleod; Barry W McColl; Alison McNeilly; Alyson A Miller; Scott Miners; Vincent Mok; Michael O'Sullivan; Bettina Platt; Emily S Sena; Matthew Sharp; Patrick Strangward; Stefan Szymkowiak; Rhian M Touyz; Rebecca C Trueman; Claire White; Chris McCabe; Lorraine M Work; Terence J Quinn Journal: J Cereb Blood Flow Metab Date: 2020-03-09 Impact factor: 6.200
Authors: Jeroen C de Jonge; Lisa J Woodhouse; Hendrik Reinink; H Bart van der Worp; Philip M Bath Journal: Trials Date: 2020-10-26 Impact factor: 2.279