D M Heiferman1, N C Pecoraro2, A W Wozniak2, K C Ebersole3, L M Jimenez3, M R Reynolds2, A J Ringer3, J C Serrone2. 1. From the Department of Neurological Surgery (D.M.H., N.C.P., M.R.R., J.C.S.), Clinical Research Office (A.W.W.), Loyola University Stritch School of Medicine and Loyola University Medical Center, Maywood, Illinois daniel@heiferman.com. 2. From the Department of Neurological Surgery (D.M.H., N.C.P., M.R.R., J.C.S.), Clinical Research Office (A.W.W.), Loyola University Stritch School of Medicine and Loyola University Medical Center, Maywood, Illinois. 3. Department of Neurological Surgery (K.C.E.), University of Kansas Medical Center, Kansas City, Kansas; and Mayfield Brain & Spine (L.M.J., A.J.R.), Cincinnati, Ohio.
Abstract
BACKGROUND AND PURPOSE: The modified TICI score is the benchmark for quantifying reperfusion after mechanical thrombectomy. There has been limited investigation into the reliability of this score. We aim to identify intra-rater and inter-rater reliability of the mTICI score among endovascular neurosurgeons. MATERIALS AND METHODS: Four independent endovascular neurosurgeons (raters) reviewed angiograms of 67 patients at 2 time points. κ statistics assessed inter- and intrarater reliability and compared raters'-versus-proceduralists' scores. Reliability was also assessed for occlusion location and by dichotomizing modified TICI scores (0-2a versus 2b-3). RESULTS: Interrater reliability was moderate-to-substantial, weighted κ = 0.417-0.703, overall κ = 0.374 (P < .001). The dichotomized modified TICI score had moderate-to-substantial interrater agreement, κ statistics = 0.468-0.715, overall κ = 0.582 (P < .001). Intrarater reliability was moderate-to-almost perfect, weighted κ = 0.594-0.81. The dichotomized modified TICI score had substantial-to-almost perfect reliability, κ = 0.632-0.82. Proceduralists had fair-to-moderate agreement with raters, weighted κ = 0.348-0.574, and the dichotomized modified TICI score had fair-to-moderate agreement, κ = 0.365-0.544. When proceduralists and raters disagreed, proceduralists' scores were higher in 79.6% of cases. M1 followed by ICA occlusions had the highest agreement. CONCLUSIONS: The modified TICI score is a practical metric for assessing reperfusion after mechanical thrombectomy, though not without limitations. Agreement improved when scores were dichotomized around the clinically relevant threshold of successful revascularization. Interrater reliability improved with time, suggesting that formal training of interventionalists may improve reporting reliability. Agreement of the modified TICI scale is best with M1 and ICA occlusion and becomes less reliable with more distal or posterior circulation occlusions. These findings should be considered when developing research trials.
BACKGROUND AND PURPOSE: The modified TICI score is the benchmark for quantifying reperfusion after mechanical thrombectomy. There has been limited investigation into the reliability of this score. We aim to identify intra-rater and inter-rater reliability of the mTICI score among endovascular neurosurgeons. MATERIALS AND METHODS: Four independent endovascular neurosurgeons (raters) reviewed angiograms of 67 patients at 2 time points. κ statistics assessed inter- and intrarater reliability and compared raters'-versus-proceduralists' scores. Reliability was also assessed for occlusion location and by dichotomizing modified TICI scores (0-2a versus 2b-3). RESULTS: Interrater reliability was moderate-to-substantial, weighted κ = 0.417-0.703, overall κ = 0.374 (P < .001). The dichotomized modified TICI score had moderate-to-substantial interrater agreement, κ statistics = 0.468-0.715, overall κ = 0.582 (P < .001). Intrarater reliability was moderate-to-almost perfect, weighted κ = 0.594-0.81. The dichotomized modified TICI score had substantial-to-almost perfect reliability, κ = 0.632-0.82. Proceduralists had fair-to-moderate agreement with raters, weighted κ = 0.348-0.574, and the dichotomized modified TICI score had fair-to-moderate agreement, κ = 0.365-0.544. When proceduralists and raters disagreed, proceduralists' scores were higher in 79.6% of cases. M1 followed by ICA occlusions had the highest agreement. CONCLUSIONS: The modified TICI score is a practical metric for assessing reperfusion after mechanical thrombectomy, though not without limitations. Agreement improved when scores were dichotomized around the clinically relevant threshold of successful revascularization. Interrater reliability improved with time, suggesting that formal training of interventionalists may improve reporting reliability. Agreement of the modified TICI scale is best with M1 and ICA occlusion and becomes less reliable with more distal or posterior circulation occlusions. These findings should be considered when developing research trials.
Authors: Sang Hyun Suh; Harry J Cloft; Jennifer E Fugate; Alejandro A Rabinstein; David S Liebeskind; David F Kallmes Journal: Stroke Date: 2013-02-14 Impact factor: 7.914
Authors: Raul G Nogueira; Helmi L Lutsep; Rishi Gupta; Tudor G Jovin; Gregory W Albers; Gary A Walker; David S Liebeskind; Wade S Smith Journal: Lancet Date: 2012-08-26 Impact factor: 79.321
Authors: Jeffrey L Saver; Mayank Goyal; Alain Bonafe; Hans-Christoph Diener; Elad I Levy; Vitor M Pereira; Gregory W Albers; Christophe Cognard; David J Cohen; Werner Hacke; Olav Jansen; Tudor G Jovin; Heinrich P Mattle; Raul G Nogueira; Adnan H Siddiqui; Dileep R Yavagal; Blaise W Baxter; Thomas G Devlin; Demetrius K Lopes; Vivek K Reddy; Richard du Mesnil de Rochemont; Oliver C Singer; Reza Jahan Journal: N Engl J Med Date: 2015-04-17 Impact factor: 91.245
Authors: Pooja Khatri; Joddi Neff; Joseph P Broderick; Jane C Khoury; Janice Carrozzella; Thomas Tomsick Journal: Stroke Date: 2005-10-13 Impact factor: 7.914
Authors: Bruce C V Campbell; Peter J Mitchell; Timothy J Kleinig; Helen M Dewey; Leonid Churilov; Nawaf Yassi; Bernard Yan; Richard J Dowling; Mark W Parsons; Thomas J Oxley; Teddy Y Wu; Mark Brooks; Marion A Simpson; Ferdinand Miteff; Christopher R Levi; Martin Krause; Timothy J Harrington; Kenneth C Faulder; Brendan S Steinfort; Miriam Priglinger; Timothy Ang; Rebecca Scroop; P Alan Barber; Ben McGuinness; Tissa Wijeratne; Thanh G Phan; Winston Chong; Ronil V Chandra; Christopher F Bladin; Monica Badve; Henry Rice; Laetitia de Villiers; Henry Ma; Patricia M Desmond; Geoffrey A Donnan; Stephen M Davis Journal: N Engl J Med Date: 2015-02-11 Impact factor: 91.245
Authors: Elizabeth A Noser; Hashem M Shaltoni; Christiana E Hall; Andrei V Alexandrov; Zsolt Garami; Edwin D Cacayorin; Joon K Song; James C Grotta; Morgan S Campbell Journal: Stroke Date: 2004-12-29 Impact factor: 7.914
Authors: Mayank Goyal; Andrew M Demchuk; Bijoy K Menon; Muneer Eesa; Jeremy L Rempel; John Thornton; Daniel Roy; Tudor G Jovin; Robert A Willinsky; Biggya L Sapkota; Dar Dowlatshahi; Donald F Frei; Noreen R Kamal; Walter J Montanera; Alexandre Y Poppe; Karla J Ryckborst; Frank L Silver; Ashfaq Shuaib; Donatella Tampieri; David Williams; Oh Young Bang; Blaise W Baxter; Paul A Burns; Hana Choe; Ji-Hoe Heo; Christine A Holmstedt; Brian Jankowitz; Michael Kelly; Guillermo Linares; Jennifer L Mandzia; Jai Shankar; Sung-Il Sohn; Richard H Swartz; Philip A Barber; Shelagh B Coutts; Eric E Smith; William F Morrish; Alain Weill; Suresh Subramaniam; Alim P Mitha; John H Wong; Mark W Lowerison; Tolulope T Sajobi; Michael D Hill Journal: N Engl J Med Date: 2015-02-11 Impact factor: 91.245
Authors: Randall T Higashida; Anthony J Furlan; Heidi Roberts; Thomas Tomsick; Buddy Connors; John Barr; William Dillon; Steven Warach; Joseph Broderick; Barbara Tilley; David Sacks Journal: Stroke Date: 2003-07-17 Impact factor: 7.914
Authors: Robert W Regenhardt; Amine Awad; Andrew W Kraft; Joseph A Rosenthal; Adam A Dmytriw; Justin E Vranic; Anna K Bonkhoff; Martin Bretzner; Mark R Etherton; Joshua A Hirsch; James D Rabinov; Aneesh B Singhal; Natalia S Rost; Christopher J Stapleton; Thabele M Leslie-Mazwi; Aman B Patel Journal: Stroke Vasc Interv Neurol Date: 2022-05-20