Lennard Wolff1, Jiahang Su2, Derek Van Loon2, Adriaan van Es3, Pieter Jan van Doormaal2, Charles Majoie4, Wim van Zwam5, Diederik Dippel6, Aad van der Lugt2, Theo van Walsum2. 1. Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Center, Doctor Molewaterplein 40, 3015 GD, Rotterdam, The Netherlands. l.wolff.1@erasmusmc.nl. 2. Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Center, Doctor Molewaterplein 40, 3015 GD, Rotterdam, The Netherlands. 3. Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands. 4. Department of Radiology & Nuclear Medicine, Amsterdam University Medical Centers, location AMC, Amsterdam, the Netherlands. 5. Department of Radiology and Nuclear Medicine, Maastricht University Medical Center, Maastricht, The Netherlands. 6. Department of Neurology, Erasmus MC University Medical Center, Rotterdam, The Netherlands.
Abstract
PURPOSE: Outcome of endovascular treatment in acute ischemic stroke patients is depending on the collateral circulation maintaining blood flow to the ischemic territory. We evaluated the inter-rater reliability and accuracy of raters and an automated algorithm for assessing the collateral score (CS, range: 0-3) in acute ischemic stroke patients. METHODS: Baseline CTA scans with an intracranial anterior occlusion from the MR CLEAN study (n=500) were used. For each core lab CS, ten CTA scans with sufficient quality were randomly selected. After a training session in collateral scoring, all selected CTA scans were individually evaluated for a visual CS by three groups: 7 radiologists, 13 junior and 9 senior radiology residents. Two additional radiologists scored CS to be used as reference, with a third providing a CS to produce a 2 out of 3 consensus CS in case of disagreement. An automated algorithm was also used to compute CS. Inter-rater agreement was reported with intraclass correlation coefficient (ICC). Accuracy of visual and automated CS were calculated. RESULTS: 39 CTA scans were assessed (1 corrupt CTA-scan excluded). All groups showed a moderate ICC (0.689-0.780) in comparison to the reference standard. Overall human accuracy was 65± 7% and increased to 88± 5% for dichotomized CS (0-1, 2-3). Automated CS accuracy was 62%, and 90% for dichotomized CS. No significant difference in accuracy was found between groups with different levels of expertise. CONCLUSION: After training, inter-rater reliability in collateral scoring was not influenced by experience. Automated CS performs similar to residents and radiologists in determining a collateral score.
PURPOSE: Outcome of endovascular treatment in acute ischemic stroke patients is depending on the collateral circulation maintaining blood flow to the ischemic territory. We evaluated the inter-rater reliability and accuracy of raters and an automated algorithm for assessing the collateral score (CS, range: 0-3) in acute ischemic stroke patients. METHODS: Baseline CTA scans with an intracranial anterior occlusion from the MR CLEAN study (n=500) were used. For each core lab CS, ten CTA scans with sufficient quality were randomly selected. After a training session in collateral scoring, all selected CTA scans were individually evaluated for a visual CS by three groups: 7 radiologists, 13 junior and 9 senior radiology residents. Two additional radiologists scored CS to be used as reference, with a third providing a CS to produce a 2 out of 3 consensus CS in case of disagreement. An automated algorithm was also used to compute CS. Inter-rater agreement was reported with intraclass correlation coefficient (ICC). Accuracy of visual and automated CS were calculated. RESULTS: 39 CTA scans were assessed (1 corrupt CTA-scan excluded). All groups showed a moderate ICC (0.689-0.780) in comparison to the reference standard. Overall human accuracy was 65± 7% and increased to 88± 5% for dichotomized CS (0-1, 2-3). Automated CS accuracy was 62%, and 90% for dichotomized CS. No significant difference in accuracy was found between groups with different levels of expertise. CONCLUSION: After training, inter-rater reliability in collateral scoring was not influenced by experience. Automated CS performs similar to residents and radiologists in determining a collateral score.
Authors: Ralph L Sacco; Scott E Kasner; Joseph P Broderick; Louis R Caplan; J J Buddy Connors; Antonio Culebras; Mitchell S V Elkind; Mary G George; Allen D Hamdan; Randall T Higashida; Brian L Hoh; L Scott Janis; Carlos S Kase; Dawn O Kleindorfer; Jin-Moo Lee; Michael E Moseley; Eric D Peterson; Tanya N Turan; Amy L Valderrama; Harry V Vinters Journal: Stroke Date: 2013-05-07 Impact factor: 7.914
Authors: Olvert A Berkhemer; Ivo G H Jansen; Debbie Beumer; Puck S S Fransen; Lucie A van den Berg; Albert J Yoo; Hester F Lingsma; Marieke E S Sprengers; Sjoerd F M Jenniskens; Geert J Lycklama À Nijeholt; Marianne A A van Walderveen; René van den Berg; Joseph C J Bot; Ludo F M Beenen; Anna M M Boers; Cornelis H Slump; Yvo B W E M Roos; Robert J van Oostenbrugge; Diederik W J Dippel; Aad van der Lugt; Wim H van Zwam; Henk A Marquering; Charles B L M Majoie Journal: Stroke Date: 2016-01-28 Impact factor: 7.914
Authors: Jessica C Tan; William P Dillon; Songling Liu; Felix Adler; Wade S Smith; Max Wintermark Journal: Ann Neurol Date: 2007-06 Impact factor: 10.422
Authors: Olvert A Berkhemer; Puck S S Fransen; Debbie Beumer; Lucie A van den Berg; Hester F Lingsma; Albert J Yoo; Wouter J Schonewille; Jan Albert Vos; Paul J Nederkoorn; Marieke J H Wermer; Marianne A A van Walderveen; Julie Staals; Jeannette Hofmeijer; Jacques A van Oostayen; Geert J Lycklama à Nijeholt; Jelis Boiten; Patrick A Brouwer; Bart J Emmer; Sebastiaan F de Bruijn; Lukas C van Dijk; L Jaap Kappelle; Rob H Lo; Ewoud J van Dijk; Joost de Vries; Paul L M de Kort; Willem Jan J van Rooij; Jan S P van den Berg; Boudewijn A A M van Hasselt; Leo A M Aerden; René J Dallinga; Marieke C Visser; Joseph C J Bot; Patrick C Vroomen; Omid Eshghi; Tobien H C M L Schreuder; Roel J J Heijboer; Koos Keizer; Alexander V Tielbeek; Heleen M den Hertog; Dick G Gerrits; Renske M van den Berg-Vos; Giorgos B Karas; Ewout W Steyerberg; H Zwenneke Flach; Henk A Marquering; Marieke E S Sprengers; Sjoerd F M Jenniskens; Ludo F M Beenen; René van den Berg; Peter J Koudstaal; Wim H van Zwam; Yvo B W E M Roos; Aad van der Lugt; Robert J van Oostenbrugge; Charles B L M Majoie; Diederik W J Dippel Journal: N Engl J Med Date: 2014-12-17 Impact factor: 91.245
Authors: I Y L Tan; A M Demchuk; J Hopyan; L Zhang; D Gladstone; K Wong; M Martin; S P Symons; A J Fox; R I Aviv Journal: AJNR Am J Neuroradiol Date: 2009-01-15 Impact factor: 3.825
Authors: Jessalyn K Holodinsky; Amy Y X Yu; Zarina A Assis; Abdulaziz S Al Sultan; Bijoy K Menon; Andrew M Demchuk; Mayank Goyal; Michael D Hill Journal: Curr Neurol Neurosci Rep Date: 2016-05 Impact factor: 5.081
Authors: Esmee Venema; Maxim J H L Mulder; Bob Roozenbeek; Joseph P Broderick; Sharon D Yeatts; Pooja Khatri; Olvert A Berkhemer; Bart J Emmer; Yvo B W E M Roos; Charles B L M Majoie; Robert J van Oostenbrugge; Wim H van Zwam; Aad van der Lugt; Ewout W Steyerberg; Diederik W J Dippel; Hester F Lingsma Journal: BMJ Date: 2017-05-03
Authors: Valery L Feigin; Mohammad H Forouzanfar; Rita Krishnamurthi; George A Mensah; Myles Connor; Derrick A Bennett; Andrew E Moran; Ralph L Sacco; Laurie Anderson; Thomas Truelsen; Martin O'Donnell; Narayanaswamy Venketasubramanian; Suzanne Barker-Collo; Carlene M M Lawes; Wenzhi Wang; Yukito Shinohara; Emma Witt; Majid Ezzati; Mohsen Naghavi; Christopher Murray Journal: Lancet Date: 2014-01-18 Impact factor: 79.321
Authors: Jiahang Su; Lennard Wolff; Pieter Jan van Doormaal; Diederik W J Dippel; Wim van Zwam; Wiro J Niessen; Aad van der Lugt; Theo van Walsum Journal: Neuroradiology Date: 2022-09-27 Impact factor: 2.995