Lauren L Czerniak1, Spencer W Liebel2, Gian-Gabriel P Garcia3, Mariel S Lavieri4, Michael A McCrea5,6, Thomas W McAllister7, Steven P Broglio8. 1. Department of Industrial and Operations Engineering, University of Michigan, 1205 Beal Ave, Ann Arbor, MI, 48109, USA. czernL@umich.edu. 2. Department of Psychiatry, University of Michigan, Ann Arbor, MI, 48109, USA. 3. Massachusetts General Hospital Institute for Technology Assessment, Harvard Medical School, Boston, MA, 02114, USA. 4. Department of Industrial and Operations Engineering, University of Michigan, 1205 Beal Ave, Ann Arbor, MI, 48109, USA. 5. Department of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA. 6. Department of Neurology, Medical College of Wisconsin, Milwaukee, WI, USA. 7. Department of Psychiatry, Indiana University School of Medicine, Indianapolis, IN, USA. 8. Michigan Concussion Center, University of Michigan, Ann Arbor, MI, USA.
Abstract
BACKGROUND: To optimally care for concussed individuals, a multi-dimensional approach is critical and a key component of this assessment in the athletic environment is computer-based neurocognitive testing. However, there continues to be concerns about the reliability and validity of these testing tools. The purpose of this study was to determine the sensitivity and specificity of three common computer-based neurocognitive tests (Immediate Post-Concussion Assessment and Cognitive Testing [ImPACT], CNS Vital Signs, and CogState Computerized Assessment Tool [CCAT]), to provide guidance on their clinical utility. METHODS: This study analyzed assessments from a cohort of collegiate athletes and non-varsity cadets from the NCAA-DoD CARE Consortium. The data were collected from 2014-2018. Study participants were divided into two testing groups [concussed, n = 1414 (baseline/24-48 h) and healthy, n = 8305 (baseline/baseline)]. For each test type, change scores were calculated for the components of interest. Then, the Normative Change method, which used normative data published in a similar cohort, and the Reliable Change Index (RCI) method were used to determine if the change scores were significant. RESULTS: Using the Normative Change method, ImPACT performed best with an 87.5%-confidence interval and 1 number of components failed (NCF; sensitivity = 0.583, specificity = 0.625, F1 = 0.308). CNS Vital Signs performed best with a 90%-confidence interval and 1 NCF (sensitivity = 0.587, specificity = 0.532, F1 = 0.314). CCAT performed best when using a 75%-confidence interval and 2 NCF (sensitivity = 0.513, specificity = 0.715, F1 = 0.290). When using the RCI method, ImPACT performed best with an 87.5%-confidence interval and 1 NCF (sensitivity = 0.626, specificity = 0.559, F1 = 0.297). CONCLUSION: When considering all three computer-based neurocognitive tests, the overall low sensitivity and specificity results provide additional evidence for the use of a multi-dimensional assessment for concussion diagnosis, including symptom evaluation, postural control assessment, neuropsychological status, and other functional assessments.
BACKGROUND: To optimally care for concussed individuals, a multi-dimensional approach is critical and a key component of this assessment in the athletic environment is computer-based neurocognitive testing. However, there continues to be concerns about the reliability and validity of these testing tools. The purpose of this study was to determine the sensitivity and specificity of three common computer-based neurocognitive tests (Immediate Post-Concussion Assessment and Cognitive Testing [ImPACT], CNS Vital Signs, and CogState Computerized Assessment Tool [CCAT]), to provide guidance on their clinical utility. METHODS: This study analyzed assessments from a cohort of collegiate athletes and non-varsity cadets from the NCAA-DoD CARE Consortium. The data were collected from 2014-2018. Study participants were divided into two testing groups [concussed, n = 1414 (baseline/24-48 h) and healthy, n = 8305 (baseline/baseline)]. For each test type, change scores were calculated for the components of interest. Then, the Normative Change method, which used normative data published in a similar cohort, and the Reliable Change Index (RCI) method were used to determine if the change scores were significant. RESULTS: Using the Normative Change method, ImPACT performed best with an 87.5%-confidence interval and 1 number of components failed (NCF; sensitivity = 0.583, specificity = 0.625, F1 = 0.308). CNS Vital Signs performed best with a 90%-confidence interval and 1 NCF (sensitivity = 0.587, specificity = 0.532, F1 = 0.314). CCAT performed best when using a 75%-confidence interval and 2 NCF (sensitivity = 0.513, specificity = 0.715, F1 = 0.290). When using the RCI method, ImPACT performed best with an 87.5%-confidence interval and 1 NCF (sensitivity = 0.626, specificity = 0.559, F1 = 0.297). CONCLUSION: When considering all three computer-based neurocognitive tests, the overall low sensitivity and specificity results provide additional evidence for the use of a multi-dimensional assessment for concussion diagnosis, including symptom evaluation, postural control assessment, neuropsychological status, and other functional assessments.
Authors: M W Collins; S H Grindel; M R Lovell; D E Dede; D J Moser; B R Phalin; S Nogle; M Wasik; D Cordry; K M Daugherty; S F Sears; G Nicolette; P Indelicato; D B McKeag Journal: JAMA Date: 1999-09-08 Impact factor: 56.272
Authors: M Aubry; R Cantu; J Dvorak; T Graf-Baumann; K Johnston; J Kelly; M Lovell; P McCrory; W Meeuwisse; P Schamasch Journal: Br J Sports Med Date: 2002-02 Impact factor: 13.800
Authors: Ruben J Echemendia; Jared M Bruce; Christopher M Bailey; James Forrest Sanders; Peter Arnett; Gray Vargas Journal: Clin Neuropsychol Date: 2012-09-25 Impact factor: 3.535
Authors: R J Elbin; Anthony P Kontos; Nate Kegel; Eric Johnson; Scott Burkhart; Philip Schatz Journal: Arch Clin Neuropsychol Date: 2013-04-21 Impact factor: 2.813
Authors: Steven P Broglio; Michael S Ferrara; Stephen N Macciocchi; Ted A Baumgartner; Ronald Elliott Journal: J Athl Train Date: 2007 Oct-Dec Impact factor: 2.860
Authors: Esther Y Chin; Lindsay D Nelson; William B Barr; Paul McCrory; Michael A McCrea Journal: Am J Sports Med Date: 2016-06-08 Impact factor: 6.202
Authors: John J Leddy; Mohammad Nadir Haider; James M Noble; Brian Rieger; Steven Flanagan; Jacob I McPherson; Kenneth Shubin-Stein; Ghazala T Saleem; Louis Corsaro; Barry Willer Journal: Curr Neurol Neurosci Rep Date: 2021-11-24 Impact factor: 5.081
Authors: Lauren L Czerniak; Spencer W Liebel; Hannah Zhou; Gian-Gabriel P Garcia; Mariel S Lavieri; Michael A McCrea; Thomas W McAllister; Paul F Pasquina; Steven P Broglio Journal: Sports Med Date: 2022-10-14 Impact factor: 11.928
Authors: Kristin Wilmoth; Benjamin L Brett; Natalie A Emmert; Carolyn M Cook; Jeffrey Schaffert; Todd Caze; Thomas Kotsonis; Margaret Cusick; Gary Solomon; Jacob E Resch; C Munro Cullum; Lindsay D Nelson; Michael McCrea Journal: Neuropsychol Rev Date: 2022-08-30 Impact factor: 6.940