Evaluation of CogSport for acute concussion diagnosis in cricket.

OBJECTIVE: The diagnosis of sport-related concussion is a challenge for practitioners given the variable presentation and lack of a universal clinical indicator. The aim of this study was to describe the CogSport findings associated with concussion in elite Australian cricket players, and to evaluate the diagnostic ability of CogSport for this cohort.
METHODS: A retrospective study design was used to evaluate CogSport performance of 45 concussed (male n=27, mean age 24.5±4.5 years; female n=18, 23.5±3.5 years) compared with 45 matched non-concussed (male n=27, mean age 27.3±4.5 years; female n=18, 24.1±4.5 years) elite Australian cricket players who sustained a head impact during cricket specific activity between July 2015 and December 2019.
RESULTS: Median number of reported symptoms on the day of injury for concussed players was 7 out of 24, with a median symptom severity of 10 out of 120. CogSport performance deteriorated significantly in concussed cricket players' Detection speed (p<0.001), Identification speed (p<0.001), One Back speed (p=0.001) and One Back accuracy (p=0.022) components. These components, when considered independently and together, had good diagnostic utility.
CONCLUSION: This study demonstrated good clinical utility of CogSport for identifying concussed cricket players, particularly symptoms and Detection, Identification and One Back components. Therefore, CogSport may be considered a useful tool to assist concussion diagnosis in this cohort, and the clinician may place greater weight on the components associated with concussion diagnosis. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The diagnostic ability of CogSport to differentiate between a concussed and non-concussed player was good.

CogSport performance for four out of eight tests, Detection speed, Identification speed, One Back speed and One Back accuracy, are typically worse than baseline for players with a concussion diagnosis.

Neurocognitive testing, such as CogSport, is a useful clinical adjunct tool to assist the diagnosis of concussion following head impact in elite cricket players.

An abbreviated test, using only two or three components of the CogSport test, could be valuable in time-pressured environments, without reducing the diagnostic ability of the test.

Introduction

Sport-related concussion has become a common and significant diagnosis in the athletic population.1 The diagnosis of sport-related concussion is a challenge for practitioners given the variable presentation and lack of a universal clinical indicator.2 3 Sport-related concussion has been described as an evolving physiological process and neurocognitive symptoms may take minutes to hours to become apparent, adding to the complexity of diagnosis for the medical practitioner at the time of injury if an athlete does not report any symptoms.2 The Sport Concussion Assessment Tool (SCAT-5) is currently the gold standard for concussion diagnosis, but it is thought to lack reliability due to the subjective nature of some components and lack of definitive diagnostic ability.

Computerised neurocognitive tests have become an increasingly popular tool over the past 15 years due to the ease and standardisation of test administration, including baseline testing and ability to quantify return of cognitive performance after a concussion diagnosis.3 4 Neurocognitive tests have been shown to improve diagnostic sensitivity of concussion by 19% compared with diagnosis based on symptoms alone.5 These tests have been able to help diagnose subtle impairments following a head injury, revealing decreased scores in concussed athletes who did not report any symptoms up to 4 days after sustaining a head injury.6–8 Reproducible tests such as these are also useful in monitoring an athlete’s neurocognitive recovery, which can occur on a very individualised timeline.9

There are a number of different computerised neurocognitive tests used to assess sport-related concussion. A study in 2012 found the Computerised Cognitive Assessment Tool for Sport (CogSport; Cogstate) was the second most widely used computerised neurocognitive test.10 CogSport combines both reported symptoms and a neurocognitive test comprising of four components which are completed over 10–15 min in a predetermined order: Detection (psychomotor function), Identification (attention/decision making), One Back (working memory) and One Card Learning (new learning).7 11 Measures of speed and accuracy on these components have been shown to be highly reliable in healthy young people.12

The ‘baseline method’ has been recommended within the literature for neurocognitive tests to compare postinjury data scores with preinjury baseline results.11 13 Louey et al11 found the CogSport test to have both a high sensitivity and specificity for detecting cognitive impairment associated with sport-related concussion when using the baseline method. Uncontrolled influences, such as social and cultural backgrounds, learning, language and past head injuries can then be accounted for with each individual’s baseline test, thus improving the test’s sensitivity and reliability.5 11 14

Injury surveillance in Australian cricket has shown the match incidence rate of concussions per 1000 player days was 2.3 and 2.0 for the elite men and women, respectively.15 This was equivalent to one concussive injury sustained every 9000 balls. Only 30% of concussions were diagnosed immediately, at time of the head impact, with the remainder diagnosed as ‘delayed concussions’.15 It is important that these athletes can be assessed immediately to determine whether they can return to the field of play in a safe manner, in the interest of player welfare.

Cricket Australia has employed CogSport as part of a standardised diagnostic protocol to assess elite cricket players who sustain a head impact during a match or training for the previous four seasons, presenting a valuable opportunity for retrospective analysis. The aim of this study was to describe which CogSport findings are associated with concussion in elite Australian cricket players, and to evaluate the diagnostic ability of CogSport for this cohort.

Methods

Study design and setting

Participants

Forty-five elite Australian cricket players (male n=27, mean age 24.5±4.5 years; and female n=18, mean age 23.5±3.5 years) who were diagnosed with concussion following a head-impact during an organised cricket activity between July 2015 and December 2019.

Players who sustained a subjectively determined, potentially concussive head impact and completed the assessment but were later confirmed to not have sustained a concussion were included for comparison as ‘non-concussed players. The sample of non-concussed players were selected as the most recent age-matched 27 male (27.3±4.5 years) and 18 female (24.1±4.5 years) (same number as concussed) assessments.

Concussion diagnosis

Concussion diagnosis was confirmed by experienced medical staff, based on a full clinical assessment including witnessing or witness description of the incident, clinical assessment, SCAT-5,16 and CogSport, both at the time of the incident and over subsequent days. All information pertaining to the concussive episode was obtained directly from Cricket Australia’s online Athlete’s Management System (Fair Play AMS) by the research team.

CogSport

CogSport assesses a player’s speed and accuracy in four components: psychomotor function, attention/decision making, working memory and new learning. It also assesses the presence and severity of any symptoms. Raw data for speed, accuracy and symptoms are continuous variables. The programme also considers speed and accuracy data relative to an individual’s baseline to determine whether a meaningful decline in performance constitutes a ‘fail’.

Cricket players routinely complete a baseline CogSport assessment during the pre-season. A practice test allows players to familiarise themselves with the test, and minimum performance standards, equivalent to age-matched peers, are inbuilt to ensure players complete a test which is considered ‘valid’. All players who sustain a head impact complete a CogSport test as soon as practical on the day of impact. Both baseline and posthead impact tests are completed in a quiet room, under the direction of medical staff (doctor or physiotherapist) familiar with the testing procedure.

Data analysis

CogSport raw data on the day of head impact was compared with each individual’s most recent baseline raw data to calculate intraindividual change. Data were not normally distributed, hence central tendency is reported as median and IQR. Related-samples Wilcoxon signed rank tests were used to compare postinjury and baseline data.

The diagnostic value of the four test components and overall test result were evaluated using 2×2 contingency tables to calculate sensitivity, specificity, positive predictive values (PPV) and negative predictive values (NPV) with Fisher’s exact p values. Performances for specific components which did not meet the inbuilt test integrity criteria were omitted (eg, identification accuracy <80% suggests the identification component was not completed correctly and hence the performance for this component only was omitted).

Receiver operating characteristic (ROC) curves were used to compare the diagnostic ability of different combinations of test components to identify if a certain combination of test components was superior for concussion diagnosis.

Analysis was completed using Excel (Microsoft, 2016 MSO) and SPSS (V.25, IBM).

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Results

CogSport performance and symptoms on the day of injury for concussed players

Five (out of 355; 1.4%) performances for specific components did not meet the integrity check for the test, hence these specific performances were omitted from analysis.

CogSport performance and symptoms relative to intra-individual baselines are reported in table 1. The median total number of symptoms reported on CogSport on the day of head injury was 7 out of 24 (IQR 5–11) and symptom severity was 10 out of a possible 120 (IQR 7–19).

Table 1

CogSport performance and symptoms for concussed elite male and female cricket players, expressed as Raw score and intraindividual change from most recent baseline

	Male (n=27)		Female (n=18)		All (n=45)		P value
	Raw score	Intraindividual change	Raw score	Intraindividual change	Raw score	Intraindividual change
Detection speed (ms)	307 (290–338)	19 (-3–65)	325 (290–415)	34 (14–100)	315 (289–374)	27 (0–71)	<0.001
Detection accuracy (%)	100 (97–100)	0 (-1–3)	99 (97–100)	0 (-3–0)	100 (97–100)	0 (-3–3)	0.855
Identification speed (ms)	466 (436–520)	49 (3–98)	472 (443–558)	44 (3–91)	471 (437–551)	49 (0–95)	<0.001
Identification accuracy (%)	98 (94–100)	0 (-3–2)	98 (97–100)	0 (-3–0)	98 (94–100)	0 (-3–0)	0.479
One Card Learning speed (ms)	818 (675–883)	14 (-102–106)	799 (720–982)	18 (2–115)	805 (687–905)	15 (-79–119)	0.176
One Card Learning accuracy (%)	77 (70–80)	2 (-6–11)	71 (71–76)	−7 (-12–4)	74 (71–79)	0 (-12–6)	0.664
One Back speed (ms)	625 (543–709)	40 (-15–110)	660 (577–762)	84 (2–136)	635 (563–754)	53 (-5–132)	0.001
One Back accuracy (%)	97 (92–100)	−3 (-3–0)	97 (94–99)	−2 (-5–0)	97 (94–100)	−3 (-3–0)	0.022
Total no of symptoms (of 24)	7 (5–11)	5 (3–10)	8 (5–13)	5 (1–11)	7 (5–10)	6 (4–10)	<0.001
Total symptom severity (of 120)	10 (7–19)	8 (4–17)	14 (6–26)	7 (3–22)	10 (7–16)	10 (6–16)	<0.001

Data presented as median (IQR).

The most commonly reported symptoms for concussed players were: headache (91%, median severity 2 out of 5 (IQR 2–4)), ‘don’t feel right’ (78%, median severity 2 (1–3)), pressure in head (71%, median severity 2 (1–3)), fatigue or low energy (58%, median severity 1 (1–3)), feeling slowed down (51%, median severity 1 (1–3)), balance problems or dizzy (51%, median severity 1 (1–2)) and feeling ‘dinged’ or ‘dazed’ (44%, median severity 2 (1–3)).

Number of symptoms and symptom severity were strongly associated with concussion diagnosis, along with the speed to complete three of the four test components: Detection, Identification, One Back (all p≤0.001).

Diagnostic ability of CogSport

The diagnostic ability of the CogSport test taken as a whole (performance on four components plus symptoms) was: sensitivity 67% (95% CI 54% to 78%), specificity 96% (95% CI 79% to 100%), PPV 98% (95% CI 87% to 100%), and NPV 51% (95% CI 42% to 60%) (p<0.001). The diagnostic ability of separate and combined components, without the addition of patient-reported symptoms, is detailed in tables 2 and 3, relative to normative values and individual baselines respectively. ROC curve analysis (table 4) demonstrated that combinations of test components, without symptoms, were superior to chance, yet no particular combination of components or comparison between individual baseline and normative values was better than others (all p>0.2).

Table 4

Area under the receiver operating characteristic curve

	Normative valuesArea (95% CI)	Individual baselineArea (95% CI)
2-component (D, I)	0.633 (0.518 to 0.749)	0.667 (0.554 to 0.780)
3-component (D, I, OCL)	0.633 (0.518 to 0.749)	0.711 (0.602 to 0.820)
3-component (D, I, OB)	0.644 (0.530 to 0.759)	0.700 (0.590 to 0.810)
4-component (D, I, OCL, OB)	0.644 (0.530 to 0.759)	0.722 (0.615 to 0.830)

D, detection; I, identification; OB, One Back; OCL, one card learning.

Discussion

Sport-related concussion is a challenge for clinicians to diagnose. Objective assessments such as the CogSport neurocognitive test may assist clinicians with their diagnosis. The purpose of this retrospective study was to describe the findings of CogSport test on the day of injury for elite Australian cricket players diagnosed with a concussion, and to evaluate the diagnostic ability of CogSport for this cohort.

Concussion was associated with impaired performance on three of the four CogSport components, in addition to increased number of symptoms and symptom severity. The diagnostic ability of CogSport was enhanced when combining multiple test components, particularly three-component and four-component combinations. Scoring against individual preinjury baseline performance also helped improve the diagnostic ability.

Symptoms

Symptoms most commonly reported by concussed cricket players included headache, ‘don’t feel right’, pressure in head and fatigue, which are consistent with findings in other athlete cohorts.17–20 Concussed cricket players in this study reported a median total number of seven symptoms, yet they were typically of relatively low self-reported severity. While self-reported symptoms are a common and telling aspect of concussion diagnosis,20 21 clinicians must be mindful that symptoms may develop up to 24–72 hours postinjury.22 23 There is also a potential risk for athletes to intentionally under-report any symptoms in order to avoid a diagnosis of concussion.24 Therefore, an objective measure such as CogSport may complement symptom reporting.

CogSport performance

Concussed cricket players typically performed slower for detection, identification and One Back components compared with individual baselines. There was also a decline in the One Back accuracy. A previous study in a similar population has demonstrated that such changes can not be dismissed as normal decline as a result of the physical and mental exertion of participating in a cricket match.25

Slower performance on CogSport following head injury are indicative of cognition changes.12 The simple psychomotor reaction time measure (Detection) has previously been shown to provide the most sensitive index of cognition changes after a concussive head injury.4 Psychomotor (Detection) and attentional (Identification) components have also been observed to be impaired in elite Australian Rules Footballers and Rugby Union players after a concussion when compared with preseason baseline.11 26

The present results suggest the CogSport test, in various forms, were all meaningful in its proficiency to differentiate between concussed and non-concussed players. The use of the baseline method is superior to the normative method, which is consistent with the literature,11 however, both are acceptable. The baseline method was more sensitive, possibly negating the influence of individual factors such as age, gender, education and language.17 22 27 Our results support those of Louey et al11 who found the CogSport normative method to have moderate sensitivity and high specificity for concussion diagnosis in male elite or professional athletes.

Symptoms add further value to the neurocognitive components. This is consistent with previous findings suggesting symptoms and neurocognitive tests can complement one another to assist diagnosis.5 24 In settings where athletes may be prone to under-reporting symptoms, the addition of an objective neurocognitive test may be particularly useful.2 20 24 Czerniak et al1 recently compared a number of computerised neurocognitive tests, including CogSport, and found that while there was an overall low sensitivity and specificity to these type of tests, they should not be abandoned, but used in conjunction with clinical examination and as part of a multidimensional approach. The findings of this study support this approach.

The CogSport test takes approximately 10–15 min for completion of all four components. The findings of this study support the possibility that the test may be abbreviated to three (Detection, Identification and One Back) or even the first two components without considerably reducing the diagnostic ability of the test. An abbreviated test would be valuable in time-pressured environments such as determining whether or not a player is able to return to the field of play during a match which is in-play. This is with the caveat that the clinician may choose to complete the entire four component test if they feel it is required, and only use the CogSport results to inform their clinical judgement along with other assessment(s). The complete four component CogSport test would still remain beneficial in a recovery assessment or when time taken for testing is not critical.

Limitations

A key limitation of this study is that the CogSport results are not independent of concussion diagnosis, as they were used at the time to assist clinical diagnosis. However, the clinical diagnosis is informed by multiple assessments over several days and was completed without knowledge of this study. A further limitation is that the severity of injury and individual factors such as past history of concussions, cultural backgrounds and language or other confounding medical conditions such as headache disorders/migraines or mental health issues which have been shown to affect concussion diagnosis were not accounted for.17 22 27 However, a focus on intraindividual change from baselines reduces the potential impact of individual factors on findings. A learning effect with repeated CogSport assessment may have improved CogSport performance,28 29 yet the findings of concussed players observed in this study suggest this source of error may be minimal.

While this concussive sample size is a larger cohort compared with any current literature documenting concussion in cricket, with only 27 male and 18 female cricketers, the small number meant we were unable to adequately compare results between the male and female cricketers to find any gender variance. Males and females may present differently when concussed,19 30 31 hence combining data in this study may have affected findings. Future research is needed to investigate potential gender differences in concussed players.

Conclusion

This study used a retrospective design in a large cohort of elite Australian cricket players to investigate the changes in the CogSport test, including reported symptoms, that are associated with a concussive head injury and to evaluate the diagnostic ability of CogSport in these athletes. The results of this study suggest the total symptoms and symptom severity score recorded in CogSport are elevated in concussed cricket players. CogSport performance for Detection speed, Identification speed, One Back speed and One Back accuracy are typically worse than baseline for concussed players. Therefore, clinicians may put more weight on these test components when interpreting results to inform their clinical judgement when diagnosing concussion. The diagnostic ability of CogSport to differentiate between a concussed and non-concussed player was noteworthy, however was further improved when combining the CogSport test with a player’s reported symptoms. The assessment of a concussion continues to be multimodal and neurocognitive testing, such as CogSport, is a useful clinical adjunct tool to assist the diagnosis of concussion following head impact in elite cricket players.

Table 2

Diagnostic ability of CogSport for concussion diagnosis compared to CogSport normative values

	Sensitivity% (95% CI)	Specificity % (95% CI)	Positive predictive value % (95% CI)	Negative predicative value % (95% CI)	Fisher exact P value
D	27 (15 to 42)	98 (88 to 100)	92 (62 to 99)	57 (53 to 62)	0.002
I	18 (8 to 33)	98 (88 to 100)	89 (51 to 98)	55 (51 to 59)	0.015
OCL	0 (0 to 8)	100 (92 to 100)	N/A	49 (49 to 49)	N/A
OB	11 (4 to 25)	100 (92 to 100)	100	54 (51 to 56)	0.026
2-component (D, I)	31 (18 to 47)	96 (85 to 99)	88 (63 to 97)	58 (53 to 63)	0.002
3-component (D, I, OCL)	31 (18 to 47)	96 (85 to 99)	88 (63 to 97)	58 (53 to 63)	0.002
3-component (D, I, OB)	33 (20 to 49)	96 (85 to 99)	88 (65 to 97)	59 (54 to 64)	<0.001
4-component (D, I, OCL, OB)	33 (20 to 49)	96 (85 to 99)	88 (65 to 97)	59 (54 to 64)	<0.001

D, detection; I, identification; NA, not available; OB, one back; OCL, one card learning.

Table 3

Diagnostic ability of CogSport for concussion diagnosis compared to individual baseline performance

	Sensitivity% (95% CI)	Specificity % (95% CI)	Positive predictive value % (95% CI)	Negative predicative value % (95% CI)	Fisher exact P value
D	47 (32 to 62)	87 (73 to 95)	78 (61 to 89)	62 (55 to 69)	0.001
I	23 (11 to 38)	96 (85 to 99)	83 (54 to 96)	56 (52 to 60)	0.014
OCL	25 (13 to 40)	91 (78 to 97)	73 (49 to 89)	54 (49 to 59)	0.087
OB	20 (10 to 35)	98 (88 to 100)	90 (54 to 99)	56 (52 to 60)	0.007
2-component (D, I)	49 (34 to 64)	84 (71 to 94)	76 (60 to 87)	62 (55 to 69)	0.001
3-component (D, I, OCL)	64 (49 to 78)	78 (63 to 89)	74 (62 to 84)	69 (59 to 77)	<0.001
3-component (D, I, OB)	56 (40 to 71)	84 (71 to 94)	78 (63 to 88)	66 (57 to 73)	<0.001
4-component (D, I, OCL, OB)	67 (51 to 80)	78 (63 to 89)	75 (63 to 84)	70 (60 to 78)	<0.001

D, detection; I, identification; OB, One Back; OCL, one card learning.