Effect of a concussion on subsequent baseline SCAT performance in professional rugby players: a retrospective cohort study in global elite Rugby Union.

OBJECTIVES: This study assessed whether concussion affects subsequent baseline performance in professional rugby players. Annual baseline screening tests are used to guide return-to-play decisions and concussion diagnosis during subsequent screens. It is important that baseline performances are appropriate and valid for the duration of a season and unaffected by factors unrelated to the current head impact event. One such factor may be a concussion following baseline assessment.
SETTING: The World Rugby concussion management database for global professional Rugby Union. PARTICIPANTS: 501 professional rugby players with two baseline Sports Concussion Assessment Tools (SCATs) and an intervening concussion (CONC) were compared with 1190 control players with successive annual SCAT5s and no diagnosed concussion (CONT). PRIMARY AND SECONDARY OUTCOME MEASURES: Symptom endorsement, cognitive and balance performance during annual SCAT baseline assessments.
RESULTS: Players with a diagnosed concussion (CONC) endorsed fewer symptoms (change -0.42, 95% CI -0.75 to -0.09), and reported lower symptom severity scores during their second assessment (T2, p<0.001) than non-concussed players (CONT). Concussed players also improved Digits Backward and Final Concentration scores in T2 (p<0.001). Tandem gait time was improved during T2 in CONT. No other sub-mode differences were observed in either group.
CONCLUSIONS: Reduced symptom endorsement and improved cognitive performance after concussion may be the result of differences in the motivation of previously concussed players to avoid exclusion from play, leading to under-reporting of symptoms and greater effort in cognitive tests. Improved cognitive performance may be the result of familiarity with the tests as a result of greater exposure to concussion screening. The changes are small and unlikely to have clinical significance in most cases, though clinicians should be mindful of possible reasons, possibly repeating sub-modes and investigating players whose baseline scores change significantly after concussion. The findings do not necessitate a change in the sport's concussion management policy. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is a large-scale, real-life study that compares 501 professional men’s Rugby Union players who experienced a concussion and who have a baseline Sports Concussion Assessment Tool (SCAT) assessment on either side of this head injury event, to 1190 professional men’s players with two successive SCAT5s but without an intervening concussion.

Data are obtained in a controlled environment, as per standardised assessments and an operational definition of concussion created by World Rugby.

A limitation is that descriptive characteristics of players within the cohort, including age, ethnicity, playing experience and language, are unavailable.

Study design (cross-sectional of a census sample) does not allow evaluation of the mechanism or factors responsible for observed changes between T2 and T1.

Changes to the SCAT, including a version update affecting symptom reporting and a change in word-length, reduce sample number for some sub-mode analyses.

Introduction

Concussion is the most prevalent injury in professional Rugby Union,1 with an incidence between 14 and 20 concussions per 1000 match hours.1–4 In most cases, cognitive deficits, balance and symptoms improve rapidly during the first 2 weeks after injury, resolving within 1 month,5 but in some cases, impairments may persist for longer, particularly with respects to symptoms. It has been found that previously concussed collegiate athletes report more symptoms at a greater severity than non-concussed players,6 with conflicting results on neurocognitive performance in college athletes, adolescents and adult jockeys.7–9 Should such concussion-related decrements persist beyond return-to-play, it may invalidate the baseline thresholds against which subsequent off-field and diagnostic screens are assessed.

It is important, therefore, that the baseline scores and clinical reference limits that are used to interpret abnormal scores for each sub-mode be appropriate and valid for the duration of a season, and unaffected by factors unrelated to the current head impact event. One factor that may directly affect a player’s baseline performance is a concussion occurring after baseline measurement.

A concussion may also alter screening performance indirectly, through repeated exposure to tests and resultant learning effects. Rugby Union has used a three-point Head Injury Assessment (HIA) process since 2014. This process has been adapted from the Sports Concussion Assessment Tool (SCAT), and assesses a player at the time of head impact event during matches (HIA1), 3 hours after the match (HIA2) and after two nights’ rest (HIA3). It is possible that players who undergo the process after head injuries become familiar with the symptoms, the questions asked during cognitive sub-modes and the performance of balance tests. Anecdotally, team doctors have occasionally reported players achieving better cognitive sub-mode scores during HIA2 or HIA3 testing than during baseline tests, along with reports that players learn cognitive sub-modes in order to pass diagnostic assessments. Collectively, a consideration is whether the diagnostic process after a concussion, consisting of the off-field screen, followed by the HIA2 and HIA3 assessments, create a learning effect in players who undergo those tests more frequently than non-concussed peers.

In either scenario, a previously obtained baseline assessment may be rendered invalid, either by persistent concussion-related impairments, or by learning effects owing to more frequent exposure to the screen sub-modes. Given that return to play and diagnostic decisions are made in comparison to a baseline SCAT, it is important to consider whether any changes are long-lived and large enough to compromise future clinical decisions. Assessing the degree and duration of potential changes in the SCAT5 sub-modes after a concussion will guide clinicians as to whether a player requires an updated baseline assessment after concussion, or whether the current requirement of annual baseline testing is sufficient.

Aims

The aim of the present study was to assess whether baseline SCAT performance is affected by previous concussion. Specifically, we explored how each SCAT sub-mode performance was affected by an intervening concussion, compared with a group of similarly matched players who did not experience a concussion between two successive annual baseline assessments. Since the control group comprised a group of players who underwent two SCAT5 baseline assessments but did not experience a concussion, we are also able to obtain a preliminary analysis of whether two repeated baselines influence performance, possibly through learning effects.

Methods

Study design, setting and study population

A retrospective cohort study was performed using data from the World Rugby Head Injury Assessment (HIA) database, which contains baseline and diagnostic concussion screen results from the professional game. The HIA process used by Rugby Union is a three-phase protocol, adapted from the SCAT. The first phase combines an abridged SCAT (HIA1 off-field screen) conducted at the time of the head impact event with video review and clinical examinations to guide return to play decisions. After the game, full SCAT assessments within 3 hours of the match (HIA2 screen) and after two nights’ rest (HIA3 screen) are used to support the diagnosis of concussion.10 During all three assessments, a player’s performance in various sub-modes is assessed against their own previously performed SCAT5 baseline assessment, or if a baseline assessment is absent, against normative data (clinical reference limits) derived from a rugby-playing population.11

In order to use the HIA process, a competition must adhere to mandatory competition player welfare standards (World Rugby Player Welfare Site) that ensures a standardised approach to concussion detection and management as well as data collection. The source population thus comprises the majority of eligible professional male players in domestic and international competitions that underwent mandatory baseline SCAT assessment between 2015 and 2019.

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of the research.

Baseline screening

Baseline SCAT assessments are typically administered before commencement of the relevant competition season or tournament, according to methods described previously.12 For the purposes of the present study, cases were identified as all professional players who had a baseline screen both before (CONC T1) and after (CONC T2) a documented concussion between 2017 and 2019. Concussions were identified as per the World Rugby operational definition, which requires either an abnormal HIA2 screen, HIA3 screen, a doctor diagnosis or the presence of a so-called Criteria 1 sign on-field at the time of the head impact event.1 Women were excluded from the analysis, since previous research has found greater symptom endorsement which may affect both the initial assessment and resultant changes.13 A control group (CONT) consisted of all professional male players with two SCAT5 baseline assessments but no documented concussions over the same sampling period (CONT T1 and CONT T2). Since we wished to explore only how a ‘recent concussion’ affected SCAT performance, we excluded cases where the T2 screen was conducted more than a year after the concussion, as well as baseline screens were done more than 2 years apart were excluded from analysis.

In some cases, the baseline screens at T1 and T2 were different versions of the SCAT, which changed from SCAT3 to SCAT5 during the sampling period, or used a different word length list. For our analysis of word-list sub-modes (Immediate memory and Delayed recall) and symptom endorsement, we analysed only paired and directly comparable sub-modes, and thus excluded all cases and controls where a sub-mode was assessed differently in T2 compared with T1. We report the number of available paired comparisons in the results. Since the sample was a fixed census sample, precision was indicated by the size of the CIs.

Raw scores were compared using descriptive statistics, while the change in each sub-mode was calculated as T2 minus T1 for CONC and CONT, and reported as mean change and 95% CIs. The proportion of players who improved, worsened and achieved the same score during T2 was determined for each group. An improvement in the sub-mode performance was identified when symptoms endorsed, errors made during balance tests and time taken to complete the tandem gait assessment reduced, whereas cognitive sub-modes were deemed to improve when their score increased in T2.

A Wilcoxon signed-rank test was performed to assess whether sub-mode scores were significantly different between T2 and T1 in CONC and CONT. The null hypothesis (T2=T1) was rejected when p<0.004, based on a Bonferroni correction of the original alpha of 0.05, divided by the 12 SCAT subdomains assessed (0.05/12=0.004).

The period between subsequent SCAT tests in both groups was assessed using a Mann-Whitney U test.

An ordinal regression model was developed to determine whether sub-mode performance changed differently in concussed players compared with controls. The change in sub-mode score (T2 minus T1) was categorised as ‘worse’ (−1), ‘no change’ (0) or ‘improved’ (+1) as the dependent variable, and the number of days between tests used as an independent variable. Based on the fact that the period between tests was significantly different between the two groups (p<0.001), days between tests was added as an independent variable to the logistic regression. Thus, an adjusted for time between tests OR was calculated for each sub-mode, reporting the odds of sub-mode performance improving more at T2 in CONC than at T2 in CONT. An OR greater than 1 and a p value less than 0.0042 thus means that an improvement in sub-mode performance in CONC T2 was more likely than in CONT T2. A likelihood ratio test was run after each regression that found a significant relationship in order to test the proportional odds assumption.

The influence of the period between the concussion and T2 in this group was examined by dividing the concussion cases into those players who had experienced a concussion within 3 months of the T2 baseline (recent, CONC REC), and players whose concussion occurred more than 3 months before the T2 assessment (distant, CONC DIST). REC and DIST were compared in terms of their distribution with the three outcome categories (‘worsen’, ‘no change’ and ‘improve’) using a Fisher’s exact test. As there were 22 symptoms, the Bonferroni corrected p value for this Fisher’s exact test was set at 0.0022 (0.05/22) for these analyses.

Results

501 players with a baseline SCAT assessment on each side of a diagnosed concussion were analysed for the CONC group. The median period between T1 and T2 baseline assessments was 362 days (IQR 318 to 395 days) and the period between the concussion and the T2 baseline screen was 222 days (IQR 139 to 295). The control cohort consisted of 1190 players who underwent two SCAT5 assessments without a concussion in the period between. Median period between T1 and T2 was 334 days (IQR 294 to 365 days).

The available paired cases for analysis, along with sub-mode means, 95% CI and medians are shown for T1 and T2 in the two groups in table 1. Mean Immediate memory score was greater in CONC than CONT at T1 and T2. Mean symptom number and severity were lower in CONC than CONT at T2, and Digits backward, Final concentration scores and Delayed Recall scores were higher in CONC than CONT at T2. No differences were found between median scores for any sub-mode at T1.

Table 1

Descriptive statistics of sub-mode performance in players who experienced a concussion (CONC) and a control group with no concussions (CONT)

	Concussions (CONC, n=501)				CONTROLS (CONT, n=1190)
	T1 (pre)		T2 (post)		T1		T2
	Mean (95% CI)	Median	Mean (95% CI)	Median	Mean (95% CI)	Median	Mean (95% CI)	Median
Symptom number(n=143 CONC)	1.10 (0.78 to 1.42)	0	0.66*(0.41 to 0.91)	0	1.42(1.26 to 1.58)	0	1.34(1.19 to 1.49)	0
Symptom severity	1.63 (1.11 to 2.15)	0	0.95*(0.58 to 1.32)	0	2.20(1.93 to 2.47)	0	2.08(1.82 to 2.34)	0
Orientation	4.85(4.82 to 4.88)	5	4.81(4.78 to 4.84)	5	4.82(4.8 to 4.84)	5	4.83(4.81 to 4.85)	5
Immediate memory(n=119 CONC; 723 CONT)	22.36*(21.74 to 22.98)	22	22.85†(22.16 to 23.54)	23	21.32(21.06 to 21.58)	21	21.60(21.34 to 21.86)	22
Digits backward	3.21(3.13 to 3.29)	3	3.45†(3.38 to 3.52)	4	3.20(3.15 to 3.25)	3	3.25(3.20 to 3.30)	4
Concentration	4.14 (4.05 to 4.23)	4	4.39†(4.32 to 4.46)	5	4.08(4.02 to 4.14)	4	4.15(4.10 to 4.20)	4
Delayed recall(n=119 CONC; 723 CONT)	7.37(7.05 to 7.69)	7	7.78†(7.48 to 8.08)	8	7.04(6.9 to 7.18)	7	7.07(6.94 to 7.20)	7
Tandem gait assessment	10.66 (10.34 to 10.98)	11	10.40(10.10 to 10.70)	11	10.83(10.73 to 10.93)	11	10.53(10.43 to 10.63)	11
M-BESS
Double leg	0.01(0 to 0.02)	0	0.00(-0.01 to 0.01)	0	0.01(0 to 0.02)	0	0.01(0 to 0.02)	0
Single leg	2.08(1.90 to 2.26)	2	1.84(1.68 to 2.00)	1	1.88(1.77 to 1.99)	1	1.78(1.68 to 1.88)	1
Tandem stance	0.84(0.72 to 0.96)	0	0.74(0.63 to 0.85)	0	0.78(0.71 to 0.85)	0	0.74(0.67 to 0.81)	0
Total errors	2.93(2.67 to 3.19)	2	2.59(2.37 to 2.81)	2	2.68(2.53 to 2.83)	2	2.54(2.4 to 2.68)	2

Sub-mode scores are shown as means (95% CI) and medians.

*denotes significantly lower than in CONT at the corresponding time point.

†denotes significantly greater than in CONT at the corresponding time point

M-BESS, Modified Balance Error Scoring System.

Table 2 shows the change in performance for each sub-mode, calculated as the difference between T2 and T1, whether the sub-mode score improved, worsened or remained the same compared with T1, and the proportion of players who fell into each category for that sub-mode. The Wilcoxon signed-rank test p value evaluates whether the T2 versus T1 change in that sub-mode was significant within each group. The OR evaluates whether an improvement in sub-mode score was more likely in the CONC than CONT group, with a ratio greater than 1 indicating that the sub-mode improved more in T2 of CONC than in T2 of CONT.

Table 2

Changes in sub-mode performance in CONC and CONT

Sub-mode	Cohort	N	Change in sub-mode score (post–pre, mean (95% CI)	Change in score	Improved (%)	Worse (%)	No change (%)	Wilcoxon signed-rank test p value	OR (95% CI) for odds of improvement more likely in CONC than CONT
Symptom number	CONT	1190	−0.07 (-0.22 to 0.07)	No change	24.8	23.9	51.3	0.551	1.52 (1.10 to 2.11), p=0.011
	CONC	143	−0.42 (-0.75 to -0.09)	Decrease	30.8	14.0	55.2	0.002*
Symptom severity	CONT	1190	−0.11 (-0.36 to 0.12)	No change	25.6	23.9	50.4	0.387	1.54 (1.11 to 2.13), p=0.009
	CONC	143	−0.67 (-1.17 to -0.17)	Decrease	32.2	14.0	53.8	0.001*
Orientation	CONT	1190	0.01 (-0.02 to 0.04)	No change	12.2	11.5	76.3	0.619	1.26 (0.98 to 1.61), p=0.073
	CONC	501	−0.03 (-0.08 to 0)	No change	11.2	15.4	73.5	0.072
Immediate memory (10-word list)	CONT	723	0.28 (0 to 0.57)	No change	48.4	41.9	9.7	0.040	1.21 (0.83 to 1.75), p=0.320
	CONC	119	0.49 (-0.15 to 1.15)	No change	52.1	36.1	11.8	0.075
Delayed recall (10-word list)	CONT	723	0.02 (-0.11 to 0.16)	No change	38.0	36.7	25.3	0.759	1.39 (0.97 to 1.99), p=0.073
	CONC	119	0.4 (0.1 to 0.7)	No change	45.4	28.6	26.1	0.016
Digits backward	CONT	1190	0.04 (0 to 0.1)	No change	27.5	22.9	49.7	0.034	1.33 (1.08 to 1.62), p=0.007
	CONC	501	0.23 (0.14 to 0.31)	Increase	30.9	15.4	53.7	<0.001*
Final concentration	CONT	1190	0.06 (0.01 to 0.12)	No change	29.4	24.3	46.3	0.014	1.34 (1.09 to 1.64), p=0.005
	CONC	501	0.25 (0.17 to 0.34)	Increase	32.7	15.8	51.5	<0.001*
Tandem gait time	CONT	1185	−0.29 (-0.41 to -0.17)	Decrease	45.9	33.9	20.2	<0.001	0.94 (0.76 to 1.17), p=0.583
	CONC	433	−0.28 (-0.5 to -0.07)	No change	48.5	41.6	9.9	0.012
Double leg errors	CONT	1190	0 (-0.01 to 0.01)	No change	1.0	1.4	97.6	0.355	1.31 (0.63 to 2.72), p=0.464
	CONC	501	0 (-0.01 to 0.01)	No change	1.2	0.8	98.0	0.528
Single leg errors	CONT	1190	−0.1 (-0.22 to 0.02)	No change	37.7	35.6	26.6	0.219	1.08 (0.88 to 1.32), p=0.460
	CONC	501	−0.23 (-0.44 to -0.03)	No change	39.5	33.1	27.3	0.038
Tandem stance errors	CONT	1190	−0.04 (-0.12 to 0.04)	No change	28.6	28.2	43.2	0.719	1.08 (0.08 to 1.31), p=0.471
	CONC	501	−0.09 (-0.24 to 0.05)	No change	27.7	25.7	46.5	0.351
Total balance errors	CONT	1190	−0.13 (-0.31 to 0.03)	No change	39.0	38.7	22.3	0.451	1.20 (0.98 to 1.46), p=0.081
	CONC	501	−0.33 (-0.63 to -0.04)	No change	43.1	34.9	22.0	0.043

The relative performance change (T2 – T1) is indicated for each sub-mode and cohort.

Performance in Concussions (CONC) and Controls (CONT)

*denotes significant change in sub-mode score T2 versus T1. The OR shows the odds (95% CI) of sub-mode performance improving in CONC relative to CONT during the second assessment. Bonferroni correction resulted in significance accepted at p<0.004.

Figure 1 shows the changes in selected sub-modes for illustrative purposes, with symptoms (top left panel) and one sub-mode from cognitive (digits backward score, middle left panel) and one sub-mode from the balance (single leg errors, bottom left panel) sub-modes selected by virtue of being significantly different in CONC than CONT. Frequency histograms of sub-mode score changes for each of the three illustrative subdomains in the CONT group, the recent concussions (REC CONC, concussion within 3 months of POST) and distant concussions (DIST CONC, concussion more than 3 months before T2) are also shown.

Figure 1

Changes in sub-mode performance (T2 – T1) for symptom number, digits backward and single leg errors (left column), and histograms depicting proportion of players within each cohort changing in the three selected sub-modes. Data are shown for all Controls (CONT) and Concussions (CONC) are divided into those where the concussion occurred more than 3 months before the T2 baseline (DIST) and fewer than 3 months before the T2 baseline (REC). SCAT, Sports Concussion Assessment Tool.

Symptoms

Symptom number and severity were significantly lower at T2 than T1 in the CONC group (p=0.0022 and p=0.001 for number and severity, respectively), with 30.8% of players reporting fewer symptoms at T2, compared with 14.0% who endorsed more symptoms at T2 (table 2 and figure 1). Of the players who reported fewer symptoms (30.8%, table 2), almost half (14.0%) decreased by only one symptom, while only 10% of players reduced symptom number by 3 or more. In the CONT group, symptom number and severity were similar between T2 and T1.

While there was a trend towards a decrease in symptom number and symptom severity in CONC compared with CONT (OR 1.52 (1.10 to 2.11, p=0.011) for symptom number and for severity (OR 1.54 (1.11 to 2.13, p=0.009)), this did not reach significance using the Bonferroni corrected p value (table 2).

Cognitive sub-modes

Digits backward score was significantly better at T2 than T1 in CONC (p<0.001) but was unchanged in CONT. Final concentration score, of which Digits Backward makes up four of the five possible correct answers, was similarly improved in T2 compared with T1 in CONC, but was not different in the CONT group at T2 compared with T1.

Orientation scores, Immediate Memory and Delayed Recall performance were not different between T2 and T1 in either cohort (table 2).

Balance sub-modes

Tandem gait time was significantly faster in T2 compared with T1 (improvement of 0.29 s, table 2) in CONT, but was not different in CONC (p=0.012). There was no difference in the odds of improvement in CONC and CONT.

Balance errors during Double, Single Leg and Tandem Stance were unchanged T2 versus T1 in both CONC and CONT. Total balance errors were thus unaffected by concussion and did not change over time in the CONT group.

Effect of time duration between concussion and second baseline test (post) gap on sub-mode performance

The CONC cohort (n=501) was divided into players whose concussion occurred within 3 months of T2 baseline testing (REC, n=60) and those whose concussion occurred more than 3 months before T1 baseline (DIST, n=441). Table 3 shows the proportion of each group improving, worsening and staying the same in T2 versus T1, and the results of the χ2 analysis for each sub-mode.

Table 3

χ2 analysis of improvements in SCAT5 sub-modes for recent (<3 months) and distant (>3 months) concussed players

Sub-mode	Cohort	N	Improved (%)	Worse (%)	No change (%)	P value
Symptom number	Distant	123	30.1	15.4	54.5	0.548
	Recent	20	35.0	5.0	60.0
Symptom severity	Distant	123	31.7	15.4	52.8	0.528
	Recent	20	35.0	5.0	60.0
Orientation	Distant	441	10.7	15.2	74.1	0.501
	Recent	60	15.0	16.7	68.3
Immediate memory(10-word list)	Distant	101	52.5	36.6	10.9	0.819
	Recent	18	50.0	33.3	16.7
Delayed recall(10-word list)	Distant	101	44.6	27.7	27.7	0.644
	Recent	18	50.0	33.3	16.7
Digits backward	Distant	441	29.0	16.3	54.6	0.032
	Recent	60	45.0	8.3	46.7
Final concentration	Distant	441	30.6	16.6	52.8	0.026
	Recent	60	48.3	10.0	41.7
Tandem gait time	Distant	381	49.3	40.9	9.7	0.602
	Recent	52	42.3	46.2	11.5
Double leg errors	Distant	441	1.1	0.7	98.2	0.341
	Recent	60	1.7	1.7	96.7
Single leg errors	Distant	441	40.8	30.8	28.3	0.016
	Recent	60	30.0	50.0	20.0
Tandem stance errors	Distant	441	28.1	26.1	45.8	0.733
	Recent	60	25.0	23.3	51.7
Total balance errors	Distant	441	43.8	33.1	23.1	0.051
	Recent	60	38.3	48.3	13.3

Bonferroni correction resulted in significance accepted at p<0.002.

SCAT, Sports Concussion Assessment Tool.

No significant differences were found for any sub-mode, with the proportion of players improving and worsening similar in REC and DIST concussion cases. A similar proportion of improved and worsened scores was found for Final Concentration score (table 3).

Discussion

This study assessed whether SCAT5 sub-modes were altered in a group of professional rugby players who experienced a concussion, and whether this might have implications for the assessment of annual baseline screens and the clinical use and interpretation of return-to-play and diagnostic screens in the season after the concussion.

Symptom endorsement after concussion

Our first important finding was that players who experienced a concussion reported fewer symptoms during their T2 baseline screen than during T1 (table 2), with over twice as many players reporting fewer symptoms during T2. In contrast, there was no difference in symptom endorsement in T2 compared with T1 of the non-concussed control group.

The clinical significance of these changes is questionable, given the small size of the symptom number and severity reductions we document (table 2, figure 1). For example, while 30.8% of the concussed cohort reported fewer symptoms at T2, almost half of these players reported only one symptom fewer, and only 10.5% had a reduction of three or more symptoms compared with T1, a proportion similar to that observed in CONT (9.9%). Given the non-specific nature of symptoms and their potential to vary from day to day for non-injury related reasons, and that clinicians are instructed to assess symptoms reported at the time of a baseline screen or a suspected head injury in the context of the player’s normal symptom traits, these small changes are unlikely to influence clinical actions taken. This is particularly true given that recent research shows that clinicians frequently apply clinical judgement to overrule the presence of reported symptoms.14

The reduction in symptom endorsement we document contradicts research in college collision sport athletes, where a previous concussion was associated with greater baseline symptom endorsement and severity.6 Differences in study design, definitions and participant age and level may partly explain these differences. For example, previous studies have used self-reported concussion rather than a standardised clinical diagnosis of concussion. Given that athletes may under-report concussion symptoms, self-reporting may group concussed athletes with those who have truly never experienced a concussion.7 In our study, concussion was confirmed as part of the World Rugby operational definition for concussion,10 though it is still possible that players from our CONT group experienced concussions that were not detected at the time of injury.

The observed reduction in symptom endorsement in concussed players during T2 may be the result of familiarity with the concussion screening process, because these players are exposed to the SCAT more than non-concussed players as part of World Rugby’s HIA process. This may lead to these concussed players knowingly withholding symptoms that would exclude them from sporting participation in future. Desire to continue playing has been shown to be a primary motivator for under-reporting concussions.15 Players who experienced a previous concussion were less likely to report symptoms when assessed 2 hours after their next concussive event,6 suggested to be the result of less distress due to familiarity with the screen, as well as their deliberate minimisation of symptoms to return to play more quickly, having experienced a concussion-induced absence previously.6 In the short-term (first 2 weeks) after concussion, male athletes have been shown to report more symptoms than female athletes, a difference attributed to male players’ desire to avoid removal from sport.16

It may therefore be that players who suffer concussions become more familiar with the screening and clinical process and recognise that their symptom endorsement increases the amount of time they may remain on the side-line in the event of a future concussion. While the time period between injury and reporting is considerably greater in this study than previous research, it is possible that players continue to knowingly downplay any symptoms when they are assessed prior to their next season.

We attempted to explore this possibility by analysing a sub-cohort of players who had experienced their concussion within 3 months of T2, but found no significant effects in this analysis (table 3). This is likely due to a small sample size for recent concussions, since we did find tendencies for changes in symptom endorsement after more recent concussions, sufficient to hypothesise that the time between injury and screen may be a relevant factor. This is an area that should be considered for future research.

Our second important finding was that most cognitive sub-modes are unaffected by concussion, and that no cognitive sub-modes change in the CONT group. We found a significant improvement in Digits Backward score in T2 in CONC, and thus improved Final Concentration score, of which Digits Backwards comprises four of the five available correct answers (table 2, figure 1). Mean Delayed recall score was significantly higher at T2 in concussed players (table 1), but no statistical difference was found in the change from T1 to T2, or the odds of an improvement in CONC relative to CONT.

We suggest that this may be the result of test exposure, motivation to perform better after a concussion, and memory, which may be related to the period between concussion testing and repeat baseline, since we found that a higher proportion of the 60 REC cases improved on concentration testing compared with DIST cases (table 3).

Previous research supports this possibility, since previously concussed college athletes have been found to score better than never-concussed athletes in most cognitive sub-modes of the SCAT3.17 This was attributed either to effort differences between concussed and non-concussed players, or a learning effect as a result of repeated exposures to the tests during diagnostic and return-to-play protocols.

Other research has found no effects of distantly-occurring concussion on cognitive function, assessed using computerised or neurocognitive tests,7 while others have found detrimental effects of distant concussions on neuropsychological assessments in jockeys8 and high school athletes.9 18 Differences between these findings and the present study may exist as a result of severity of concussions being greater among jockeys, the age of the players assessed, the time period between the concussion and the neurocognitive assessment and self-reported versus clinically diagnosed concussions as inclusion criteria, as described by Bruce and Echemendia.7

In the present study, since the CONT group performed only a single assessment approximately 1 year apart, compared with multiple assessments (HIA1, HIA2 and HIA3) with the cognitive sub-modes in the CONC group, it seems reasonable to suggest that repeated exposure to testing, combined with a possible desire to perform well in the cognitive sub-modes after concussion, improves performance through both learning and effort mechanisms. This effect was not found for Immediate Memory, which may reflect the greater challenge of using the 10-word lists in SCAT5 screens.

The implication of this finding is that annual baseline screening is an important requirement for cognitive sub-modes, to ensure that each player’s baseline reference scores are updated to account for possible learning from more frequent testing. The paradox is that repeating the screen may be a contributing factor to improved scores, and so clinicians are advised to vary the word or number sequence within each cognitive sub-mode, particularly if a player presents with improved scores compared with previous screens. Our finding also reinforces that the tests used to assess cognitive function should be varied to prevent learning, and potentially increased in difficulty to eliminate ceiling effects and the ease with which words or number sequences can be memorised.

Balance sub-modes and concussion

We found that tandem gait time improved in T2 in the CONT group, but not in CONC. The improvement observed was 0.29 s (0.17 to 0.41 s), though there was no significant difference in the odds of a CONT player improving compared with previously concussed player (table 2), and the proportion of players in each group whose performance changed by more than one second was similar.

Other balance sub-modes were similar between CONT and CONC. We did find a tendency for a greater proportion of players to make more single leg errors in REC cases (50% worse, compared with 30% with fewer errors, n=60) compared with DIST cases (30.8% more errors compared with 40.8% making fewer errors, table 3). This could either be the result of the direct effects of concussive injury on balance in the more recent aftermath of concussion, or may be related to detraining after the concussive injury, but further research may be required to explore this.

It may be significant that balance sub-modes, unlike symptom endorsement or cognitive performance, can be neither learnt, practiced nor biassed by subjective reporting. The absence of differences in T2 of CONC, with the exception of small impairments in tandem gait time and single leg balance after more recent concussions, may reflect a more objective standard for the evaluation of lasting effects of concussion, though the precise manifestation of brain injury on various sub-modes is complex.

Given that a previous concussion does not affect balance scores significantly, and that we do not detect balance improvements as a result of repeat testing, it may be considered that annual screening of balance sub-modes is not necessary. However, we advise caution before any changes are made to current process, since we cannot assure that more recent concussions do not present with affected balance, or how repeated concussions may alter balance performance. As such, until further evidence confirms no effect on balance, we would advise that balance should be assessed as part of annual screening.

Limitations

Our analysis compared SCAT5 performance in a concussed group to a control group who underwent two SCAT5 assessments approximately 1 year apart, with no documented concussion between them. It is possible that a concussion or head injury was undiagnosed during this period, which may affect players in CONT, and possibly prior to T1 SCAT5 in both groups. The relative stability of testing both after concussion and in the control group means that though this cannot be excluded, it is unlikely to materially affect the results.

We also did not exclude players who would have performed multiple SCATs prior to the analysis period in this study, whether in the form of the off-field screen, diagnostic screens or baseline assessments. Our cohort included players with between zero and three SCAT assessments prior to their T1 assessment in the present study, though the proportion of players with these prior tests was similar between CONC and CONT. For this study, we chose to include players irrespective of previous SCAT history, because the formalised use of the SCAT5 in rugby means that all players in their second season or more of rugby who undergo the diagnostic and baseline are going to be exposed to repeated testing. Thus, for the sake of external validity of our findings, we felt it preferred to include all players, irrespective of their previous SCAT history.

The study design does not allow us to quantify the magnitude of any possible learning effects, in part because of the range of factors that may affect test performance, such as time between tests, time between concussion and T2 screens, language, ethnicity, age and fitness levels both pre-injury and post-injury, all of which have been found to affect performance during cognitive performance and symptom endorsement.19–23 As such, we can identify and postulate that possible learning effects explain the changes in performance in T2 versus T1, but we cannot quantify the magnitude of these changes distinct from normal test to test variation, and this would require a specific study to assess. Similarly, we are unable to account for possible differences in characteristics between the CASE and CONT group in the present study. Any differences, which may include age, playing level, experience, ethnic group and language, could influence differences between the groups.

Finally, the study design used here does not allow us to commit to a change in sub-modes score that would always have clinical meaning. For all the above-mentioned limitations, future research may explore how multiple tests, rather than two tests, influence performance as a result of potential learning effects. This research should assess players who are known not to have been concussed, undertaking tests with the same frequency as would typically occur in concussed players and then compared with concussed players to quantify the magnitude of learning effects and the variation in sub-mode scores that is clinically meaningful.

Clinical implications

We have described that symptom endorsement during baseline assessments is reduced in players with a previous concussion, that digits backwards and final concentration scores are improved during a baseline screen after a concussion, and that single leg balance performance may be impaired when baseline assessments are done within 3 months of the concussive event. We do not believe that these changes are clinically meaningful, given the small changes we observed, and the non-specificity and lack of precision of the sub-modes.

With respect to symptoms, given the small reduction in symptom endorsement, and that symptoms endorsed during SCAT5 screens are assessed relative to a player’s trait symptoms,17 our finding does not meaningfully alter the practice of annual baseline tests, or necessitate a policy change for the management of previously concussed players during baseline or diagnostic screens. Clinicians should however be mindful of the possibility that previously concussed players may deliberately report fewer symptoms, and investigate the basis of any reductions to symptom number or severity. This may take the form of repeating the screen on different day, or a more in-depth consultation with the player regarding symptoms.

The improvements in cognitive sub-modes were small (0.23 improvement in Digits backward, for example, table 2), and unlikely to influence clinical decisions and actions at the time of suspected head injury events. This is particularly true given that it has recently been shown that doctors frequently apply clinical judgement to abnormal sub-modes during return to play and diagnostic screens.14 As such, the small changes we document in symptoms, digits backwards scores and tandem gait time do not necessitate a change in baseline screening policy, or use of baseline or diagnostic screens.

The findings are however important to guide clinicians in their assessment of changes in sub-mode performance. During baseline screens, clinicians should be mindful that reduced symptoms or improved cognitive scores may indicate either learning effects, greater motivation or an unwillingness to disclose symptoms that could prejudice a player’s selection for teams. In this regard, changes to baseline performance should be assessed and interpreted with caution when a player has experienced a previous concussion. This may involve follow-up investigation and the repetition of the sub-modes in questions, using different word or number sequences, and in-depth consultation on symptoms where relevant.

Further research is required to explore whether multiple testing, consisting of three or more SCAT assessments, produces learning effects and to what degree sub-mode performance may change even in the absence of concussions.