Cumulative Effects of Concussion History on Baseline Computerized Neurocognitive Test Scores: Systematic Review and Meta-analysis.

CONTEXT: It is unclear whether individuals with a history of single or multiple clinically recovered concussions exhibit worse cognitive performance on baseline testing compared with individuals with no concussion history.
OBJECTIVE: To analyze the effects of concussion history on baseline neurocognitive performance using a computerized neurocognitive test. DATA SOURCES: PubMed, CINAHL, and psycINFO were searched in November 2015. The search was supplemented by a hand search of references. STUDY SELECTION: Studies were included if participants completed the Immediate Post-concussion Assessment and Cognitive Test (ImPACT) at baseline (ie, preseason) and if performance was stratified by previous history of single or multiple concussions. STUDY
DESIGN: Systematic review and meta-analysis. LEVEL OF EVIDENCE: Level 2. DATA EXTRACTION: Sample size, demographic characteristics of participants, as well as performance of participants on verbal memory, visual memory, visual-motor processing speed, and reaction time were extracted from each study.
RESULTS: A random-effects pooled meta-analysis revealed that, with the exception of worsened visual memory for those with 1 previous concussion (Hedges g = 0.10), no differences were observed between participants with 1 or multiple concussions compared with participants without previous concussions.
CONCLUSION: With the exception of decreased visual memory based on history of 1 concussion, history of 1 or multiple concussions was not associated with worse baseline cognitive performance.

With an estimated 3.8 million sports- and recreation-related mild traumatic brain injuries (ie, concussions) in the United States every year,[42] concussion continues to be a public health concern. Concussion affects various parts of brain functioning that may result in temporary cognitive changes and symptoms.[5] Because cognitive declines after concussion, as measured by current concussion test batteries, typically resolve or the patient returns to preinjury cognitive performance within days to weeks from injury, the effects of concussion are considered temporary.[10] Because of the growing concern of cumulative, long-term cognitive effects of concussion[21,51] as well as the lack of longitudinal studies to quantify possible neurodegeneration after multiple concussions, population-based differences in computerized neurocognitive tests (CNTs) based on history of clinically recovered concussions have been used to examine for possible cumulative effects of concussions.[4]

Evidence for cumulative cognitive effects after concussion remains inconclusive. While some investigations documented small cognitive effects of multiple clinically recovered concussions,[35,52] others did not support possible cumulative effects of previous concussions.[34] Furthermore, some investigators supported a dose-response relationship that differentiates between the effects of a single concussion versus multiple concussions.[29] A meta-analysis of studies published prior to 2010 examined the residual cognitive effects of multiple concussions but did not examine the effects of a single previous concussion.[4] This meta-analysis included 8 studies reporting on 7 different cognitive domains obtained from multiple cognitive testing batteries.[4] The differences in the psychometric properties of the testing batteries coupled with the small number of reviewed studies (n = 8) assessing multiple cognitive domains (k = 7) may have obscured any actual cognitive differences in selective constructs of neurocognitive function (eg, reaction time, verbal memory, etc).[4]

Previous research suggests that cognitive effects of concussion are construct specific, and therefore, an aggregated effect size among multiple cognitive domains may dilute construct-specific cognitive changes. For instance, although a previous analysis demonstrated no significant aggregated cognitive declines after concussion (d = 0.006), an exploratory analysis revealed lingering effects of concussion on construct-specific domains such as executive function (d = 0.24) and delayed memory (d = 0.16).[4]

The purpose of this study was to review possible cumulative effects of 1 or multiple concussions on construct-specific baseline cognitive performance. To overcome possible influences of multiple testing batteries, the evidence was reviewed pertaining to a single CNT battery that systematically collects the presence and number of previous concussions on baseline testing, allowing for examination of our research question. Additionally, this CNT battery was chosen because it is the most widely used and validated CNT battery in individuals with concussion.[3] The Immediate Post-concussion Assessment and Cognitive Test (ImPACT) has been utilized to quantify the acute effects of concussions at all levels.[14,18,50] For example, ImPACT was used by 90% of athletic trainers in Division I National Collegiate Athletic Association athletics.[18] ImPACT consists of 6 cognitive test modules (design memory, word memory, symbol match, Xs and Os, color match, and 3-letter memory). The 6 modules are utilized to generate 4 composite scores (verbal memory, visual memory, visual-motor processing speed, and reaction time).[36,44,47,59] A number of investigations have reported on the test’s validity and utility in identifying the effects of concussion.[3,8,27,41,45,46,63] However, several investigators continue to be skeptical about the reliability of CNTs, including ImPACT, and potential implications of fair to moderate reliability on the utility of CNTs after concussion.[2,6,53,58]

Methods

Data and Literature Sources

An electronic literature search of studies published between January 1999 and November 2015 was completed. Studies published before 1999 were not included in this search, as the earlier version of ImPACT is no longer in use. The searched databases included PubMed, CINAHL, and psycINFO. The search terms used were the following: ImPACT OR immediate post-concussion assessment and cognitive test OR impact testing OR neurocognitive testing OR neurocognitive OR neuropsychological testing OR neuropsychological AND concussion OR mTBI OR mild traumatic brain injury OR post concussive syndrome OR mild head injury OR closed head injury. The search filters of English-language publications and studies that included human participants were applied. A manual search of the citations of reviewed studies and an electronic search on the ImPACT test website were performed. Review articles, abstracts, case studies, editorials, and gray literature were excluded from the review. Gray literature was excluded because it does not often include the necessary level of detail that allows for thorough examination of methodological and reporting qualities needed for inclusion in this meta-analysis.

Study Selection

Studies were included if participants completed the ImPACT test at baseline (ie, preseason) and if performance was stratified by history of single or multiple concussions. Studies were excluded if they met at least 1 of the following exclusion criteria: (1) ImPACT test modules or subscales were reported instead of composite scores, (2) study utilized version 1.0 of ImPACT, (3) baseline scores were not stratified by previous concussion history, or (4) ImPACT scores were not baseline (eg, studies examining ImPACT performance in patients currently recovering from concussion).

Data Extraction

Two reviewers identified potential studies after an independent review of the titles and abstracts. The same 2 reviewers completed an independent review of potential studies and extracted the data using a piloted Microsoft Excel spreadsheet. Disagreement on the extracted data was resolved by consensus between the 2 reviewers. If disagreement remained, a third reviewer was consulted to resolve the disagreement. Variables recorded included sample size, demographic characteristics of participants, and performance of participants on verbal memory, visual memory, visual-motor processing speed, and reaction time of the ImPACT test.

Assessment of Reporting Quality

The reporting quality of each study was assessed using the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) instrument.[65] The STROBE instrument addresses 22 fundamental aspects of the methods and reporting of observational studies. Each aspect was assigned a numerical value of “1” when explicitly described and a numerical value of “0” if inadequately described or absent. As such, a total score out of 22 was reported, with higher STROBE scores reflecting better reporting quality.[65] Because of the potential disparity that can exist in the analysis based on the specific scale employed,[38] reporting quality scores were not used as weights (ie, moderators) in the pooled analysis.

Statistical Analysis

Assessment of Heterogeneity

Heterogeneity refers the extent of variability between studies. Statistical tests are used to quantify the degree of heterogeneity between studies. In this meta-analysis, heterogeneity was assessed using the Q statistic as a test of the null hypothesis of homogeneity, examined at the null P value of P < 0.10. The I2 index was used to estimate the degree of heterogeneity present across studies when the null hypothesis was rejected at P < 0.10.[32,33] Higgins and Thompson[32] described I2 values in interpretation of magnitude as percentages of 25% (I2 = 25), 50% (I2 = 50), and 75% (I2 = 75), indicating low, medium, and high heterogeneity, respectively. Given that the purpose of this meta-analysis is to generalize the findings to the overall population, we utilized the random-effects model to account for possible interstudy heterogeneity.

Assessment of Publication Bias

Because studies documenting positive findings are more likely to be published compared with studies with negative findings, meta-analyses can be subject to publication bias. In this meta-analysis, publication bias was examined by visual inspection of the funnel plots. For outcomes where funnel plots indicated asymmetry as potential evidence of publication bias, the Egger regression intercept test (beta coefficient, t value, P value) was used against a 1-tailed test (P < 0.05). Statistical evidence of publication bias was further investigated using the nonparametric data augmentation trim and fill method described by Duval and Tweedie.[22]

Mean Differences and Effect Size Calculations

To examine the possible chronic effects of concussion, a pooled random-effects analysis was completed where the ImPACT composite scores for participants with 1 concussion (μˆ1) and for participants with 2 or more concussions (μˆ2) were compared with participants reporting no previous concussions (μˆ0).

Effect size (ES) is a calculation that allows researchers to describe the size of an effect beyond the level of statistical significance.[25] Effect size provides interpretable data that are independent of units of measurement and influence of sample size.[25] The effect size (ie, Hedges g) of concussion history groups was calculated by subtracting the mean score of the individuals with a histiory of concussion (μˆ1 or μˆ2) from the mean scores of individuals without a histiory of concussion (μˆ0). The differences between groups were then divided by the pooled standard deviation. A positive effect size for verbal memory, visual memory, and visual-motor processing speed indicates that participants with no concussion exhibited better baseline scores when compared with participants with concussion history. A positive effect size for reaction time indicates that participants with no concussion history exhibited worse baseline scores when compared with participants with concussion history. All effect sizes were adjusted using the Hedges sample size bias correction before being entered in the analysis.[30] The Hedges g effect sizes were interpreted as small (g ≤ 0.2), medium (g = 0.2-0.5), and large (g ≥ 0.8).[13] All statistical analyses were completed using R statistical software with the Metafor package.[64]

To examine whether the results of the meta-analysis were influenced by 1 study, a leave-1-out estimation sensitivity analysis was completed. During this analysis, 1 study was dropped, and the parameters were then estimated without it.[16]

Results

Identification of Studies

The initial search identified 5968 studies. After removal of duplicates (n = 321), 5647 abstracts were screened by 2 reviewers. The majority of the identified studies included the word “impact,” which is unrelated to the acronym “ImPACT” that is the subject of this review. After reviewing full texts, 17 samples that were obtained from 13 studies were included in the quantitative analysis to examine the effects of 1 and of multiple concussions on baseline ImPACT performance (Figure 1). Participants included 2423 without concussion history, 877 participants with 1 concussion, and 578 participants with multiple concussions. Participants included high school–aged athletes, college-aged athletes, and professional athletes.

Figure 1.

Search process.

Quality Scores and Heterogeneity Assessment

The reporting quality for the studies examined was moderate to high, with STROBE scores ranging from 17 to 21 (Table 1). The heterogeneity observed for the ImPACT scores was low to medium, ranging from 0% to 35.4% for the analysis of 1 concussion and from 20.7% to 53.5% for the analysis of multiple concussions (Table 2).

Table 1.

Characteristics of reviewed studies

	Number of Concussions Reported in the Studies
Study	0	1	2	≥2	≥3
Broglio et al[7]	×	×	×		×
Brooks et al[11]	×	×		×
Covassin et al[19]	×	×	×		×
Covassin et al[17] (male)	×	×	×		×
Covassin et al[17] (female)	×	×	×		×
Covassin et al[20]	×			×
Elbin et al[23]	×			×
Gardner et al[26]	×				×
Iverson et al[34]	×	×	×
Iverson et al[35]	×				×
Solomon et al[61]	×	×		×*
Solomon and Haase[60]	×	×	×		×[†]
Solomon and Kuhn[62]	×	×		×
McKay et al,[49] male: 13-15 y	×	×		×
McKay et al,[49] male: 16-17 y	×	×		×
McKay et al,[49] female: 13-15 y	×	×		×
McKay et al,[49] female: 16-17 y	×	×		×

Reported a group of participants with 2-3 concussions. We subsume this group as “≥2.”

Included the participants with 3 and 4 concussions. We subsume these two groups under the category “≥3.”

Table 2.

Assessment of heterogeneity, publication bias, and mean differences in baseline ImPACT scores among concussion history groups

Test Category	Cochran Q, P Value	I 2	Egger Regression Intercept Test	Parameter	Parameter Estimate (95%CI)	SE	P Value
Differences between participants with no concussion and participants with 1 concussion
Verbal memory	9.8, 0.6	0%	t = 1.45, P = 0.17	τ12	0.00	0.58	—
				µ0 − µ1	0.25 (−0.47, 0.97)	0.37	0.502
Visual memory	9.98, 0.6	0%	t = 0.54, P = 0.60	τ12	0.00	0.93	—
				µ0 − µ1	1.31 (0.39, 2.23)	0.47	0.006
Visual-motor processing speed	17.43, 0.1	9.0%	t = 1.30, P = 0.22	τ12	0.12	0.39	—
				µ0 − µ1	0.49 (−0.09, 1.07)	0.29	0.098
Reaction time	16.81, 0.2	35.4%	t = −0.45, P = 0.66	τ12	0.82	0.99	—
				µ0 − µ1	−0.47 (−1.28, 0.35)	0.42	0.262
Differences between participants with no concussion and participants with 2 or more concussions
Verbal memory	38.83, 0.001	53.5%	t = 0.72, P = 0.48	τ22	5.64	3.41	—
				µ0 − µ2	0.71 (−0.80, 2.22)	0.77	0.356
Visual memory	29.61, 0.02	28.9%	t = −0.21, P = 0.84	τ22	3.63	3.46	—
				µ0 − µ2	−0.07 (−1.62, 1.49)	0.80	0.934
Visual-motor processing speed	20.83, 0.2	20.7%	t = 0.81, P = 0.43	τ22	0.84	1.05	—
				µ0 − µ2	0.17 (−0.70, 1.04)	0.44	0.702
Reaction time	34.16, 0.005	45.4%	t = −1.26, P = 0.23	τ22	2.35	1.72	—
				µ0 − µ2	0.00 (−0.70, 1.04)	0.55	0.997

Publication Bias

The Egger regression intercept test suggested that there were no asymmetries present for the effects of 1 concussion (Table 2). However, visual inspection of funnel plots suggested a possible asymmetry for verbal memory and for visual memory (see Figure A1 in the Appendix, available in the online version of this article). After filling for potential missing studies (see Figure A2 in the Appendix), the effects of 1 concussion did not differ from the effects reported below. For the analysis of multiple concussions, the Egger regression intercept test suggested that there were no asymmetries present (see Figure A3 in the Appendix). Nonetheless, the Duval and Tweedie trim and fill method was used and revealed that all effects remain insignificant, even after adjustment for several possible missing studies (see Figure A4 in the Appendix).

Effects of Concussion History on Baseline ImPACT Performance

With the exception of visual memory, no significant mean differences (Table 2) and no significant effect sizes were observed between patients with a history of 1 concussion (n = 877) when compared to participants with no concussion history (n = 2423) (Figure 2a, 2c, and 2d). Participants with 1 previous concussion demonstrated a significantly worse visual memory score (mean difference, 1.31; P = 0.006) when compared with participants with no concussion history (Table 2). However, closer inspection revealed a small effect size (Hedges g = 0.10; P = 0.012) (Figure 2b).

Figure 2.

Effect of 1 concussion on baseline ImPACT (Immediate Post-concussion Assessment and Cognitive Test) performance.

The comparison between participants with no concussion (n = 2423) when compared with participants with 2 or more concussions (n = 578) revealed no significant mean differences on any of the ImPACT scores (Table 2). Similarly, the analysis of the effect sizes revealed no significant effect sizes for the history of multiple concussions on baseline ImPACT performance (Figure 3).

Figure 3.

Effect of multiple concussions on baseline ImPACT (Immediate Post-concussion Assessment and Cognitive Test) performance.

Discussion

This review indicates that, with the exception of reduced visual memory in participants with 1 concussion, no differences were found in baseline ImPACT performance when comparing those with and those without concussion history. Some researchers have suggested there is a dose-response relationship between concussion history and cognitive performance.[29] However, outside of the likely single spurious association reported here (decreased visual memory between those with and without 1 concussion), the lack of relationship between concussion history and cognitive performance in 7 of the 8 comparisons conducted in this meta-analysis (87.5%) did not support this theory. These findings are comparable to those that report no significant long-term cognitive effects in individuals with history of multiple concussions.[4]

One of many possibilities can explain the lack of a discernible relationship between concussion history and baseline cognitive performance. First, despite the case reports demonstrating neurodegeneration of former athletes,[54,55] no prospective investigation with adequate controls has demonstrated that these declines are present outside of a highly selective sample,[39,48] which suggests that there may be no long-term cognitive consequences to concussion. Second, should long-term cognitive changes exist, they may be subtle and may not be captured using brief computerized neurocognitive tests designed to document larger cognitive effects resulting from a current concussion (eg, ImPACT). A comprehensive and individualized multifaceted cognitive examination that allows for incremental difficulties in cognitive loading may enable investigation of possible subtle long-term concussion cognitive effects.[12] For instance, some individuals with previous concussions exhibited subtle cognitive declines when examined while using tasks that require greater cognitive loading.[21] Third, all participants in the reviewed studies were adolescents or young adults. Therefore, any possible long-term changes may have been masked by cognitive reserve. Last, participants with history of concussion may have taken the test multiple times and may have experienced a learning effect that may offset possible cognitive changes.

Recent evidence demonstrated that effects of concussion are heterogeneous.[15,24,31] Therefore, individuals with multiple concussions may experience lingering effects that are not cognitive in nature.[31,37,57] This explanation is supported by previous investigations demonstrating long-term effects of concussion on functional magnetic resonance imaging (fMRI) that were not captured through cognitive task performance.[37,57] For instance, persistent changes in electrophysiology have been demonstrated in those 3.4 years postinjury when compared with those with no concussion history, while ImPACT scores did not differ between the 2 groups.[9] Utilizing other evaluative measures, such as advanced imaging and biomarkers, may provide additional understanding of possible long-term effects of multiple concussions by identifying changes in brain physiology that may not result in observable cognitive changes.[1,56]

Although this study focused on history of concussion and long-term neurocognitive decline, it should be noted that concussion history may not be the only factor pertinent to long-term neurocognitive health. Future studies should examine the cumulative effects of number and magnitude of subconcussive blows on long-term neurocognitive performance.

This study is not without limitations, including the search strategy that was limited to the ImPACT test. Although the ImPACT test systematically collects injury surveillance on history and on the number of previous concussions, this information was self-reported and subject to recall bias.[40] Similar to a previous meta-analysis examining cognitive declines after concussion,[4] self-reported concussion history may affect the findings of this review. Because participants in all groups considered for this investigation are subject to recall bias, it is unlikely to introduce systematic bias.[4] Although it would have been ideal to examine whether time between prior concussion and subsequent baseline ImPACT testing is a moderator of cumulative effects, time since injury is not reported in the majority of the retrieved studies. Additionally, many of the reviewed studies were retrospective or were completed in various testing environments that may have affected cognitive performance.[43] The ImPACT test scores could be affected by suboptimal performance of test takers, which then results in invalid baseline scores.[28] The ImPACT test has built-in validity indicators to document invalid baselines. However, most studies did not explicitly report that scores were examined to ensure their validity against built-in validity indicators.

Conclusion

With the exception of decreased visual memory based on history of 1 concussion, history of 1 or of multiple concussions was not associated with worsened baseline cognitive performance.