Validity of the 2014 traumatic encephalopathy syndrome criteria for CTE pathology.

INTRODUCTION: Validity of the 2014 traumatic encephalopathy syndrome (TES) criteria, proposed to diagnose chronic traumatic encephalopathy (CTE) in life, has not been assessed.
METHODS: A total of 336 consecutive brain donors exposed to repetitive head impacts from contact sports, military service, and/or physical violence were included. Blinded to clinical information, neuropathologists applied National Institute on Neurological Disorders and Stroke/National Institute of Biomedical Imaging and Bioengineering CTE criteria. Blinded to neuropathological information, clinicians interviewed informants and reviewed medical records. An expert panel adjudicated TES diagnoses.
RESULTS: A total of 309 donors were diagnosed with TES; 244 donors had CTE pathology. TES criteria demonstrated sensitivity and specificity of 0.97 and 0.21, respectively. Cognitive (odds ratio [OR] = 3.6; 95% confidence interval [CI]: 1.2-5.1), but not mood/behavior or motor symptoms, were significantly associated with CTE pathology. Having Alzheimer's disease (AD) pathology was significantly associated with reduced TES accuracy (OR = 0.27; 95% CI: 0.12-0.59). DISCUSSION: TES criteria provided good evidence to rule out, but limited evidence to rule in, CTE pathology. Requiring cognitive symptoms in revised criteria and using AD biomarkers may improve CTE pathology prediction.

BACKGROUND

Chronic traumatic encephalopathy (CTE) is a neurodegenerative disease associated with exposure to repetitive head impacts (RHI). Most CTE cases have been diagnosed in former amateur and professional contact and collision sports (CCS) athletes, but CTE has also been diagnosed in military veterans with combat exposure and others who have suffered frequent head impacts. , , , , CTE only can be definitively diagnosed by neuropathologic examination. A National Institute on Neurological Disorders and Stroke (NINDS)/National Institute of Biomedical Imaging and Bioengineering (NIBIB) panel defined the pathognomonic lesion of CTE as the perivascular accumulation of hyperphosphorylated tau (p‐tau) in neurons and astrocytes in an irregular pattern, most prominent at the depths of the cortical sulci. The panel concluded that CTE is unique and can be reliably distinguished from other neurodegenerative diseases, including Alzheimer's disease (AD) and frontotemporal lobar degeneration (FTLD). Clinically, impulsivity, explosivity, depression, memory impairment, and executive dysfunction have been reported to occur in CTE. ,

In 2014, based on a comprehensive literature review of symptoms of autopsy‐confirmed CTE cases, research diagnostic criteria for traumatic encephalopathy syndrome (TES) were proposed to diagnose CTE in life (Table 1). The authors indicated that these criteria were only intended for clinical research settings. TES was conceptualized as a broad, umbrella term meant to describe the clinical presentation of CTE and other long‐term consequences of RHI. To meet these criteria for TES, there must be sufficient exposure to RHI, including from CCS, military service, or physical violence. There must also be at least 1 year of impairment in cognition (including impairment in memory, executive function, or attention) or symptoms of mood (e.g., depression) or behavior (e.g., explosivity). At least two supportive features (documented decline, delayed symptom onset after RHI exposure, impulsivity, anxiety, apathy, paranoia, suicidality, headache, or parkinsonism) must also be present. Last, CTE likelihood is designated (probable, possible, and unlikely) based on an algorithm that incorporates presence of a progressive course, whether criteria for another disorder are satisfied more consistently, and a positive proposed “potential” biomarker. Although there are no established biomarkers for CTE pathology, fluid and imaging biomarkers were included to meet criteria for probable CTE with an eye for the future. Additionally, several of the proposed biomarkers were included to rule out AD pathology rather than to identify CTE pathology.

TABLE 1

Traumatic encephalopathy syndrome (TES) criteria

A. General criteria
All five criterion (1–5) must be met for diagnosis
1. History of multiple impacts
Types of injuries	Concussion or mTBI. If no other RHI, then minimum of four.
	Moderate/severe TBI. If no other RHI, then minimum of two.
	Sub‐concussive trauma.
Source of RHI exposures	Contact sports. Minimum of 6 years.
	Military service.
	Other RHI exposure including physical violence (i.e., intimate partner violence or childhood abuse).
2. Other neurological disorder that likely accounts for all clinical features
Exclude if	A single TBI.
	Or persistent PCS.
Can be present	Substance abuse.
	Post‐traumatic stress disorder.
	Mood/anxiety disorders.
	Other neurodegenerative diseases.
3. Clinical features must be present for a minimum of 12 months
4. “Core clinical features”
At least one must be present	Cognitive. Difficulties identified by standardized mental status or cognitive neuropsychological test at least 1.5 SD below normative mean. a
	Behavioral. Described as explosive, short fuse, out of control, physically and/or verbally violent. Or intermittent explosive disorder.
	Mood. Feeling overly sad, depressed, or hopeless. Or diagnosis of major depressive disorder or persistent depressive disorder.
5. “Supportive features”
At least two must be present	Documented decline (1 year), delayed onset, impulsivity, anxiety, apathy, paranoia, suicidality, headache, and motor.

Abbreviations: CSF, cerebrospinal fluid; CTE, chronic traumatic encephalopathy; mTBI, mild traumatic brain injury; PCS, post‐concussion syndrome; PET, positron emission tomography; p‐tau, phospho‐tau; RHI, repetitive head impacts; SD, standard deviation; TBI, traumatic brain injury; TES, traumatic encephalopathy syndrome.

Criteria were modified for the current study because the majority of donors did not have neuropsychological testing during life. Instead, if neuropsychological testing was not available or was not performed close to death, subjective report of memory, executive function, or attentional impairment, together with clinician judgment, was sufficient to meet the cognitive criteria.

Source: Table adapted from Mariani et al. (2020).

To date, the inter‐rater reliability and diagnostic validity of the 2014 TES criteria have not been assessed. As part of the Understanding Neurological Injury and Traumatic Encephalopathy (UNITE) Study, we have applied these criteria retrospectively to more than 330 brain donors with RHI exposure recruited after publication of the TES criteria. Comprehensive clinical histories were obtained from next‐of‐kin (NOK) of the deceased and medical records were reviewed. All cases were presented to an expert clinical panel and TES consensus diagnoses were adjudicated. Here, we report on the reliability and validity of the 2014 TES criteria using CTE pathology as the gold standard. We also investigate individual components of the criteria to identify the best predictors of CTE pathology and whether the presence of other pathologies was associated with a reduction in accuracy.

METHODS

Donor recruitment and selection into the study

Consecutive recruitment into the Veterans Affairs (VA)–Boston University (BU)–Concussion Legacy Foundation (CLF) Brain Bank occurred between 2014 and 2019, beginning after publication of the 2014 TES research diagnostic criteria. Data collection was planned before the clinical and neuropathological assessments were performed and was summarized previously. A recruitment goal of 300 donors was specified a priori based on previous donation rates. Recruitment occurred in the following ways: NOK contacted the brain bank near the time of death (n = 274 [85.1%]), donors joined the BU Brain Donation Registry in life (n = 10 [3.1%]), a CLF representative contacted NOK (n = 11 [3.2%]), or a medical examiner contacted the brain bank (n = 26 [7.7%]). To be eligible for brain bank donation, donors needed to have a history of RHI exposure from CCS with at least 2 years of college‐level play, military service, or physical violence (i.e., intimate partner violence or child abuse), regardless of whether symptoms manifested during life. Inclusion criteria for CCS athletes who died before the age 35 were relaxed, allowing for any level of organized CCS play, as young donors with RHI exposure are of high research interest. Potential donors were excluded if the post mortem interval exceeded 72 hours or if less than one brain hemisphere was available for donation. Figure S1 in supporting information shows a Standards for the Reporting of Diagnostic Accuracy Studies (STARD) flow diagram of included and excluded donors. Table S1 in supporting information shows a STARD checklist. Donors’ NOK provided written consent for brain donation, as well as permission to publish de‐identified results, before being made aware of the results. No NOK withdrew consent for participation after the pathologic studies were performed or reported. Institutional review board approval was obtained through BU Medical Campus and Bedford VA Hospital.

Neuropathological assessment

Neuropathologists (VEA, BRH, TDS, ACM) were blinded to the donors’ RHI exposure, clinical history, and brain bank membership (as multiple brain banks exist at BU). Neuropathological processing and gross and microscopic examination followed previously established methods. , , Staining included Luxol fast blue, hematoxylin and eosin, Bielschowsky silver, p‐tau (AT8; pSer202, pThr205), α‐synuclein, amyloid beta (Aβ), and phosphorylated transactive response DNA binding protein 43 kDa. CTE was diagnosed using NINDS/NIBIB neuropathological criteria. Two donors had poor AT8 staining and were excluded from analyses because CTE could not be reliably diagnosed. As suggested by the CTE criteria, when p‐tau pathology was suggestive of CTE, but did not meet formal criteria, additional sampling was performed prior to unblinding. In these cases, large coronal slabs of the cerebral hemispheres were also cut at 50 μm on a sledge microtome and stained as free‐floating sections using AT8 or CP13 (pSer202). Donors diagnosed with CTE were assigned a CTE stage (I to IV, increasing with severity) using validated criteria. , Diagnosis and staging of CTE were agreed upon by consensus of all four neuropathologists. Well‐established neuropathological criteria were used to diagnose additional neurodegenerative diseases, including AD, Lewy body disease (LBD), and FTLD. , , , , , , , , , As with CTE, when pathology was suggestive of these diagnoses, but formal criteria were not met, additional sampling was performed as suggested in the criteria and as is common practice in neuropathology. The neuropathological examination also included an assessment of a range of cerebrovascular pathologies, including atherosclerosis and arteriolosclerosis (scored on a 0 to 3 scale, 3 most severe), and gross and microscopic infarcts (scored as counts). A donor was considered to have cerebrovascular disease if atherosclerosis or arteriolosclerosis were scored as 2 or 3 or if at least one micro‐ or macro‐infarct was present.

HIGHLIGHTS

2014 traumatic encephalopathy syndrome (TES) criteria demonstrated good reliability.

TES criteria demonstrated high sensitivity, but low specificity, for chronic traumatic encephalopathy (CTE) pathology.

Cognitive, but not mood/behavior or motor symptoms, predicted CTE pathology.

Having Alzheimer's disease pathology was significantly associated with reduced TES accuracy.

Results informed revised TES criteria recently published by a National Institute on Neurological Disorders and Stroke workgroup.

RESEARCH IN CONTEXT

Systematic review: A PubMed literature review showed that the inter‐rater reliability and diagnostic validity of proposed criteria to diagnose chronic traumatic encephalopathy (CTE) in life have never been assessed using gold‐standard neuropathology.

Interpretation: In this clinicopathological study of 336 brain donors with repetitive head impact exposure from contact sports, military service, and physical violence, the 2014 traumatic encephalopathy syndrome (TES) research diagnostic criteria demonstrated good reliability and provided moderate to strong evidence to rule out, but limited evidence to rule in, CTE pathology. When individual TES criteria components were assessed, cognitive symptoms, but not mood/behavior or motor symptoms, were significantly associated with CTE pathology. Having Alzheimer's disease (AD) pathology was significantly associated with reduced TES accuracy.

Future directions: These results inform revised TES criteria proposed by a multi‐disciplinary National Institute on Neurological Disorders and Stroke workgroup. Requiring cognitive symptoms in revised criteria and using AD biomarkers may improve CTE pathology prediction.

Collection of athletic, military, TBI, and clinical data

Retrospective data collection from informants and medical record review has been described previously. Informants for seven donors withdrew from the study and did not participate in retrospective data collection. Clinicians were blinded to the neuropathological examination and conclusions. Informants were interviewed before receiving the results of the neuropathological examination.

Online questionnaires queried the donor's demographics, athletic history (type of sports played, level, position, age of first exposure, and duration), and military history (branch, location of service, and duration of combat exposure) using the BU RHI Exposure Assessment. Telephone interviews with the informants were performed by a doctoral‐level “lead” clinician (JM, MLA, DHD, TS, BD) with expertise in neurodegenerative disorders and chronic, long‐term effects of traumatic brain injury (TBI) and RHI exposure. The lead clinician asked semi‐structured questions about cause of death and medical, neurological, and psychiatric histories. History of mild to severe TBI was assessed using the Ohio State University TBI Identification Method short form and two questionnaires adapted from published studies that address military‐related head injuries and concussions. , Using an unstructured format similar to a history obtained in a memory disorders clinic, the lead clinician obtained a precise chronology of symptoms related to cognition, behavior/mood, and daily function. Motor functioning, sleep, headaches, substance use, and family history of mental illness and dementia were queried in the same manner. When the interview was completed, the lead clinician answered several summary questions adapted from the National Alzheimer's Coordinating Center Clinician Judgment of Symptoms form about predominant symptoms (cognitive, mood, behavior, and motor), symptom onset, and disease progression.

With permission from each donor's legal NOK or legally authorized representative, medical records were requested from health‐care providers, including original brain imaging (rather than only the report). If available, the lead clinician reviewed all records related to psychiatric and neurological care, neuropsychological evaluations, brain imaging, relevant medical history, and medications.

When the phone interview and medical record review were completed, the lead clinician qualitatively summarized the donors’ clinical history (e.g., presence and course of cognitive, mood, behavior, and motor symptoms; functional independence; and cause of death) into a narrative summary to be presented at the clinicopathological conference (CPC).

Clinicopathological conference

Clinical consensus methodology is based on recommendations from Bertens et al. A panel of doctoral‐level clinicians composed of neuropsychologists, neurologists, psychiatrists, physiatrists, and neurosurgeons who specialize in neurodegenerative disease and TBI convened at twice‐monthly CPCs. Like the lead clinician, all panel clinicians were blinded to neuropathological data until voting was completed. At least three and as many as seven panel members were present for each CPC. For each case, the prepared clinical summary was distributed to the panel and presented by the lead clinician. In addition to the disease course, the summary included age and cause of death; a subjective assessment of informant reliability (0 = unreliable, 1 = somewhat reliable, 2 = very reliable); prior RHI history; past medical, educational, and occupational history; living situation prior to death; substance‐use history; family history; and salient features from medical records. When available, this included neuropsychological testing, neuroimaging (including a read from a behavioral neurologist if original images were available), cerebrospinal fluid biomarkers, diagnoses made during life, and medications prescribed. At the conclusion of the clinical presentation and prior to any discussion, each panel member voted independently, whether modified criteria for TES were met, including TES subtype and the likelihood of underlying CTE (probable, possible, or unlikely; Table 1C). The 2014 research diagnostic criteria for TES were modified because the TES cognitive subtype requires impairment on neuropsychological testing, but the majority of donors did not have testing during life. Instead, if neuropsychological testing was not available or was not performed close to death, subjective report of memory, executive function, or attentional impairment, together with clinician judgment, was sufficient to meet TES cognitive subtype criteria. The clinicians also recorded a primary clinical diagnosis, and if appropriate, contributing clinical diagnoses, independent of the TES criteria. Next, the clinical panel discussed the case, including questioning the lead clinician about specific details, with the goal of reaching a consensus diagnosis, using the identical format as the previous independent voting. Consensus required a majority of the clinicians present to agree on the diagnosis. When a consensus diagnosis was reached, panel members individually voted again, informed by the discussion, as a means to report dissent from the consensus diagnosis. Subsequently, a neuropathologist presented the pathological diagnoses, including gross and microscopic images.

Statistical analyses

For all analyses, we were interested in the component of the 2014 research diagnostic criteria for TES that indicates the likelihood of underlying CTE pathology. Therefore, all analyses used TES with possible or probable CTE (Table 1C) to indicate the presence of a clinical diagnosis. Because TES with probable CTE requires biomarker data that still have not been sufficiently validated to use in clinical decision making and were only available for a limited subset of donors, we considered TES with possible or probable CTE together, indicated as “TES‐CTE” subsequently.

Using the presence of CTE pathology based on the NINDS/NIBIB neuropathological criteria as the gold standard, we calculated measures of TES‐CTE validity (sensitivity: frequency of the clinical diagnosis among donors with CTE pathology, specificity: frequency of the absence of the clinical diagnosis among donors without CTE pathology, positive likelihood ratio [LR+]: ratio of the frequency of a positive TES‐CTE clinical diagnosis among donors with CTE pathology to the frequency of a positive TES‐CTE clinical diagnosis among donors without CTE pathology, and negative likelihood ratio [LR−]: ratio of the frequency of a negative TES‐CTE clinical diagnosis among donors with CTE pathology to the frequency of a negative TES‐CTE clinical diagnosis among donors without CTE pathology) based on pre‐consensus individual diagnoses and on consensus diagnoses. We also calculated inter‐rater reliability between clinicians both pre‐ and post‐consensus using intraclass correlations (ICC) estimated by a generalized linear mixed effects models with a logit link function, which provides added flexibility for varying identity and number of raters.

Because dementia is typically stratified by early versus late onset and clinical syndromes vary with age, we also ran models stratified by age 60 years. Because the TES criteria may perform differently for varying levels of CTE pathology, we re‐calculated the measures of validity for three different levels of pathology: (1) CTE stage ≥ II, (2) CTE stage ≥ III, and (3) CTE stage IV only. To compare performance of the TES criteria with other neurodegenerative diagnostic criteria, we also calculated measures of validity of clinical diagnostic criteria for AD dementia using modified 2011 National Institute on Aging–Alzheimer's Association criteria, and Parkinson's disease dementia (PDD) using modified 2005 International Parkinson and Movement Disorders Society recommendations (PDD), and dementia with Lewy bodies (DLB) using modified 2005 McKeith criteria (DLB). PDD and DLB were grouped together due to small sample size and because they are related disorders.

Next, we investigated which individual clinical components of the TES criteria predicted CTE pathology. These analyses were limited to donors who met the TES RHI exposure criteria (Table 1A.1.) because, in practice, these clinical features would only be considered if the RHI exposure criteria were first met. With the presence of CTE pathology as the outcome, we ran separate (due to multicollinearity) logistic regression models with the following predictors: presence of cognitive symptoms (in the domains of memory, executive function, and/or attention), presence of mood/behavior symptoms (either depressive or explosive symptoms), presence of at least two parkinsonian motor symptoms, and presence of symptoms for at least 12 months. Even though a progressive course and presence of at least two supportive features (see list in Table 1A.5) are components of the TES criteria, we did not include them in models because we were underpowered to detect associations as most informants reported a progressive course (n = 314 [95.7%]) and at least two supportive features (n = 323; 98.5%). Models were adjusted for race and age ≥ 60 years. Similar to earlier analyses, we also ran models with an age ≥ 60 years interaction term.

Based on results from the above regression analyses, in post hoc analyses, we re‐categorized donors such that cognitive symptoms would be required to meet TES‐CTE criteria, that is, only those who received a TES‐CTE cognitive or mixed subtype diagnosis were considered to have TES‐CTE. We re‐calculated the sensitivity, specificity, LR+, and LR− of TES‐CTE consensus diagnoses, again using the presence of CTE pathology as the gold standard.

Next, we tested whether having other pathologies may be associated with accuracy (frequency with which the clinical diagnosis matched the CTE pathological diagnosis among all donors) of a TES‐CTE diagnosis. We similarly stratified these analyses by age 60 years as most, but not all, other pathologies more commonly occur later in life. We ran binary logistic regression models with the following predictors: presence of AD, FTLD, cerebrovascular disease, and LBD pathologies. Models were adjusted for race.

Based on results from the above regression analyses, in post hoc analyses, we re‐categorized donors such that the presence of AD pathology would exclude a TES‐CTE diagnosis. We re‐calculated the measures of validity, again using the presence of CTE pathology as the gold standard. Last, we re‐categorized donors such that both cognitive impairment and the absence of AD pathology were required to receive a TES‐CTE diagnosis and re‐calculated the measures of validity.

Data collection was conducted using REDCap 8.5.1. Statistical analyses were conducted using R 3.4.2 and SAS 9.4.

RESULTS

The study included 244 donors with CTE pathology (72.6%) and 92 without CTE pathology (27.4%). Of the 244 donors with CTE, 46 (18.9%) had stage I CTE, 46 (18.9%) had stage II, 82 (33.7%) had stage III, and 69 (28.4%) had stage IV. A total of 204 (61.8%) donors had informants rated as very reliable, 123 (37.3%) had informants rated as somewhat reliable, and 3 (0.9%) had informants rated as unreliable. Table 2 shows demographic, head trauma‐related, and other neuropathological characteristics of donors stratified by CTE neuropathological status. The mean number of consensus panel members was 5.1 (0.9 standard deviation [SD]). A total of 309 donors received a TES‐CTE consensus diagnosis and 27 donors did not. Table S2 in supporting information shows demographic, head trauma‐related, and neuropathological characteristics of donors stratified by TES‐CTE status. Table 3A shows a 2 × 2 table of TES‐CTE consensus status by CTE pathological status. Table 3B shows the corresponding sensitivity, specificity, LR+, and LR− obtained pre‐consensus and at consensus. It also shows the inter‐rater reliability obtained pre‐ and post‐consensus. In general, inter‐rater reliability (ICC: 0.75 pre‐consensus) was good. Sensitivity (0.94 pre‐consensus and 0.97 consensus) was very high at the expense of specificity (0.21 pre‐consensus and 0.21 consensus), providing moderate to strong evidence to rule out CTE pathology (LR−: 0.29 pre‐consensus and 0.14 consensus), but limited evidence to rule in CTE pathology (LR+: 1.19 pre‐consensus and 1.23 consensus). Table S3 in supporting information shows these same data stratified by age 60 years. In general, the criteria performed better in the older group. Table S4 in supporting information shows the performance of the criteria after recategorizing donors’ CTE pathological status based on stage (i.e., [1] CTE stage ≥ II, [2] CTE stage≥ III, and [3] CTE stage IV only). With each increasing stage threshold, the criteria become increasingly sensitive with further reduction in specificity. Tables 3C and 3D describe the 73 false positive diagnoses (i.e., those with a TES‐CTE diagnosis, but without a CTE neuropathology diagnosis) and Table 3E describes the 8 false negative diagnoses (i.e., those without a TES‐CTE diagnosis, but with a CTE neuropathological diagnosis). A total of 65 (89%) of the false positives had a different neurodegenerative disease and/or cerebrovascular disease. The remaining false positives had a variety of pathologies reported to be associated with substance use (Purkinje cell loss), TBI (hemosiderin‐laden macrophages around white matter vessels, white matter degeneration), and aging (primary age‐related tauopathy, age‐related tau astrogliopathy). One false positive had features suggestive of CTE, but did not have a pathognomonic lesion. All of the false negatives had low severity CTE pathology (stage I/II). Figure 1 shows representative examples of neuropathology from donors with false positive and false negative diagnoses. Table S5 in supporting information shows the frequency of consensus TES subtypes, symptom timeline, individual clinical symptoms, other diagnoses made at the consensus meetings, and available objective clinical data (i.e., neuropsychological test scores, brain imaging, and amyloid and/or tau biomarker data) stratified by CTE pathological status.

TABLE 2

Demographic, head trauma‐related, and other neuropathological characteristics by CTE neuropathological status

	CTE pathology absent (n = 92)	CTE pathology present (n = 244)	Total (n = 336)
Demographics
Mean age (SD)	50.6 (22.7)	63.6 (18.6)	59.8 (20.7)
Black race (%)	8 (8.7)	38 (15.6)	46 (13.7)
Hispanic ethnicity (%)	3 (3.3)	5 (2.0)	8 (2.4)
Women (%)	3 (3.3)	0	3 (0.9)
Mean education in years (SD)	14.9 (2.6)	16.2 (1.8)	15.8 (2.2)
Head trauma related
Contact sports (%) a	74 (80.4)	238 (97.5)	312 (92.9)
Football (%)	62 (67.4)	217 (88.9)	279 (83.0)
Professional highest level (%)	8 (8.7)	102 (41.8)	110 (32.7)
College/semi‐professional highest level (%)	24 (26.1)	103 (42.2)	127 (37.8)
High school/youth highest level (%)	30 (32.6)	14 (5.7)	44 (13.1)
Boxing (%)	5 (5.4)	19 (7.8)	24 (7.1)
Professional highest level (%)	0	3 (1.2)	3 (0.9)
Amateur highest level (%)	4 (4.3)	16 (6.6)	20 (6.0)
Mixed martial arts (%)	0	3 (1.2)	3 (0.9)
Ice hockey (%)	7 (7.6)	20 (8.2)	27 (8.0)
Professional highest level (%)	1 (1.1)	8 (3.3)	9 (2.7)
Semi‐professional highest level (%)	3 (3.3)	3 (1.2)	6 (1.8)
College/juniors highest level (%)	0	1 (0.4)	1 (0.3)
High school/youth highest level (%)	3 (3.3)	7 (2.9)	10 (3.0)
Rugby (%)	4 (4.3)	12 (4.9)	16 (4.8)
Amateur wrestling (%)	10 (10.9)	20 (8.2)	30 (8.9)
Soccer (%)	11 (12.0)	17 (7.0)	28 (8.3)
Professional highest level (%)	0	0	0
College/semi‐professional highest level (%)	2 (2.2)	2 (0.8)	4 (1.2)
High school/youth highest level (%)	8 (8.7)	13 (5.3)	21 (6.3)
Lacrosse (%)	3 (3.3)	7 (2.9)	10 (3.0)
Other (%)	1 (1.1)	3 (1.2)	4 (1.2)
Military service (%) b	25 (27.2)	64 (26.2)	89 (26.5)
With combat (%)	7 (7.6)	5 (2.0)	12 (3.6)
Physical violence c	5 (5.4)	8 (3.3)	13 (3.9)
Four or more concussions (%)	70 (76.1)	203 (83.2)	273 (81.3)
Median concussion count (IQR)	10 (36)	40 (122)	25 (92)
Moderate to severe TBI (%)	10 (10.9)	11 (4.5)	21 (6.3)
Two or more moderate to severe TBIs (%)	3 (3.3)	0 (0)	3 (0.9)
Other neuropathologies (%)	47 (51.1)	171 (71.1)	218 (64.9)
AD pathology (%)	18 (19.6)	38 (15.6)	56 (16.7)
Mean CERAD neuritic plaque score (SD)	0.5 (1.0)	0.6 (0.9)	0.6 (0.9)
Mean Braak neurofibrillary tangle stage (SD)	1.7 (2.2)	2.7 (1.9)	2.4 (2.0)
Lewy body pathology (%)	15 (16.3)	46 (18.9)	61 (18.2)
Brainstem predominant (%)	6 (6.5)	21 (8.6)	27 (8)
Limbic/neocortical (%)	9 (9.8)	25 (10.2)	34 (10.1)
Frontemporal lobar degeneration pathology (%)	12 (13.0)	30 (12.3)	42 (12.5)
Tau pathology (%)	9 (9.8)	13 (5.3)	22 (6.5)
TDP‐43 pathology (%)	3 (3.3)	18 (7.4)	21 (6.3)
Cerebrovascular pathology (%)	41 (44.6)	157 (64.3)	198 (58.9)

A total of 89 (28.5%) donors played more than one contact sport.

A total of 77 (86.5%) donors served in the military and played contact sports.

Either in the form of intimate partner violence or child abuse.

Abbreviations: AD, Alzheimer's disease; CERAD, Consortium to Establish a Registry for Alzheimer's Disease; CTE, chronic traumatic encephalopathy; IQR, interquartile range; SD, standard deviation; TBI, traumatic brain injury.

TABLE 3

TES‐CTE diagnoses by CTE pathology status: frequencies, diagnostic validity, inter‐rater reliability, and characteristics of donors inaccurately diagnosed

A. TES‐CTE consensus diagnosis by CTE pathology frequencies
	CTE pathological diagnosis
TES‐CTE clinical consensus diagnosis		Yes	No	Total
	Yes	236 (70.2%)	73 (21.7%)	309 (92.0%)
	No	8 (2.4%)	19 (5.7%)	27 (8.0%)
	Total	244 (72.6%)	92 (27.4%)	336

Notes: Pre‐consensus refers to individual diagnoses made by consensus panel members prior to discussion. Consensus refers to the group consensus diagnoses after discussion, except for inter‐rater reliability for which consensus refers to individual diagnoses made after discussion. Sensitivity: Among donors with a CTE neuropathological diagnosis, the frequency with which a TES‐CTE clinical diagnosis was made. Specificity: Among donors without a CTE neuropathological diagnosis, the frequency with which a TES‐CTE clinical diagnosis was not made. Positive likelihood ratio: Ratio of the frequency with which a TES‐CTE clinical diagnosis was made among donors with a CTE neuropathological diagnosis to the frequency with which a TES‐CTE clinical diagnosis was made among donors without a CTE neuropathological diagnosis. Negative likelihood ratio: Ratio of the frequency with which a TES‐CTE clinical diagnosis was not made among donors with a CTE neuropathological diagnosis to the frequency with which a TES‐CTE clinical diagnosis was not made among donors without a CTE neuropathological diagnosis. Inter‐rater reliability: A measure of agreement (range: 0 to 1) among consensus panel members that accounts for varying identity and number of raters.

Abbreviations: ALS, amyotrophic lateral sclerosis; ARTAG, age‐related tau astrogliopathy; ASD, autism spectrum disorder; CI, confidence interval; CTE, chronic traumatic encephalopathy; IED, intermittent explosive disorder; MVA, motor vehicle accident; OCD, obsessive compulsive disorder; PART, primary age‐related tauopathy; PPCS, persistent post‐concussion symptoms; PTSD, post‐traumatic stress disorder; TES, traumatic encephalopathy syndrome.

FIGURE 1

Representative images of common pathologies found in brain donors with discrepant clinical and neuropathological diagnoses. All images are 10‐μm paraffin‐embedded tissue sections. Calibration bars indicate 100 μm. A, Perivascular chronic traumatic encephalopathy (CTE) lesions in the dorsolateral frontal (DLF) cortex: Immunostaining is with mouse monoclonal antibody for phosphorylated tau (p‐tau; AT8; Pierce Endogen) and counterstaining is with Luxol fast blue‐hematoxylin and eosin (LHE). Positive p‐tau immunostaining appears dark red. Neurofibrillary tangles (NFTs) and dot‐like and threadlike neurites encircle blood vessels (arrows). B, Lewy body inclusion in the dorsolateral frontal lobe: Immunostaining is with rabbit polyclonal antibody for alpha‐synuclein (Chemicon) and counterstaining is with LHE. Positive intraneuronal alpha‐synuclein immunostaining appears dark red. C, Pathological hallmarks of Alzheimer's disease (AD) in the DLF cortex: Staining is with Bielschowsky silver. NFTs (blue arrow), neuropil threads (green arrow), and neuritic plaques (black arrow) are densely distributed in the neuropil. D, AD in the CA1 region of the hippocampus: AT8‐immunostaining. P‐tau‐positive NFTs (blue arrow), neurites, and neuritic plaques (black arrow) are densely distributed throughout CA1. E, Microinfarct in septal cortex: Staining is with LHE. There is pallor, cystic tissue loss, and gliosis. F, Arteriolosclerosis in the deep white matter: LHE staining. A small arteriole shows hyalinized thickening of the vessel wall (black asterisk)

Table S6 in supporting information and Figure 2 show the association between each clinical component of the TES‐CTE criteria and CTE pathological status modeled in a logistic regression, adjusted for age > 60 years and race. Having cognitive symptoms and having clinical features present for ≥12 months were each significantly associated with having CTE pathology. On average, donors with cognitive symptoms had 3.6× the odds of having CTE pathology (P < .001) compared to donors without cognitive symptoms. On average, donors with core features present for ≥12 months had 3.5× the odds of having CTE pathology (P = .03) compared to donors with core features for <12 months. Having behavior/mood symptoms or motor symptoms was not significantly associated with CTE pathology. There were no significant interactions observed between clinical features and age ≥ 60 years.

FIGURE 2

Association of traumatic encephalopathy syndrome (TES) criteria components on odds of chronic traumatic encephalopathy (CTE) pathology. The presence of cognitive symptoms was significantly associated with 3.6× increased odds of CTE pathology and the presence of features for ≥12 months was significantly associated with 3.5× increased odds of CTE pathology. There were not significant associations for behavior/mood symptoms or motor symptoms with CTE pathology. Models were adjusted for age ≥ 60 and race. Red bars indicate significant associations at the P = .05 level

In post hoc analyses, when we re‐categorized donors to require cognitive symptoms to meet TES‐CTE criteria, sensitivity remained high (changing from 0.97 to 0.90) and specificity increased markedly (0.21 to 0.48) resulting in increases in LR+ (1.23 to 1.73) and LR− (0.14 to 0.21; Table S7 in supporting information).

When we assessed how the TES criteria compared to more established neurodegenerative clinical criteria applied in the same setting, we found that the sensitivity and specificity for AD and DLB/PD were less sensitive and more specific (Table S8 in supporting information). In analyses investigating how other neurodegenerative pathologies were associated with the accuracy of a TES‐CTE diagnosis, the presence of AD pathology was significantly associated with a 3.7× reduction in accuracy of a TES‐CTE diagnosis among older donors (age ≥ 60). Accuracy was not significantly associated with the presence of LBD, FTLD, or cerebrovascular disease pathology among older or younger donors (Table S9 in supporting information, Figure 3). In post hoc analyses, when we re‐categorized donors such that the presence of AD pathology would exclude a TES‐CTE diagnosis, sensitivity declined (from 0.97 to 0.82) and specificity increased (from 0.21 to 0.40) resulting in increases in LR+ (1.23 to 1.36) and LR− (0.14 to 0.46; Table S10 in supporting information). In post hoc analyses, when we re‐categorized donors such that both cognitive impairment and the absence of AD pathology were required to receive a TES‐CTE diagnosis, sensitivity declined (from 0.97 to 0.75) and specificity increased (from 0.21 to 0.61) resulting in increases in LR+ (1.23 to 1.92) and LR− (0.14 to 0.41; Table S11 in supporting information).

FIGURE 3

Association of other pathologies on accuracy of traumatic encephalopathy syndrome‐chronic traumatic encephalopathy (TES‐CTE) diagnosis in donors age ≥ 60. Among donors age > 60, the presence of Alzheimer's disease (AD) pathology was significantly associated with reduced accuracy of the TES‐CTE consensus diagnoses by 3.7× compared to diagnoses made in the absence of AD pathology. There were not significant associations for Lewy body disease, frontotemporal lobar degeneration, or cerebrovascular pathology with accuracy of the TES‐CTE consensus diagnoses. Models were adjusted for race. The red bar indicates significant associations at the P = .05 level

DISCUSSION

In 2014, research diagnostic criteria for TES were proposed for use in clinical research settings to diagnose CTE in life. Diagnosis in life is necessary for counseling patients and developing therapeutics. The current study's goal was to assess the inter‐rater reliability and diagnostic validity of TES criteria using CTE pathology as the gold standard. Among 336 brain donors, recruited after publication of the TES criteria, who had a range of RHI exposure, including from CCS, military service, or physical violence, TES criteria were relatively reliable (ICC: 0.75) and were very sensitive (0.97), but had low specificity (0.21). They provided moderate to strong evidence to rule out (LR− = 0.14), but limited evidence to rule in (LR+ = 1.23) CTE pathology. Most of the false positives had other neurodegenerative or cerebrovascular pathology. All of the false negatives had low severity of CTE pathology (stage I/II). Having cognitive symptoms was significantly associated with having CTE pathology; by augmenting the TES criteria to require cognitive symptoms, specificity improved (0.48) with limited reduction in sensitivity (0.90). Among older donors (age ≥ 60 years), if AD pathology was present, accuracy of a TES diagnosis was markedly reduced.

The 2014 criteria were developed to capture the clinical presentation of CTE as well as other possible long‐term consequences of RHI, including other neurodegenerative diseases. Additional elements were included (progressive course, whether criteria for another disorder were satisfied more consistently, imaging and fluid biomarkers) to indicate the probability of CTE pathology. Importantly, the listed biomarkers in the 2014 criteria were included with an eye for the future, but still have not been sufficiently validated to be used in diagnostic decision making. Because these biomarkers had not been sufficiently validated and were frequently unavailable in our study, we chose to combine possible and probable CTE, which we termed “TES‐CTE.” We found the TES criteria to be very sensitive at the expense of specificity. The authors of the 2014 TES criteria did explain that their goal was to initially prioritize sensitivity and then improve specificity in future, more restrictive versions of the criteria, as is customary for other neurodegenerative diseases. Indeed, early AD and FTD criteria were more sensitive than specific, and specificity improved in later versions when biomarkers were introduced. , As our study demonstrates, even among a group with extensive RHI exposure, AD and PDD/DLB criteria showed higher specificity for their corresponding pathologies than TES‐CTE, albeit with limited biomarker data.

As the TES criteria were much more sensitive than specific, there were far more false positives (i.e., those with a TES diagnosis, but without CTE pathology) than false negatives (i.e., those without a TES diagnosis, but with CTE pathology). Most of the false positives had another neurodegenerative disease (mostly AD, DLB, or FTLD), cerebrovascular pathology, or both. Having AD pathology was significantly associated with reduced odds of an accurate diagnosis among those ≥age 60. Further, when we augmented the criteria in post hoc analyses, excluding donors with AD pathology, specificity increased from 0.21 to 0.40, with an acceptable reduction in sensitivity from 0.95 to 0.82. Nonetheless, CTE pathology frequently occurred among those with AD pathology (n = 36; 68% of donors with AD). Together with previous work, this suggests that having a positive AD biomarker does not preclude the possibility of comorbid CTE pathology, but that clinicians should be less confident that CTE pathology is present in the setting of a positive AD biomarker, even though it is not a marker of CTE pathology. All of the false negatives had low severity of CTE pathology (stage I/II). It is likely that in a subset of donors, low severity of pathology may not manifest as sufficient symptoms to meet TES criteria. Indeed, when we required higher stage CTE to be present, sensitivity rose at the expense of specificity. Multiple factors may contribute to resilience in individuals with CTE pathology including older age of first exposure to RHI and higher job attainment. ,

We examined individual components of the TES criteria to investigate which were most predictive of CTE pathology. We found that having cognitive symptoms was significantly associated with CTE pathology, increasing the odds by 3.6‐fold. Further, mood and behavior symptoms, considered a core component of the criteria, were not significantly associated with CTE pathology. Of note, all brain donors had a history of RHI from CCS, military service, or physical violence and these types of exposures have previously been shown to be associated with chronic mood and behavior symptoms. , That mood and behavior symptoms were not associated with CTE pathology may suggest that they are related to non–tau‐mediated mechanisms like white matter rarefaction. Alternatively, we may not be detecting a true association because of limitations in measuring mood and behavior symptoms using a dichotomous measurement rather than a scale with multiple items that capture symptom severity. In light of the identified association with cognitive symptoms and the lack of association with mood and behavior symptoms, we augmented the criteria in post hoc analyses to require cognitive symptoms rather than requiring cognitive or mood/behavior symptoms. This augmentation increased specificity by an additional 27% without compromising sensitivity. This insight was incorporated into revised TES criteria recently published by an NINDS workgroup composed of multi‐disciplinary experts in the field. Notably, by re‐categorizing donors such that both cognitive impairment and the absence of AD pathology were required to receive a TES‐CTE diagnosis, the specificity further improved to 0.61 with a reduction in sensitivity to 0.75. These values for sensitivity and specificity are on par with AD and related dementias. , , Combining these requirements may be an ideal formulation for “probable CTE” in a future TES iteration.

Study strengths include the large sample size; validation in brain donors that were not used to develop the TES criteria; and a rigorous approach that used blinded clinicians and neuropathologists, and a large multi‐disciplinary consensus panel. Study limitations include the lack of prospective clinical data collection, which would have allowed for uniform neuropsychological, neuropsychiatric, and biomarker data collection. Instead, these data were non‐uniform and inconsistently present. This limitation prompted the use of modified criteria to allow for cognitive complaints rather than formal neuropsychological testing. However, the retrospective approach allowed for the review of a large number of brain donors over a relatively short period of time. Findings from the current study will inform future prospective studies, allowing for improved study design. Additionally, the study relied on retrospective informant report that is subject to recall bias. However, in memory disorders clinics, history‐taking from an informant is crucial, commonplace, and often more valuable than the history obtained from the patient. Further, informants were blinded to neuropathology until they completed the clinical interviews and questionnaires, so knowledge of the diagnosis could not have led to differences in reporting by disease status. Additionally, the VA‐BU‐CLF Brain Bank, like all brain banks, is subject to selection bias. Although inclusion is based on RHI exposure, but not clinical symptoms, NOK may be more likely to donate if concerning symptoms were present, leading to inflated CTE prevalence in the brain bank. Therefore, in this study, we do not emphasize accuracy or positive and negative predictive values, as they are a function of disease prevalence. However, sensitivity and specificity are not a function of disease prevalence and are instead a characteristic of the criteria being used. The sensitivity and specificity are nonetheless susceptible to selection bias if factors associated with selection are differentially related to the TES criteria and the presence of CTE pathology. Additionally, consensus diagnosis was an iterative process with potential for clinician learning from the neuropathology feedback at each consensus conference. However, compared to the first half of the study, the second‐half diagnostic accuracy did not improve and inter‐rater reliability remained very similar. The TES criteria are quite detailed and may not leave much opportunity for diagnostic drift. Donation was rejected for 82 potential donors because they did not meet the RHI inclusion criteria. As the inclusion criteria required substantial RHI exposure, nearly all brain donors met the minimum RHI exposure requirements for a TES diagnosis (n = 328; 97.6%), essentially eliminating this component of the criteria from assessment. The target population for this study is former contact sport athletes, veterans, and those who have experienced physical violence, who are seeking neurological evaluation for cognitive decline and/or mood and behavior symptoms years to decades after these exposures. The generalizability of the findings beyond individuals with this type of substantial RHI exposure is unknown. Future work should assess the criteria across a wider range of exposures. Last, donation was also rejected for 171 potential donors because of insufficient or poor‐quality tissue. Unfortunately, there is limited information on these rejected cases, which would have allowed for an investigation of and potential adjustment for selection bias.

CONCLUSION

Among RHI‐exposed brain donors, the 2014 research diagnostic criteria for TES demonstrated good inter‐rater reliability and high sensitivity, but low specificity for CTE pathology, providing moderate to strong evidence to rule out, but limited evidence to rule in, CTE pathology. When individual TES criteria components were assessed, cognitive symptoms, but not mood/behavior or motor symptoms, were significantly associated with CTE pathology. Having AD pathology was significantly associated with reduced TES accuracy. These results inform revised TES criteria that are concurrently under development by a multi‐disciplinary NINDS workgroup. Having reliable and valid criteria to diagnose CTE in life will improve patient care and accelerate the development of effective therapies.

Supporting Information

Click here for additional data file.