Hippocampal atrophy has limited usefulness as a diagnostic biomarker on the early onset Alzheimer's disease patients: A comparison between visual and quantitative assessment.

NIA-AA diagnostic criteria include volumetric or visual rating measures of hippocampal atrophy (HA) as a diagnostic biomarker of Alzheimer's disease (AD). We aimed to determine its utility as a diagnostic biomarker for early onset Alzheimer's disease (EOAD) by assessing Medial Temporal Atrophy (MTA) and hippocampal volume (HV) determination. MTA score and HV quantified by FreeSurfer were assessed in 140 (aged ≤65) subjects with biomarker supported diagnosis: 38 amnesic (A-EOAD), 20 non-amnesic (NA-EOAD), 30 late onset AD (LOAD), 20 fronto-temporal dementia (FTD) and 32 healthy controls (HC). The results showed that the proportion of MTA ≥ 1.5 was higher on LOAD and FTD than EOAD and HC but none of the MTA thresholds (≥1, ≥1.5 and ≥ 2) showed acceptable diagnostic accuracy. LOAD had lower HV than the other groups. A-EOAD HV was lower than NA-EOAD and HC but equal to FTD. The 6258 mm3 cut-off showed good diagnostic accuracy between A-EOAD and HC. Both tools showed a moderate inverse correlation. In conclusion, MTA has a limited diagnostic utility as an EOAD biomarker as it does not discriminate AD from FTD or HC in initial symptomatic stages. HV may discriminate A-EOAD from HC but not from FTD.

Introduction

Up to 10% of subjects with neurodegenerative dementias have an early onset disease, defined as a clinical onset below 65 years (Garre-Olmo et al., 2010). The most frequent cause of early-onset dementia is Alzheimer's disease (AD), usually presenting with progressive anterograde episodic memory impairment. However, non-amnestic clinical presentations, are also commonly seen in early onset AD (EOAD) and frequently overlap with other neurodegenerative dementias such as frontotemporal dementia (FTD) leading to greater rates of misdiagnosis and diagnostic delay (Balasa et al., 2011; Mendez, 2006). In this context the use of specific biomarkers is crucial in achieving an accurate diagnosis.

Hippocampal atrophy (HA) is a well-recognized feature of AD considered a neurodegeneration biomarker in current NIA-AA diagnostic criteria (Apostolova et al., 2006; Sarazin et al., 2010; Albert et al., 2011; McKhann et al., 2011). Medial Temporal Atrophy (MTA) visual rating scale is a popular HA evaluation instrument, although its utility in EOAD is still unclear (Scheltens Ph et al., 1992). Since quantitative volumetric analysis is time consuming, semiautomatic methods have been developed. Data regarding its reliability in early symptomatic stages is lacking (Cuingnet et al., 2011).

In this study we sought to elucidate the differences in HA between LOAD, EOAD and FTD, and compare visual and semiautomatic quantitative assessment diagnostic accuracy. We hypothesized that quantitative analysis would be more useful, especially in amnesic EOAD.

Methods

Subjects

The study was approved by the Hospital Clinic Barcelona Ethics Committee and all participants gave written informed consent. One hundred forty subjects evaluated at the Alzheimer's disease and other cognitive disorders Unit at Hospital Clínic de Barcelona were enrolled on this retrospective cross-sectional study. All subjects underwent a complete neurological and neuropsychological evaluation, 3 T brain MRI and a spinal tap for the determination of AD CSF biomarkers. All subjects scored ≥20 on the Mini Mental State Examination (MMSE), had a clinical onset before 65. Patients were classified into 3 groups:

EOAD group (n = 58): 28 of them with mild cognitive impairment (MCI) (Pfeiffer Functional Activities Questionnaire (FAQ) ≤ 6) and 30 with mild dementia (Pfeffer et al., 1982). All subjects had a typical AD CSF biomarker profile. So, all EOAD fulfilling the NIA-AA diagnostic criteria for MCI due to AD or AD dementia (Albert et al., 2011; McKhann et al., 2011). Based on their clinical presentation, EOAD participants were further classified into two subgroups: amnesic (A-EOAD, 38 subjects) and non-amnesic variant (NA-EOAD, 20 subjects).

LOAD group (n = 30): All subjects had a typical AD CSF biomarker profile. Nineteen of them fulfilled the NIA-AA diagnostic criteria for MCI due to AD and eleven for AD dementia (Albert et al., 2011; McKhann et al., 2011).

FTD group (n = 20): Six behavioral variant of FTD (bvFTD), seven non-fluent variant for primary progressive aphasia (nfvPPA) and seven semantic variant of primary progressive aphasia (svPPA) (Rascovsky et al., 2011; Gorno-Tempini et al., 2011). A C9orf72 expansion was present in two cases of bvFTD and three nfvPPA subjects had GRN mutation. All FTD subjects had normal AD CSF biomarkers levels.

Healthy controls (n = 32): with no cognitive complaints, cognitive performance within normative range and normal AD CSF biomarkers. They were classified as young HC (aged under 65) (n = 16) and older ones (aged over 65) (n = 16)

Genetic and CSF biomarkers determination

APOE genotype was determined through the analysis of rs429358 and rs7412 by Sanger sequencing.

All subjects underwent a spinal tap. Levels of amyloid β (Aβ42), total-tau (t-tau), and phosphorilated-tau (p-tau) were measured using commercial sandwich ELISA kits (Fujirebio, Gent, Belgium). The CSF cut-off values determined by our laboratory were used to classify subjects according to NIA-AA criteria (Balasa et al., 2014).

Brain MRI imaging

For each subject, high-resolution sagital T1-weighted MRI images were acquired in a 3Tesla scan (Siemens Magnetom Trio, Erlangen, Germany) at the Magnetic Resonance Image Core Facility, using proprietary three-dimensional magnetization-prepared rapid-acquisition gradient echo: MPRAGE sequences (TR = 2300 ms; TE = 2,98 ms; acquisition matrix 256 × 256, voxel size 1 × 1 × 1).

Visual assessment

We used T1-coronal images on midbrain to score right and left hippocampus; we scrolled through the whole hippocampus and then selected the main assessment slice in the middle of the hippocampal body, in front of the pons. It was rated according to the five-point scale developed and validated by Scheltens et al. and then averaged to obtain a single MTA value for each subject (Scheltens Ph et al., 1992; Harper et al., 2016). An expert radiologist specialized in dementia neuroimaging (N.B.) and a trained neurologist (N.F.) blinded to diagnosis and clinical data, visually assessed MRI images.

Quantitative assessment

T1 sequences were analyzed with FreeSurfer v5.3 in neuGRID platform to obtain the HV with the pipeline nG + FreeSurfer+5.3.0 + Diagnostic+v05.xml. This pipeline relies on Freesurfer-ReconAll 5.3.0 in cross sectional mode (Fischl et al., 2004; Reuter et al., 2012; Cover et al., 2016). Freesurfer segments the hippocampi of 3D T1-weighted structural brain MRI scans. The pipeline involves intensity non-uniformity correction, affine transformation to MNI template, intensity normalization, skull stripping, removal of non-brain tissue, linear and non-linear transformations to a probabilistic brain atlas and labelling of subcortical structures. The right label per each single voxel is determined using spatial localization priors (Fischl et al., 2002). The hippocampal volumetric values in mm3 are contrasted against a normative population of 238 healthy controls of ADNI. All automated hippocampal segmentations were performed on 64-bit Linux machines using the neuGRID web-portal (www.neugrid4you.eu). NeuGRID allows to efficiently analyze large amount of imaging data with >5000 CPU cores and 20 TB of physical space (Redolfi et al., 2013; Redolfi et al., 2015). The neuGRID platform provided a final report directly to the physician reporting the volumetric information, the percentile assessments, as well as pictures of the segmented hippocampi allowing a direct evaluation of the FreeSurfer segmentation results (see Supplementary material Fig. S1). Additionally, we analyzed HV using FreeSurfer's standard approach via neuGRID, i.e., native-space volume normalized by total intracranial volume. This represented a validation between atlas space values derived with an affine transformation to the native space hippocampal volume.

Statistical analysis

Statistical analysis was conducted using Stata/IC 14.2. Categorical data was analyzed by χ2 test and quantitative data by ANOVA and Student's t-test with Bonferroni post-hoc procedure. APOE Ɛ4 status was dichotomized as carrier/non-carrier and variables were compared in ε4 carrier and non-carrier groups using ANOVA. Inter-rater reliability of left, right and averaged MTA score was determined by Kappa index and intraclass correlation coefficient (ICC) by a two-way random, absolute for single-measures [ICC (2,1)] and average measures ICCs [ICC(2,k)]. The quantitative assessment's accuracy was estimated using receiver operator characteristics (ROC). Best thresholds were selected maximizing sensitivity and specificity. Pearson correlation coefficients evaluated the association between visual and quantitative assessments.

Results

Demographics

LOAD and old HC were older than other groups as expected. Groups showed no differences in gender (Table 1). AD and FTD groups showed lower MMSE scores (p < .01) compared to HC. The presence of APOE Ɛ4 allele was highest among A-EOAD (65.8%) than in FTD and HC groups (p < .05), including significant differences between A-EOAD and NA-EOAD (p = .038).

Table 1

Demographic and clinical data for each diagnostic group.

	A-EOAD (n = 38)	NA-EOAD (n = 20)	FTD (n = 20)	Young HC (n = 16)	LOAD (n = 30)	Older HC (n = 16)
Gender (% Female)	57.9	60	45	62.5	60	46.7
Age Mean	61.3 ± 5	59.7 ± 5.7	61.1 ± 4.4	57.5 ± 3.3	71.3 ± 5.2a	74.7 ± 3.9a
AAO Mean	58.3 ± 4.7	57.2 ± 5.5	58.1 ± 4.4	–	72.2 ± 4.9b	–
Time to diagnosis	3 ± 1.5	2.5 ± 1.3	3 ± 2.2	–	2.4 ± 2.2	–
MMSE score	24.3 ± 2.6	25.1 ± 3	25.8 ± 2.8	29 ± 1.2c	24.2 ± 3	28.4 ± 0.7c
APOE Ɛ4 (%)	65.8d	35	20	18.8	55.2f	6.7
CSF Aβ42 (pg/mL)	397.6±108e	366.28±124e	851.1 ± 305	893.2 ± 247	358.2±105e	733 ± 185
CSF p-tau (pg/mL)	108.6±36e	112.1±44e	48.6 ± 18	51.4 ± 9	95±47e	59.3 ± 17
CSF t-tau (pg/mL)	748.3±426e	701.9±382e	327.6 ± 158	224.3 ± 52	677±439e	299.1 ± 116

Data are presented as means ± standard deviation. A-EOAD, Amnesic Early Onset Alzheimer's disease; NA-AEOD, Non-Amnesic Early Onset Alzheimer's disease; FTD, Frontotemporal Dementia; HC, Healthy Controls; AAO, Age At Onset.

Statistically significant (p < .01) differences compared to A-EOAD, NA-EOAD, FTD, Young HC.

Statistically significant (p < .01) differences compared to A-EOAD, NA-EOAD, FTD.

Statistically significant (p < .01) differences compared to A-EOAD, NA-EOAD, FTD and LOAD.

Statistically significant (p < .05) differences compared to NA-EOAD, FTD, LOAD and young and older HC.

Statistically significant (p < .05) differences compared to FTD, young and older HC.

Statistically significant (p < .05) differences compared to older HC.

Descriptive data

MTA scores showed substantial agreement between both raters: 88% of total agreement (k = 0.83) on the left, 88% (k = 0.84) on the right and 83% (k = 0.78) on the averaged MTA). Both single and average ICC values were > 0.9 which is considered an excellent correlation between both raters (see Supplementary material Table S1). Most discrepancies were found on 0 and 1 scoring and were solved by consensus.

No significant differences were observed between right and left Scheltens' score, neither between the whole groups FTD (nfvPPA, svPPA and bvFTD) on MTA. The comparison of MTA scores between MCI due to AD and mild AD dementia subjects did not show differences, neither considering APOE Ɛ4 status.

The distribution of the visual MTA assessment is displayed in Fig. 1, Fig. 2. Proportion of participants with MTA score ≥ 1.5 was higher in LOAD (77%, p < .01) and FTD (70%, p < .01) than A-EOAD (34.3%), NA-EOAD (30%) and HC groups (6,3%). FTD was the only group with MTA ≥ 3.5 (15%). A-EOAD showed differences compared to HC (p = .032) but not to NA-EOAD or FTD.

Fig. 1

Distribution of MTA scoring for diagnostic groups and HC (percentage, %).

Fig. 2

Box plots of the MTA and HV distribution depending on the diagnosis.

Diagnostic performance

We evaluated diagnostic sensitivity and specificity for different thresholds of the MTA score among AD and FTD groups (Table 2a). MTA performed better at discriminating LOAD and FTD from HC than EOAD from HC. However, no threshold reached 80% performance on both sensitivity and specificity. In addition, MTA did not reach 60% diagnostic accuracy at discriminating between AD groups from FTD (Table 2b).

Table 2

Diagnostic performance for each diagnostic group.

a) Diagnostic accuracy for MTA visual rating of diagnostic groups vs HC.
	A-EOAD (n = 38)	NA-EOAD (n = 20)	LOAD (n = 30)	FTD (n = 20)
MTA ≥ 1	Se 58% Sp 69%	Se 45% Sp 69%	Se 90% Sp 56%	Se 80% Sp 69%
MTA ≥ 1,5	Se 34% Sp 93%	Se 30% Sp 93%	Se 77% Sp 94%	Se 70% Sp 94%
MTA ≥ 2	Se 29% Sp 94%	Se 15% Sp 94%	Se 63% Sp 100%	Se 50% Sp 94%
AUC	0.67 (0.54–0.81)	0.63 (0.46–0.80)	0.88 (0.79–0.98)	0.85 (0.72–0.97)

A-EOAD, Amnesic Early Onset Alzheimer's Disease; NA-EOAD, Non-Amnesic Early Onset Alzheimer's Disease; FTD, Frontotemporal Dementia; HC=Healthy Controls; MTA, Medial temporal Atrophy score; Se, Sensitivity; Sp, Specificity.

There were no differences in right and left HV. Thus, averaged HV were used for subsequent analyses. No significant differences were found on mean HV values among FTD clinical variants (nfvPPA, svPPA and bvFTD) neither between MCI due to AD and mild dementia due to AD. HV considering APOE Ɛ4 status did not show differences between groups.

The distribution of the HV by different diagnostic groups is shown in Fig.2. We observed significant differences on the average HV between groups (p < .001). The post hoc analysis showed that HV LOAD (4890 ± 866 mm3) was smaller than the other groups (p < .05). A-EOAD HV (5568 ± 663 mm3) was smaller than NA-EOAD (6260 ± 872 mm3) (p = .019) and HC (6766 ± 711 mm3) (p < .001), but we did not find differences between A-EOAD and FTD groups (5671.1mm3 ± 1109 mm3). No differences between NA-EOAD and HC or FTD were found. FTD had smaller HV than HC (p = .001). Furthermore, the distribution and means of the HV by different diagnostic groups using total intracranial volume normalization is shown as Supplementary material Table S2 and Fig. S2.

Fig. S2

Distribution of hippocampal atrophy by groups using total intracranial volume normalization.

ROC curves were used to determine HV threshold with a better diagnostic performance. The 5838 mm3 threshold distinguished between LOAD and HC with very good sensitivity (100%) and specificity (87%). HV threshold of 6259 mm3 showed good sensitivity (88%) and specificity (87%) to distinguish A-EOAD from HC (Fig. 3.1a). For identifying NA-EOAD versus HC, the best threshold was 6745 mm3, although it showed suboptimal sensitivity (69%) and specificity (70%) (Fig.3.1b). The best HV cut-off for FTD versus HC was 6365.5 mm3 (sensitivity 82% and specificity 75%) (Fig. 3.1c).We also compared AD groups to FTD (Fig. 3.2a and 3.2b). The best cut-off for comparing A-EOAD and FTD was 5763.5 mm3 and for comparing NA-EOAD versus FTD was 5845 mm3, although both showed sensitivity and specificity below 70%. The 5124 mm3 threshold distinguished LOAD from FTD with 75% sensitivity and 63% specificity. Moreover, diagnostic performance of HV using total intracranial volume normalization is shown in Supplementary Material Fig. S3.

Fig. 3

Diagnostic performance of HV quantitative assessment.

Fig. S3

ROC curves of hippocampal volume of diagnostic groups using total intracranial volume normalization.

Correlation between quantitative and visual assessment

There was a significant moderate inverse correlation (r = −0.537, p < .001) between MTA score and the quantitative HV evaluation.

Discussion

In this study we compared a visual scale and a semiautomatic tool for measuring HA in patients with EOAD, LOAD and FTD in order to determine the diagnostic accuracy of both tools in early symptomatic stages. In our cohort, MTA scale had little utility as a diagnostic biomarker and the quantitative measurement of HV was useful only in distinguishing LOAD and A-EOAD from HC.

To our knowledge this is the first study to compare both visual and volumetric assessment in a well-characterized biomarker-confirmed cohort of LOAD, EOAD and FTD patients.

Since its development, Scheltens' HA visual rating scale has been validated in numerous studies as a good predictor of progression from MCI to AD dementia and to discriminate AD from HC and has also been proved to correlate with volumetric methods (Heo et al., 2013; Wahlund et al., 2000). The MTA score has been proposed as the best marker of HA and the MTA ≥ 1.5 cut-off has been recommended to be used for AD diagnosis under the age of 75 (Van de Pol and Scheltens, 2014). In the case of volumetric assessment, it is not clear yet which HV cut off should be used.

In our study, MTA was not able to accurately discriminate HC from AD or FTD patients, since specificity and sensitivity did not reach the 80% threshold for any of the groups (Consensus report of the Working Group on: "Molecular and Biochemical Markers of Alzheimer's Disease", 1998). The quantitative assessment only achieved a good diagnostic performance in discriminating LOAD and A-EOAD from HC with little discriminative capacity between the other comparisons.

Most previous studies point to the fact that both visual scales and quantitative analysis are able to discriminate between AD and HC, despite relevant variability in terms of diagnostic performance and they mostly focus on LOAD patients (Heo et al., 2013; Cavedo et al., 2014). In reference to LOAD, our results are in line with previous publications in terms of sensitivity albeit greater specificity in our cohort. (Harper et al., 2016) This could be explained by the characteristics of our older HC group in whom preclinical AD had been carefully ruled out through the use of CSF biomarkers and an extensive neuropsychological evaluation. In both visual scales and quantitative analysis, the diagnostic accuracy is better in LOAD than EOAD, as well as it is better in A-EOAD than in NA-EOAD. It makes sense, since LOAD patients mostly present amnesic clinical phenotype and have more HA whereas EOAD patients frequently have a non-amnestic clinical profile related to less HA (Balasa et al., 2011; Koedam et al., 2010; Phillips et al., 2018). Poor MTA diagnostic accuracy results on EOAD may be due to the fact we have only included patients in early symptomatic stages (i.e, MCI and mild dementia, MMSE≥20). Is well known that patients in advanced stages of the disease have more HA, thus, in consequence, assessing MTA at these stages could increase its discriminative power; but on the daily clinical practice it is in the early stages of the disease when the differential diagnosis is more complex and it is precisely where biomarkers should be more useful and make the difference. On the other hand, most studies using Scheltens' MTA scale do not have CSF results and the AD diagnosis is based only on clinical criteria (Cuingnet et al., 2011; Heo et al., 2013; Wahlund et al., 2000; Van de Pol and Scheltens, 2014; Duara et al., 2013; Ferreira et al., 2015; Varon et al., 2015; Pereira et al., 2014; Shen et al., 2011; Ridha et al., 2007). We consider this fact to be a significant caveat since trying to establish AD diagnosis without biological confirmation of the disease can lead to higher rates of misdiagnosis (Beach et al., 2012). Claus et al., 2017 reported good diagnostic accuracy (83.3% sensitivity and 86.4% specificity) for MTA ≥ 1 at discriminating 18 EOAD patients from subjective cognitive impairment subjects (Claus et al., 2017). Their data does not fit with our results, which could be explained by several methodological differences such as: the use of computerized tomography (CT) scan, the lack of biomarker supported diagnosis, the wide range of clinical severity stages included in the sample and its small size.

Volumetric assessment is expected to provide an added value in AD diagnosis (Bosco et al., 2017). Several techniques have been developed to achieve the most accurate measurement and many comparisons of both tools have been published (Cuingnet et al., 2011; Heo et al., 2013; Wahlund et al., 2000; Van de Pol and Scheltens, 2014; Duara et al., 2013; Ferreira et al., 2015; Varon et al., 2015; Pereira et al., 2014; Shen et al., 2011). However, comparing them is difficult because of the heterogeneity of study samples and imaging techniques used. Cuingnet et al. compared the diagnostic performance of different quantitative methods, including volumetric measurement of HV of clinically diagnosed MCI and mild AD patients aged 55–90 (Cuingnet et al., 2011). HV evaluation was as sensitive as other methods but however less specific: 63% sensitivity and 80% specificity on distinguishing HC from AD patients and 73% sensitivity and 74% specificity between HC and MCI. By contrast, our data show acceptable diagnostic performance of the volumetric analysis to distinguish A-EOAD from HC. That difference can be explained also by methodological differences such as no biomarker supported diagnosis.

When comparing both LOAD and EOAD with FTD patients, visual and quantitative HA assessment does not appear to be a good diagnostic biomarker. This data fits well with previous works that show similar HA on both disorders (Van de Pol et al., 2006; Hornberger et al., 2012). The relevant MTA and its wide distribution found on FTD could be related to the heterogeneity and the fast disease progression of the FTD itself.

The clinical overlap between NA-EOAD and FTD sometimes leads to differential diagnosis difficulties in clinical practice and highlights the crucial importance of using disease-specific biomarkers (Beach et al., 2012). Unfortunately, in our cohort, neither visual nor quantitative assessment performed optimally at differentiating them.

In line with previous studies mostly performed in late-onset cognitive impairment, we observed a moderate correlation between MTA visual rating and volumetric techniques (Dhikav et al., 2017). The moderate correlation could be explained because MTA score is based mostly on a single slice while HV is a three-dimensional measure. These data suggest that both techniques may perform in a different way with regard to HV measurements, therefore caution must be taken before considering them equivalent.

Furthermore, variability through different HV quantification methods has been previously described (Buckner et al., 2004). Both approaches implemented in neuGRID, i.e.: HV in template space and total intracranial volume normalization methods, provided similar results. It suggests that HV diagnostic performance is consistent across the two analysis methods. Moreover, in the normalization method, the HV is not biased by extensive cortical atrophy.

Our results would support the idea that visual MTA rating may play a limited role in the clinical diagnosis of EOAD, in particular in early clinical stages. With regards to volumetric assessment, it would only show an advantage compared to visual rating in discrimination between LOAD and A-EOAD from HC. Current criteria for the clinical diagnosis of AD includes HA as a neurodegeneration biomarker (Albert et al., 2011; McKhann et al., 2011). In light of our results showing low discriminative capacity of HA between groups, this feature should be interpreted with caution in early-onset cognitive impairment especially in patients with non-amnestic presentations.

The main limitation of our study is the relatively small sample size, although, as far as we are aware this is the larger cohort study focused on EOAD patients reported until now. On the other hand, we have only used one semiautomatic volumetric analysis and it is possible that other volumetric techniques for evaluating HA may show different diagnostic performances.

Conclusion

In conclusion, the utility of HA as a diagnostic biomarker in our cohort of EOAD patients, is limited. Further studies in larger and well-characterized cohorts are needed to determine the diagnostic utility of HA as a biomarker in the early stages for EOAD patients.

The following are the supplementary data related to this article.

Fig. S1

Example of hippocampal volume report provided by NeuGRID platform.

Table S1

Inter-rater reliability measures: total agreement, Kappa index and intraclass correlation coefficient.

Table S2

Mean hippocampal volumes by groups using total intracranial volume normalization.

Disclosure statement

Authors state that there are no conflicts of interest to disclose.