Staging Disease Severity Using the Alzheimer's Disease Composite Score (ADCOMS): A Retrospective Data Analysis.

INTRODUCTION: The Alzheimer's disease (AD) composite score (ADCOMS) has been shown to be a more sensitive measure of cognitive change in early AD (i.e., mild cognitive impairment [MCI] and mild AD) than commonly used measures. This study derived ADCOMS value ranges associated with different disease severity stages across the predementia and AD continuum.
METHODS: Data from patients enrolled in the Alzheimer's Disease Neuroimaging Initiative were assessed at baseline and month 24. Data were randomly split into derivation and validation samples. Receiver-operating characteristic (ROC) curves of ADCOMS values were generated in the derivation sample to assess the sensitivity and specificity of ADCOMS cutoff values compared with existing disease severity cutoff scores using the Clinical Dementia Rating (CDR) global, CDR Sum of Boxes, Alzheimer's Disease Assessment Scale-Cognitive Subscale, and Mini-Mental State Examination. Optimal ADCOMS cutoff values for each disease stage were compared between the derivation and the validation samples using a χ2 test. The diagnostic accuracy of the derived ADCOMS cutoff values was then assessed. The analyses were repeated for the subset with positive amyloid β confirmation (Aβ +).
RESULTS: The following ADCOMS value ranges for the total population and Aβ + population were identified: < 0.29 indicative of normal cognition, 0.29 to < 0.45 indicative of MCI, 0.45-0.77 indicative of mild AD, and > 0.77 indicative of at least moderate AD. The reliability of these ADCOMS value ranges was supported by diagnostic accuracy tests and tests indicating no significant difference in the ROC curves between the derivation and validation samples.
CONCLUSION: ADCOMS values ranges can be used to assess the severity of cognitive decline. The derived severity threshold score ranges for ADCOMS will enable its use as an endpoint in clinical trials assessing disease progression and clinical outcomes of disease-modifying therapies in persons with MCI or early AD, including patients with Aβ + confirmation.

Key Summary Points

Introduction

Staging of Alzheimer’s disease (AD) severity via assessment measures is a common approach in clinical and research settings to assess disease severity and identify transitions between health states. The US Food and Drug Administration’s Guidelines for the Clinical Evaluation of Antidementia Drugs [1] states that documentation of stage or severity of dementia is key and that it must be recorded in a manner that is understandable. Clinically, AD-related dementia is often staged as mild, moderate, or severe according to thresholds of neuropsychological scales such as the Clinical Dementia Rating Scale (CDR), the Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog), and the Mini-Mental State Examination (MMSE), among others.

However, the pathology of AD precedes the development of dementia; by the time dementia is diagnosed, substantial and irreversible neurological damage has occurred. Potential disease-modifying treatments aim to halt the development of such damage and thus require the identification of patients in earlier, predementia, stages. Mild cognitive impairment (MCI) is clinically defined as a stage between cognitively normal and dementia [2] and is recognized as a formal clinical diagnosis [3]. During MCI, no functional impairment is observed, and there are only subtle cognitive changes (e.g., short-term memory complaints, word-finding difficulties) that are difficult to detect. Neuropsychological scales originally developed to diagnose and stage severity levels of AD-related dementia are insensitive to detecting MCI [4-6]. Despite this limitation, such measures continue to be commonly used to evaluate cognitive and functional outcomes and treatment effects in trials of potentially disease-modifying compounds focusing on predementia and early AD (i.e., MCI due to AD and mild AD dementia).

To address these challenges, Wang et al. [7] statistically derived a composite measure composed of items from existing scales that appears to be more sensitive to changes in cognition during the predementia and early stages of AD, despite the individual scales lacking overall sensitivity. Clinically sensitive items from the ADAS-Cog, MMSE, and CDR Sum of Boxes (CDR-SB) were identified, and the AD Composite Score (ADCOMS) was derived with the intention using it as a primary outcome measure for registration trials of disease-modifying treatments in persons with predementia and early AD-related dementia. The ADCOMS has demonstrated improved sensitivity over the individual scales and requires smaller sample sizes to detect change over time [7]. Furthermore, ADCOMS has been shown to be superior to other measures in detecting and measuring early cognitive decline and disease progression among patients with early stages of AD in several clinical trials [8-10]. For this reason, the use of ADCOMS in the assessment of treatment response in clinical trials of investigational therapies among patients with early AD is increasing [11-15].

Although there is evidence that changes in ADCOMS values are more likely to be observed during early cognitive decline than changes in scores of other measures [9, 16, 17], scoring ranges identifying specific cognitive decline severity stages for the ADCOMS have not been established or standardized. The ADCOMS was specifically developed for use in early cognitive decline; thus, the utility of ADCOMS beyond early AD is also undocumented. Identifying score ranges for the ADCOMS associated with different stages in the progression of AD will inform objective measurements of cognitive change over time and aid with monitoring the treatment effects on AD disease progression in clinical research settings.

The objective of this study was to derive ADCOMS value ranges that identify specific predementia and AD-related dementia disease severity stages. We were particularly interested in deriving ADCOMS value ranges that may be used to distinguish among patients with clinical diagnoses of normal cognition, MCI, mild AD, and more advanced AD.

Methods

Participants

Data used in the preparation of this article were obtained from the ADNI database (adni.loni.usc.edu) on December 13, 2018. The ADNI was launched in 2003 as a public-private partnership, led by principal investigator Michael W. Weiner, MD. The primary goal of the ADNI has been to test whether serial magnetic resonance imaging, positron emission tomography, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of MCI and early AD. For up-to-date information, see www.adni-info.org.

The population included in ADNI ranged from age 55–90 years (inclusive), had a Modified Hachinski score of ≤ 4, had a Geriatric Depression Scale score of < 6, had a study partner able to provide an independent evaluation of functioning, and spoke either English or Spanish. Table 1 presents a description of criteria applied in ADNI to assign patients to disease severity categories of normal cognition, MCI, and AD. Table 2 presents the number of patients within each disease severity category available for analysis. Early onset AD (defined as a diagnosis of at least mild AD before the age of 65 years) was present among 48 patients at baseline and 28 patients at month 24.

Table 1

ADNI population definitions

ADNI population definition
Cognitively normal
Normal memory function documented by scoring at specific cut points on the Logical Memory II subscale (Delayed Paragraph Recall) from the Wechsler Memory Scale-Revised
≥ 16 years of education: score of ≥ 9
8–15 years of education: score of ≥ 5
0–7 years of education: score of ≥ 3
MMSE score of 24–30
CDR score of 0, with a memory box score of 0
MCI
MMSE score of 24–30
A memory concern by either the study partner or the patient (with verification by the partner)
Objective memory loss documented by scoring at specific cut points on the Logical Memory II subscale (Delayed Paragraph Recall) from the Wechsler Memory Scale-Revised
≥ 16 years of education: score of ≤ 8 (late MCI) or ≥ 9 to 11 (early MCI)
8–15 years of education: score of ≤ 4 (late MCI) ≥ 5 (early MCI)
0–7 years of education: score of ≤ 2 (late MCI) ≥ 3 to 6 (early MCI)
CDR score of 0.5, with a memory box score of at least 0.5
Absence of significant levels of impairment in other cognitive domains
Essentially preserved activities of daily living
Absence of dementia
Mild AD
MMSE scores of 20–26
A memory concern by the either the study partner or the patient (with verification by the partner)
Objective memory loss documented by scoring at specific cut points on the Logical Memory II subscale (Delayed Paragraph Recall) from the Wechsler Memory Scale-Revised
≥ 16 years of education: score of ≤ 8
8–15 years of education: score of ≤ 4
0–7 years of education: score of ≤ 2
CDR score of 0.5 or 1.0
Meets National Institute of Neurological and Communicative Diseases and Stroke/Alzheimer’s Disease and Related Disorders Association criteria for probable AD

AD Alzheimer’s disease, ADNI Alzheimer’s Disease Neuroimaging Initiative, CDR clinical Dementia Rating Scale, MCI mild cognitive impairment, MMSE Mini-Mental State Examination

Table 2

ADNI available data

	Cognitively normal	MCI	AD
Number of available patients: total
Baseline	777	938	358
Visit at 24 months	440	483	339
Number of available patients: amyloid β biomarker
Baseline	191	441	224
Visit at 24 months	128	227	208

AD Alzheimer’s disease, ADNI Alzheimer’s Disease Neuroimaging Initiative, CDR Clinical Dementia Rating Scale, MCI mild cognitive impairment

Written informed consent was obtained from all participants in the ADNI project, and recruitment was approved by the institutional review boards of all participating institutions (see supplementary material). Permission to access and use the ADNI data was obtained by the authors from the ADNI Initiative.

Alzheimer’s Disease Composite Score (ADCOMS)

The ADCOMS contains a total of 12 cognitive and functional items, including 4 items from the ADAS-Cog, 2 items from the MMSE, and all 6 items from the CDR-SB (Table 3). To score the ADCOMS, each item is weighted according to the partial least-squares regression coefficients identified during the measure derivation [7] (Table 3). Total ADCOMS values range from 0 to 1.97, with higher scores indicative of greater impairment.

Table 3

ADCOMS items and weighting

Source: Wang et al. [7]

Scale	Item		PLS coefficient (weighting factor)
	Name	Possible score
ADAS-Cog	Delayed word recall	0–10	0.008
	Orientation	0–8	0.017
	Word recognition	0–12	0.004
	Word-finding difficulty	0–5	0.016
MMSE	Orientation to time	0–5	0.042
	Drawing	0–1	0.038
CDR-SB	Personal care	0–3	0.054
	Community affairs	0–3	0.109
	Home and hobbies	0–3	0.089
	Judgment and problem solving	0–3	0.069
	Memory	0–3	0.059
	Orientation	0–3	0.078

To score the ADCOMS, each item is weighted according to the partial least-squares regression coefficients. Total ADCOMS values range from 0 to 1.97

ADAS-Cog Alzheimer’s Disease Assessment Scale-Cognition, ADCOMS Alzheimer’s Disease Composite Score, CDR-SB Clinical Dementia Rating Scale-Sum of Boxes, MMSE Mini-Mental State Examination, PLS partial least-squares

Other AD Severity Measures

The other AD severity measures used in the analysis include the CDR, CDR-SB, MMSE, and ADAS-Cog (see supplementary material for a description of these measures). The value ranges for these measures that are currently used as cut points to identify MCI and to determine AD-related dementia disease severity are presented in Table 4.

Table 4

Cut scores for reference assessment measures

Sources: [31–33]

Scale	Cognitively normal	MCI	Mild AD	Moderate AD	Severe AD
CDR	0	0.5	1	2	3
CDR-SB	0	0.5–4.0	4.5–9.0	9.5–15.5	16–18
ADAS-Cog	< 8	8–15	16–32	≥ 33	NR
MMSE	≥ 26	≥ 26	21–25	11–20	≤ 10

AD Alzheimer’s disease, ADAS-Cog Alzheimer’s Disease Assessment Scale-Cognition, CDR Clinical Dementia Rating Scale, CDR-SB Clinical Dementia Rating Scale-Sum of Boxes, MCI mild cognitive impairment, MMSE Mini-Mental State Examination, NR not relevant because of no unique score available for this disease stage

Analyses

Overview

Analyses were conducted in several stages using the Stata statistical software package (version 15.1). First, a baseline dataset and a month 24 dataset were each randomly split to derive two data sets (a derivation set and a validation set). Then, receiver-operating characteristic (ROC) curves were used to identify the ADCOMS values with the best balance of sensitivity and specificity for distinguishing between different disease severity levels (e.g., between MCI and mild AD; see the section on ROC curves for more detail). ROC curves were applied to the ADCOMS data using each reference assessment measure separately (i.e., the CDR, CDR-SB, ADAS-Cog, and MMSE), which resulted in a range of ADCOMS values that most likely distinguish between each different disease severity levels. This approach allowed an assessment of the ADCOMS values for each of the predementia and AD-related dementia disease stages that align with those from measures commonly used in clinical practice and research settings. We then assessed whether the results from the derivation data set were comparable to those from the validation data set using a goodness of fit test.

In the next stage, we applied a ROC curve to ADCOMS data using patients with a clinical diagnosis of either MCI or mild AD. This was to test whether the ADCOMS value ranges we had derived from the reference measures do have discriminate ability. If the ADCOMS value ranges we derived have discriminate ability, then the ADCOMS value with the best balance of sensitivity and specificity from this ROC curve would fall within the previously derived ADCOMS value ranges. This allowed an assessment of whether the derived ADCOMS value ranges can discriminate between clinician-diagnosed disease stages.

In the final stage, all analyses were repeated with the use of a subset of data restricted to patients with positive amyloid β confirmation. These analyses are described in more detail below.

Derivation and Validation Sample

Two sets of data were derived for eligible patients who had observations on each of the assessment measures: one for the baseline observation (i.e., data recorded upon entry into the ADNI; n = 2073) and one for the visit at 24 months (i.e., data recorded 2 years after entry into the ADNI; n = 1262). Within each data set, the sample was randomly split by 50% to generate a derivation sample and a validation sample.

ROC Curves

Published cut point scores for the reference assessment measures were used to indicate patient disease stage (as documented in Table 4). ROC curves were applied to the ADCOMS data using each reference measure and each cut point within each reference measure. For example, a ROC curve of ADCOMS data was generated for patients with a baseline CDR score of 0 or 0.5 (i.e., cognitively normal or MCI, according to the established cut point for the CDR). ROC curves plot the sensitivity (the probability of predicting that a true positive will be a positive) against 1-specificity (the probability of predicting that a true negative will be a positive) for each possible cut point score (i.e., every ADCOMS value from 0 to 1.97). Values of the area under the ROC curve close to 1 indicate that the cut point has high diagnostic accuracy, while a value of 0.5 indicates a noninformative cut point that makes random classifications. Thus, for the current example, the ROC curve was used to indicate the ADCOMS value that had the highest sensitivity and specificity in distinguishing between patients with a CDR score indicative of cognitively normal or MCI.

This process was repeated for each measure (i.e., the CDR, CDR-SB, ADAS-Cog, and MMSE) and for each cut point for each measure (i.e., cognitively normal or MCI; MCI or mild AD; mild AD or moderate/severe AD) using both the baseline and 24-month visit data sets. The analyses were conducted using the derivation sample and the equality of each generated ROC curve was compared against the validation sample using a χ2 test.

Diagnostic Accuracy

Once the range of possible cut points for the ADCOMS value had been derived across each of the reference measures, tests of clinical diagnostic accuracy (according to ADNI assigned diagnostic categories) were conducted. First, baseline data were restricted to patients with a CDR score of 0.5: such patients could be defined as having MCI or mild AD according to the ADNI criteria (see Table 1) (n = 471 patients diagnosed with MCI; n = 84 patients diagnosed with AD). A ROC curve was generated for the validation sample, and the ADCOMS value that maximized the product of sensitivity and specificity for distinguishing between MCI and AD was identified. Next, this process was repeated with the use of the 24-month visit data restricted to patients with a CDR score of 1 at the 24-month visit (n = 70 mild AD; n = 24 moderate or severe AD) to determine the diagnostic accuracy of ADCOMS values in distinguishing between mild AD and moderate or severe AD.

The results from the ROC curves based on the published cut point scores for the reference assessment measures were reviewed together with the diagnostic accuracy tests. The ADCOMS cut points indicated by the diagnostic accuracy tests were selected as those most likely to identify patient staging of predementia and AD severity.

Sensitivity Analysis: Predisposition for AD

Finally, all analyses described above were repeated with the use of a subset of data restricted to patients with positive amyloid β confirmation (baseline, n = 856; month 24, n = 563) measured either as a biomarker (amyloid β1–42, ≤ 980 pg/ml) within cerebrospinal fluid (baseline, n = 709; month 24, n = 500) or via a positron emission tomography (PET) scan (florbetapir [formally AV-45] ≥ 1.13 pg/ml, or Pittsburgh Compound B [PIB] ≥ 1.47 pg/ml) (baseline, n = 147; month 24, n = 63). This allowed an examination of whether the derived ADCOMS cut scores are sensitive to variations in patient predisposition for developing AD. Previous research indicates concordance between CSF biomarkers and PET imaging for amyloid β, which both correlate highly with brain biopsy findings and predict clinical progression in AD [18, 19]. However, some evidence suggests that the changes measurable in the CSF occur in earlier stages of the disease [20] and the majority of our amyloid β sample were confirmed via CSF.

Results

Sample Overview

The demographic characteristics of the study population are provided in Table 5. Scores on all the assessment measures at baseline were indicative of significantly greater impairment among the AD-related dementia group versus the MCI group and significantly greater impairment among the MCI group versus the cognitively normal group. Among participants who were cognitively normal at both baseline and the 24-month visit, change scores on all the assessment measures were small (e.g., no change in ADCOMS values and an increase of 0.04 in CDR-SB scores). However, cognitively normal participants who progressed to MCI or AD at the 24-month visit had larger change scores (e.g., an increase of 0.12 in ADCOMS values and 1.31 in CDR-SB scores). The same was true for participants diagnosed with MCI (e.g., ADCOMS change scores of 0.04 and CDR-SB change scores of 1.50 among those who remained diagnosed with MCI versus 0.34 and 2.97, respectively, among those who progressed to AD). The same pattern of findings was observed among the subset of the population with positive amyloid β confirmation (Table 5). This suggests that the measures have reasonable known-groups validity and are sensitive to changes in disease severity, regardless of predisposition for developing AD.

Table 5

Participant characteristics

	Total population			Positive amyloid β confirmation
	Cognitively normalBaseline (n = 777)	MCIBaseline (n = 938)	AD Dementia (any)Baseline (n = 358)	Cognitively normalBaseline (n = 191)	MCIBaseline (n = 441)	AD Dementia (any)Baseline (n = 224)
Age at baseline, years
Mean (SD)	73.1 (6.1)	73.0 (7.6)	74.9 (7.9) *	74.5 (6.2)	73.3 (7.2)	73.9 (8.1)
Range	56 to 90	54 to 91	55 to 91	57 to 90	54 to 91	55 to 90
Female gender identity, n (%)	428 (55.1) *	381 (40.6)	159 (44.4)	109 (57.1)*	176 (39.9)	99 (44.2)
Education level, years
Mean (SD)	16.6 (2.6) *	15.9 (2.8)	15.2 (3.0) *	16.5 (2.7)	16.1 (2.8)	15.5 (2.8)
Range	6 to 20	4 to 20	4 to 20	6 to 20	6 to 20	8 to 20
ADCOMS (0–1.97)a
Baseline	n = 777	n = 938	n = 358	n = 191	n = 441	n = 224
Mean (SD)	0.05 (0.03) *	0.20 (0.10)	0.57 (0.19) *	0.05 (0.03)*	0.21 (0.10)	0.58 (0.19)*
Range	0 to 0.20	0.03 to 0.58	0.15 to 1.17	0 to 0.20	0.04 to 0.57	0.17 to 1.10
Change from baseline at month 24
Same baseline diagnosis	n = 406	n = 452	n = 159	n = 112	n = 207	n = 94
Mean (SD)	0.01 (0.05)	0.04 (0.11)	0.37 (0.27)	0.01 (0.05)	0.05 (0.11)	0.38 (0.29)
Range	− 0.15 to 0.31	0.56	− 0.21 to 1.10	− 0.15 to 0.26	− 0.19 to 0.43	− 0.21 to 1.10
Progressed to another diagnosis	n = 31	n = 177	–	n = 20	n = 113	–
Mean (SD)	0.12 (0.16)	0.34 (0.19)	–	0.10 (0.14)	0.33 (0.19)	–
Range	− 0.03 to 0.59	− 0.19 to 1.25	–	− 0.03 to 0.51	− 0.19 to 1.25	–
CDR global score (0 to 3)a
Baseline	n = 777	n = 938	n = 358	n = 191	n = 441	n = 224
Mean (SD)	0.00 (0.03) *	0.50 (0.04)	0.76 (0.26) *	0 (0)a	0.50 (0.04)	0.77 (0.26)*
Range	0 to 0.5	0 to 1	0.5 to 2	0	0 to 1	0.5 to 2
Change from baseline at month 24
Same baseline diagnosis	n = 406	n = 453	n = 159	n = 112	n = 207	n = 94
Mean (SD)	0.04 (0.13)	− 0.02 (0.14)	0.48 (0.56)	0.07 (0.18)	− 0.01 (0.15)	0.47 (0.56)
Range	0 to 0.5	− 0.5 to 0.5	− 0.5 to 2	0 to 0.5	− 0.5 to 0.5	− 0.5 to 2
Progressed to another diagnosis	n = 31	n = 177	–	n = 20	n = 113	–
Mean (SD)	0.45 (0.24)	0.34 (0.36)	–	0.43 (0.18)	0.32 (0.32)	–
Range	0 to 1	− 0.5 to 1.5	–	0 to 0.5	− 0.5 to 1.5	–
CDR-SB (0–18)a
Baseline	n = 779	n = 939	n = 360	n = 191	n = 441	n = 224
Mean (SD)	0.04 (0.13) *	1.50 (0.88)	4.40 (1.68) *	0.05 (0.16)*	1.56 (0.9)	4.44 (1.61)*
Range	0 to 1	0 to 5.5	1 to 10	0 to 1	0 to 5.5	1 to 10
Change from baseline at month 24
Same baseline diagnosis	n = 406	n = 453	n = 159	n = 112	n = 207	n = 94
Mean (SD)	0.08 (0.38)	0.27 (0.98)	3.17 (2.58)	0.12 (0.42)	0.36 (0.99)	3.23 (2.71)
Range	− 1 to 3.5	− 3 to 5.5	− 2 to 11	− 1 to 2	− 2 to 3.5	− 2 to 11
Progressed to another diagnosis	n = 31	n = 177	–	n = 20	n = 113	–
Mean (SD)	1.31 (1.41)	2.97 (1.82)	–	1.08 (1.05)	2.88 (1.80)	–
Range	0 to 5	− 3 to 10	–	0 to 4	− 3 to 10	–
ADAS-Cog (0–70)a
Baseline	n = 776	n = 938	n = 358	n = 191	n = 441	n = 224
Mean (SD)	6.85 (3.13) *	10.42 (4.59)	19.70 (6.72) *	6.52 (3.09)*	11.05 (4.70)	20.25 (6.97)*
Range	0 to 19.33	1 to 27.67	7.3 to 42.67	0 to 16.33	1.00 to 27.00	8.67 to 42.67
Change from baseline at month 24
Same baseline diagnosis	n = 405	n = 453	n = 159	n = 112	n = 207	n = 94
Mean (SD)	− 0.31 (2.83)	0.45 (3.88)	9.10 (8.24)	− 0.34 (2.65)	0.75 (4.41)	9.01 (7.97)
Range	− 9.7 to 10.3	− 11 to 20.7	− 6 to 32.3	− 7 to 6.7	− 11 to 20.7	− 6 to 31
Progressed to another diagnosis	n = 31	n = 177	–	n = 20	n = 113	–
Mean (SD)	0.82 (3.48)	5.69 (6.10)	–	0.63 (3.52)	5.55 (5.86)	–
Range	− 5 to 7	− 7 to 39	–	− 5 to 7	− 5.3 to 32.3	–
MMSE (0–30)a
Baseline	n = 778	n = 939	n = 360	n = 191	n = 441	n = 224
Mean (SD)	29.08 (1.10) *	27.62 (1.83)	23.20 (2.09) *	29.08 (1.14)*	27.49 (1.86)	23.17 (2.03)*
Range	24 to 30	19 to 30	18 to 29	24 to 30	23 to 30	19 to 27
Change from baseline at month 24
Same baseline diagnosis	n = 406	n = 453	n = 159	n = 112	n = 207	n = 94
Mean (SD)	− 0.05 (1.36)	− 0.42 (2.26)	− 4.24 (4.81)	− 0.33 (1.43)	− 0.87 (2.54)	− 4.34 (4.67)
Range	− 4 to 4	− 17 to 6	− 21 to 5	− 4 to 3	− 17 to 6	− 21 to 4
Progressed to another diagnosis	n = 31	n = 177	–	n = 20	n = 113	–
Mean (SD)	− 1.29 (1.78)	− 3.89 (3.40)	–	− 1.20 (1.58)	− 3.45 (3.31)	–
Range	− 4 to 2	− 19 to 5	–	− 4 to 2	− 19 to 5	–

AD Alzheimer’s disease, ADAS-Cog Alzheimer’s Disease Assessment Scale-Cognition, ADCOMS Alzheimer’s Disease Composite Score, CDR Clinical Dementia Rating Scale, CDR-SB Clinical Dementia Rating Scale-Sum of Boxes, MCI mild cognitive impairment, MMSE Mini-Mental State Examination, SD standard deviation, –, missing data

*p < 0.001

aParenthetical numbers next to each assessment measure refer to the possible score range on the measure. All statistical comparisons are based on MCI as the reference group and the χ2 statistic (categorical data) or analysis of variance (continuous data)

In the following subsections, the results from the ROC curves based on the published cut point scores for the reference assessment measures are presented for the baseline and the 24-month visit data. The diagnostic accuracy test results are then presented, followed by a summary and examination of the selected ADCOMS cut scores.

ROC Curves

Baseline Data

The results of the ROC curves of ADCOMS values generated using the baseline data for both the total population and the amyloid β–positive population are presented in Table 6. The ROC curves primarily suggested an optimal ADCOMS cut point score of 0.08 to 0.11 for normal cognition versus MCI. Of note, there is no threshold on the MMSE that distinguishes between normal cognition and MCI; thus, MMSE scores could not be used for this determination. The optimal ADCOMS cut point score to distinguish between MCI and mild AD varied across the different assessment measures (Table 6). There were too few patients at baseline with a CDR or CDR-SB score indicative of moderate AD; thus, ROC curves could not be generated for differentiating mild from moderate AD using these measures. On the ADAS-Cog and MMSE, optimal scores for distinguishing between mild and moderate AD also varied (Table 6). The tests of equality between the derivation and the validation sample confirmed the results (Table 6).

Table 6

ROC curve baseline results: optimal ADCOMS values

	Optimal cut point score	Area under the curve	χ2 test of equality, p value	Correctly classified, %
Total population
Cognitively normal and MCIa
CDR global (0 and 0.5)	0.10	0.976	0.157	91
CDR-SB (0 and 0.5–4.0)	0.08	0.976	0.360	93
ADAS-Cog (< 8 and 8–15)	0.11	0.811	0.210	72
MCI and mild AD
CDR global (0.5 and 1)	0.44−0.47b	0.993	0.037	91−96c
CDR-SB (0.5–4.0 and 4.5–9.0)	0.47	0.995	0.046	96
ADAS-Cog (8–15 and 16–32)	0.27−0.31b	0.871	0.013	81−82c
MMSE (≥ 26 and 21–25)	0.23	0.912	0.638	82
Mild AD and moderate ADd
ADAS-Cog (16–32 and ≥ 33)	0.69	0.913	0.606	82
MMSE (21–25 and 11–20)	0.62	0.864	0.536	79
Confirmed positive amyloid β population
Cognitively normal and MCIa
CDR global (0 and 0.5)	0.10	0.976	0.468	91
CDR-SB (0 and 0.5–4.0)	0.08	0.980	0.494	93
ADAS-Cog (< 8 and 8–15)	0.11	0.826	0.809	76
MCI and mild AD
CDR global (0.5 and 1)	0.47	0.995	0.048	95
CDR-SB (0.5–4.0 and 4.5–9.0)	0.47	0.995	0.093	96
ADAS-Cog (8–15 and 16–32)	0.33	0.872	0.620	81
MMSE (≥ 26 and 21–25)	0.34	0.907	0.428	87
Mild AD and moderate ADd
ADAS-Cog (16–32 and ≥ 33)	0.69	0.881	0.981	79
MMSE (21–25 and 11–20)	0.62	0.823	0.606	75

AD Alzheimer’s disease, ADAS-Cog Alzheimer’s Disease Assessment Scale-Cognition, ADCOMS Alzheimer’s Disease Composite Score, CDR Clinical Dementia Rating Scale, CDR-SB Clinical Dementia Rating Scale-Sum of Boxes, MCI mild cognitive impairment, MMSE Mini-Mental State Examination, ROC receiver-operating characteristic

aThe MMSE is not sensitive to distinguishing between normal cognition and MCI; thus, no results based on the MMSE are reported for the cognitively normal and MCI comparison

bBetween the validation ROC and the derivation ROC

cIn both the derivation and validation sets

dNo patients had CDR or CDR-SB scores indicative of moderate AD at baseline; thus, no results based on these measures are reported for the mild AD and moderate AD comparison

Twenty-Four-Month Visit Data

The results of the ROC curves of ADCOMS values generated using the 24-month visit data for both the total population and the amyloid β-positive population are presented in Table 7. These results confirmed the finding that an optimal ADCOMS cut score of 0.08 or 0.10 distinguishes between normal cognition and MCI. For MCI and mild AD and for mild AD and moderate AD, the suggested cut score varied across the different assessment measures (Table 7). The tests of equality between the derivation and validation samples confirmed the results (Table 7).

Table 7

ROC curve 24-month visit results: optimal ADCOMS values

	Optimal cut point score	Area under the curve	χ2 test of equality, p value	Correctly classified, %
Total population
Cognitively normal and MCIa
CDR global (0 and 0.5)	0.08	0.942	0.114	86
CDR-SB (0 and 0.5–4.0)	0.08	0.936	0.017	85
ADAS-Cog (< 8 and 8–15)	0.10	0.871	0.782	79
MCI and mild AD
CDR global (0.5 and 1)	0.49	0.986	0.391	94
CDR-SB (0.5–4.0 and 4.5–9.0)	0.49	0.992	0.111	94
ADAS-Cog (8–15 and 16–32)	0.46	0.913	0.379	85
MMSE (≥ 26 and 21–25)	0.29	0.920	0.854	88
Mild AD and moderate ADb
CDR global (1 and 2)	0.97	0.986	0.811	91
CDR-SB (4.5–9.0 and 9.5–15.5)	1.03	0.984	0.862	92
ADAS-Cog (16–32 and ≥ 33)	0.91	0.917	0.337	81
MMSE (21–25 and 11–20)	0.77	0.871	0.377	79
Confirmed positive amyloid β population
Cognitively normal and MCIa
CDR global (0 and 0.5)	0.09	0.947	0.395	87
CDR-SB (0 and 0.5–4.0)	0.08	0.939	0.057	87
ADAS-Cog (< 8 and 8–15)	0.11	0.869	0.393	81
MCI and mild AD
CDR global (0.5 and 1)	0.49	0.975	0.819	92
CDR-SB (0.5–4.0 and 4.5–9.0)	0.49	0.988	0.418	93
ADAS-Cog (8–15 and 16–32)	0.46	0.893	0.893	84
MMSE (≥ 26 and 21–25)	0.29	0.884	0.285	82
Mild AD and moderate ADb
CDR global (1 and 2)	1.13	0.997	0.374	98
CDR-SB (4.5−9.0 and 9.5−15.5)	1.13	0.995	0.914	96
ADAS-Cog (16−32 and ≥ 33)	0.98	0.942	0.095	87
MMSE (21−25 and 11−20)	0.69	0.883	0.204	79

AD Alzheimer’s disease, ADAS-Cog Alzheimer’s Disease Assessment Scale-Cognition, ADCOMS Alzheimer’s Disease Composite Score, CDR Clinical Dementia Rating Scale, CDR-SB Clinical Dementia Rating Scale-Sum of Boxes, MCI mild cognitive impairment, MMSE Mini-Mental State Examination, ROC receiver-operating characteristic

aThe MMSE is not sensitive to distinguishing between normal cognition and MCI; thus, no results based on the MMSE are reported for the cognitively normal and MCI comparison

Diagnostic Accuracy

Analyses were restricted to patients with a CDR score of 0.5 at baseline (MCI, n = 471; AD, n = 84) to determine the cut point for the ADCOMS value that differentiated between ADNI-defined clinical diagnoses of MCI or mild AD. The ROC curve demonstrated that an ADCOMS cut score of 0.31 (sensitivity = 90.5%, specificity = 86.6%) best discriminated between patients with MCI versus mild AD: 87% of patients were correctly classified. The area under the ROC curve was 0.933.

When restricting the analysis to patients with a CDR score of 1.0 at month 24 (mild AD, n = 70; moderate or severe AD, n = 24), the ROC curve demonstrated that an ADCOMS cut score of 0.77 (sensitivity = 79.2%, specificity = 68.6%) best discriminated between patients with mild AD versus moderate/severe AD: 71% of patients were correctly classified. The area under the ROC curve was 0.821.

Derived ADCOMS Staging Scores

The results from all ROC curve analyses suggested that an ADCOMS value < 0.11 is indicative of normal cognition. Correspondingly, the mean (standard deviation [SD]) ADCOMS at baseline for cognitively normal participants was 0.05 (0.03) for the total population and the population with positive amyloid β confirmation (Table 5). When the cut point scores were applied to the 24-month visit data, we found that 73% of participants from the total population (65% of patients with positive amyloid β confirmation) with an ADCOMS value < 0.11 had a diagnosis of normal cognition rather than MCI.

For MCI, the ROC results suggested the ADCOMS value should be less than a value somewhere between 0.23 and 0.49, while the diagnostic accuracy checks suggested a score of 0.31. Therefore, an ADCOMS value < 0.31 was selected as the optimal score to distinguish MCI from mild AD; thus, an ADCOMS value between 0.11 and < 0.31 is considered to be indicative of MCI. Correspondingly, the mean (SD) ADCOMS value at baseline for participants diagnosed with MCI was 0.20 (0.10) for the total population and 0.21 (0.10) for the population with positive amyloid β confirmation (Table 5). When the cut point scores were applied to the 24-month visit data, we found that 94% of participants from the total population (93% of patients from the amyloid b population) with an ADCOMS value between 0.11 to < 0.31 had a diagnosis of MCI rather than mild AD.

The results from all ROC curve analyses suggested that the ADCOMS value should be less than somewhere between 0.62 to 1.03 for mild AD. However, the diagnostic accuracy checks suggest a score of 0.77. Therefore, an ADCOMS value < 0.77 was selected as the optimal score to distinguish mild AD from moderate/severe AD; thus, an ADCOMS value between 0.31 and < 0.77 is indicative of mild AD. Correspondingly, the mean (SD) ADCOMS value for participants diagnosed with mild AD at baseline in both the total population (n = 327) and population with positive amyloid β confirmation (n = 203) was 0.56 (0.18). When the cut point scores were applied to the 24-month visit data, we found that 91% of participants from the total population (93% of patients from the amyloid β population with an ADCOMS value between 0.31 and < 0.77 had a diagnosis of mild AD rather than moderate/severe AD.

Based on the results above, an ADCOMS value ≥ 0.77 was considered indicative of moderate/severe AD. Few patients were diagnosed with moderate AD at baseline; the mean (SD) ADCOMS value for participants diagnosed with moderate/severe AD at the 24-month visit was 1.07 (0.29) for the total population (n = 102) and 1.10 (0.31) for the population with positive amyloid β confirmation (n = 58). When the cut point scores were applied to the 24-month visit data, we found that 62% of participants in the total population and 63% of participants with positive amyloid β confirmation with an ADCOMS value ≥ 0.77 had a diagnosis of moderate/severe AD rather than mild AD.

Figure 1a, b presents a box plot of ADCOMS values by diagnosis at baseline for the total population and amyloid β population, with horizontal lines representing the selected ADCOMS cut point scores. Figure 2a, b presents ADCOMS values at the 24-month visit, which shows that within each diagnosis, the interquartile range of ADCOMS values fell within the selected cut point range.

Fig. 1

Box plot of ADCOMS values at baseline by diagnosis for the total and confirmed amyloid β-positive populationsa. a Total population. b Confirmed amyloid β-positive population. AD Alzheimer’s disease, ADCOMS Alzheimer’s Disease Composite Score, MCI mild cognitive impairment. aHorizontal dashed lines represent the selected ADCOMS cut point scores (i.e., 0.11, 0.31, and 0.77). Whiskers represent the minimum and maximum values excluding outliers; the horizontal line within the box represents the median; the upper and lower portions of the box represent the upper and lower quartiles; circles represent outliers

Fig. 2

Box plot of ADCOMS values at 24-month visit by diagnosis for the total and confirmed amyloid β-positive populationsa. a Total population. b Confirmed amyloid β-positive population. AD Alzheimer’s disease, ADCOMS Alzheimer’s Disease Composite Score, MCI mild cognitive impairment. aHorizontal dashed lines represent the selected ADCOMS cut point scores (i.e., 0.11, 0.31, and 0.77). Whiskers represent the minimum and maximum values excluding outliers; the horizontal line within the box represents the median; the upper and lower portions of the box represent the upper and lower quartiles; circles represent outliers

Table 8 presents values of the biomarkers total tau, tau phosphorylated at threonine 181 (p-tau181), and amyloid β1–42 as measured in CSF at baseline and the number of patients carrying the apolipoprotein ε4 allele (APOE4) gene according to ADCOMS staging groups. People staged as having normal cognition using the ADCOMS have significantly lower mean tau and p-tau181 levels and significantly higher mean amyloid β1–42 values than those staged as having early AD (soluble amyloid β1–42 is known to decrease as patients progress[21]). Additionally, the likelihood of being an APOE4 carrier increased across the ADCOMS staging groups, such that people staged as having moderate/severe AD had the highest likelihood of carrying this gene.

Table 8

Biomarker values and APOE4 genotype according to ADCOMS staging score group

	Normal cognition (ADCOMS < 0.11)N = 946	MCI (ADCOMS > 0.11 and < 0.31)N = 679	Mild AD (ADCOMS ≥ 0.31 and < 0.77N = 412	Moderate/Severe AD (ADCOMS ≥ 0.77)N = 59
CSF Tau (pg/ml)
N	491	431	251	40
Mean (SD)	237.7 (96.0)a	290.8 (128.6)a	361.1 (154.5)b	391.0 (142.1)
CSF P-tau181 (pg/ml)
N	491	431	251	40
Mean (SD)	21.9 (10.0)a	28.3 (14.6)a	36.2 (16.8)b	37.9 (14.2)
CSF Amyloid β1–42 (pg/ml)
N	491	431	251	40
Mean (SD)	1181.9 (440.6)a	964.0 (439.3)a	670.1 (303.5)b	622.7 (288.4)
APOE4 carrier
N	671	614	381	56
Yes: n (%)	19 (2.8)a	73 (11.9)a	68 (17.9)b	11 (19.6)

Statistical significance was assessed using a one-way analysis of variance for continuous measures (tau, ptau181, amyloid β1–42) with Bonferonni correction for multiple comparisons and a χ2 goodness of fit test for APOE4

AD Alzheimer’s disease, ADCOMS Alzheimer’s Disease Composite Score, APOE4 apolipoprotein ε4 allele, CSF cerebrospinal fluid, MCI mild cognitive impairment

ap < 0.001 compared with all other ADCOMS staging subgroups

bp < 0.001 compared with all other ADCOMS staging subgroups except the ADCOMS ≥ 0.77 (moderate/severe AD) subgroup

Discussion

This study aimed to derive ADCOMS value ranges for disease severity levels including predementia (cognitively normal and MCI) and AD-related dementia (mild and moderate/severe) that could differentiate between patients these different disease severity levels. The use of score ranges or cut point scores on cognitive and functional measures to determine patient disease severity is common within AD. For example, the MMSE is a commonly applied assessment tool that is well known among clinicians and is often used to stage dementia in clinical practice and within clinical trials. Typically, scores of 21–26 denote mild AD, 10–20 denote moderate AD, and < 10 denotes severe AD, although there is some variation in the application of these scores (e.g., a cut point score of 24 is sometimes used for mild AD [22]). Similarly, the CDR has demonstrated high validity and reliability [23-26] in distinguishing between levels of AD severity, and there is evidence that MMSE scores map well onto CDR categories [27]. However, as earlier noted, the ADCOMS has been shown to demonstrate greater sensitivity in measuring early cognitive decline in those with predementia and in measuring change during clinical trials [8-10]. Since the ADCOMS is increasingly being used as a primary or secondary endpoint in clinical trials of potential disease-modifying treatments currently being tested in those with MCI or mild dementia [11-15], identifying scoring for the ADCOMS related to different stages in the progression of AD is warranted to aid in the monitoring of treatment effects.

Using a large sample of participants from the North American ADNI study, we derived the following severity scoring ranges for the ADCOMS measure: a score of < 0.11 is indicative of normal cognition; a score of 0.11 to <0.31 is indicative of MCI; a score of 0.31-0.77 is indicative of mild AD; a score of > 0.77 is indicative of at least moderate AD. These scores were derived from ADNI study entry data when participants were predominantly diagnosed as cognitively normal or with MCI and from 2-year follow-up data when participants had begun transitioning to mild and moderate AD. Thus, we were able to derive severity staging scores across the predementia and early AD continuum. The derived results were consistent for the derivation and validation samples. Additionally, differences in biomarkers and genotypes across the derived staging score groups provided further diagnostic accuracy confirmation. Moreover, we replicated the results among a subset of the population who had positive amyloid β confirmation, indicating that the scores are not sensitive to variations in patient predisposition for developing AD.

To derive the cut point scores, analyses were conducted using four assessment measures that have varying degrees of sensitivity to early cognitive decline. Thus, the suggested cut points varied according to the reference measure. This was particularly the case for analyses using the MMSE, which consistently suggested lower ADCOMS cut point scores. As earlier noted, the MMSE is insensitive to early cognitive decline; the lower ADCOMS cut point scores suggested from analyses using the MMSE may be reflective of this. Although the ADCOMS contains two items from the MMSE (orientation to time and construction), these items have been shown to be related to AD progression among individuals with MCI [28], which may contribute to the improved sensitivity of ADCOMS values in measuring early cognitive decline. Furthermore, the ADCOMS contains all domains from the CDR-SB, and scores on these domains are weighted higher than scores on the other items that make up the ADCOMS. This may explain why the proportions of patients correctly classified were consistently highest when using the CDR and CDR-SB reference cut points. However, the ADCOMS differentially weights the CDR-SB scores such that functional scores contribute more than cognitive scores, which means measurement between the assessment tools is not equal. Since function is known to be largely preserved during early cognitive decline but impaired upon AD onset [29, 30], the ADCOMS weighting structure may influence its ability to detect mild AD over the CDR-SB.

There are some limitations to consider when assessing the findings from this study. First, the ADNI data source contains only small numbers of patients with moderate AD and very few with severe AD. This may have influenced the selection of the ADCOMS cut score indicative of moderate/severe AD and the known-groups validity of the cut score, since scores may not have been normally distributed and generalizable to the wider population of patients with moderate/severe AD. Future research using a larger sample of patients with moderate/severe AD would allow an assessment of the validity of our findings. Another limitation of the present study was the partial reliance on previously suggested cut point scores for the CDR, CDR-SB, MMSE, and ADAS-Cog. These cut point scores are imprecise measures for which sensitivity and specificity have not been formally assessed (excepting CDR-SB [31]), and these cut point scores may not accurately distinguish patients in the early stage of AD from cognitively normal patients. However, they have been used extensively in the clinical trial setting and demonstrated high reliability and validity. Our findings also confirmed the validity of these cut point scores and their diagnostic discriminatory power.

Strengths of the current study include the use of a large, rich, longitudinal data source; replication across randomly derived samples and over time; primary use of the ROCs for the ADNI-assigned diagnostic groups in the final selection of the ADCOMS cut points; and sensitivity analyses using a patient population predisposed to developing AD. However, the ADNI population is homogeneous, and patients were carefully selected for study entry based on strict eligibility criteria. To further validate these findings, research examining the utility of these interpretative guidelines for staging AD and MCI using data from an alternative heterogeneous sample of patients with MCI and AD would be beneficial.

Conclusion

Since Wang et al. [7] have shown that in persons with MCI and mild AD, the ADCOMS may be a more sensitive measure of disease progression than the MMSE, ADAS-Cog, CDR, or CDR-SB, it is currently being used as an endpoint for clinical trials of potentially disease-modifying treatments given to persons with MCI or mild AD dementia. Our study has developed score ranges for the ADCOMS that can be consistently mapped to existing criterion measures including CDR-SB and other instruments in AD to distinguish between different levels of disease severity as diagnosed in the population included in the ADNI database. The derived severity staging thresholds for ADCOMS will enable its use as an endpoint in clinical trials assessing disease progression and clinical outcomes of disease-modifying therapies in persons with MCI or early AD, including patients with Aβ + confirmation. Additional research is needed to show the extent to which changes in this measure over time are clinically meaningful to patients.

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 153 kb)

Why carry out this study?

A primary treatment goal for persons with symptoms related to Alzheimer’s disease (AD) in the predementia or early dementia stages is to slow or reverse disease progression.

The AD Composite Score (ADCOMS) has been shown to be more sensitive to changes in cognition in those with early AD than other measures of AD disease progression.

Staging scores for the ADCOMS indicating different levels of cognitive decline are needed to assess the impact of potential disease-modifying treatments on disease progression in clinical trials in those with mild cognitive impairment or mild dementia.

What was learned from the study?

Data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) were used to derive staging score ranges for the ADCOMS measure that could be used to distinguish among those with normal cognition, mild cognitive impairment, mild AD, and moderate/severe AD.

These staging score ranges were replicated in patients with a confirmed positive amyloid β biomarker.