Quantifying Knowledge of Alzheimer's Disease: An Analysis of the Psychometric Properties of the Alzheimer's Disease Knowledge Scale.

INTRODUCTION: The Alzheimer's Disease Knowledge Scale (ADKS) is one of the most popular instruments for assessing a person's knowledge regarding Alzheimer's disease (AD). The objective of this study was to explore ADKS item characteristics with item response theory (IRT) procedures.
METHODS: A noninterventional web-based study was conducted. A nonparametric IRT procedure, Mokken analysis, was used to explore the underlying latent structure of the ADKS and ADKS item characteristics regarding scalability and violations of the monotone homogeneity (MH) model. A random-effects meta-analysis was implemented that combined ADKS scores from independent studies.
RESULTS: A total of 447 employees of a pharmaceutical company participated in the study. The mean ADKS score was 21.2 (SD 2.8). Mokken analysis showed that most ADKS items (22 of 30) do not fit to any scale and can be considered to be scale independent. Two items (#1: particularly prone to depression; #20: depression can be mistaken for AD) fit to a domain relating to depression, another two items (#2: mental exercise can prevent AD development; #8: benefit of psychotherapy) can be related to potential prevention and improvement, and four items (#12: poor nutrition can make the symptoms worse; #18: high cholesterol may increase the risk of AD; #26: high blood pressure may increase the risk of AD; #27: genes can only partially account for AD development) fit to a risk factor domain. As expected from those results, neither the overall scale (H = 0.033) nor its items showed appropriate scalability index values, suggesting that ADKS does not fit to a MH model. Eleven items showed violations of the assumptions of the MH model. The meta-analytical average score was 21.78 (95% CI 20.67-22.90), with healthcare professionals and caregivers showing the highest levels of AD knowledge.
CONCLUSION: Although the ADKS does not present a unidimensional structure, its independent items together provide a comprehensive spectrum of information regarding AD knowledge.

Key Summary Points

Digital Features

This article is published with digital features, including a summary slide, to facilitate understanding of the article. To view digital features for this article go to https://doi.org/10.6084/m9.figshare.13526369.

Introduction

Alzheimer’s disease (AD) leads to irreversible progressive cognitive impairment. It is the most common type of dementia, affecting millions of people worldwide [1, 2].

Although information about cognitive impairment has been widely disseminated through numerous awareness campaigns and educational programs, several studies have revealed different gaps in the public’s knowledge regarding dementia and AD [3-6]. Adequate AD knowledge throughout society could lead to earlier identification of this disorder in the population, which would result in earlier appropriate healthcare for people with dementia and would decrease the stigma associated with AD [3]. In addition, understanding people’s perceptions of AD can be crucial when carrying out specific psychosocial intervention strategies [3].

Various instruments have been developed to assess knowledge of dementia and AD, including the Alzheimer’s Disease Knowledge Test (ADKT), the University of Alabama Alzheimer’s Disease Knowledge Test for Health Professionals (UAB-ADKT), the Dementia Quiz (DQ), the Knowledge about Memory Loss and Care test (KAML-C), the Alzheimer’s Disease Knowledge Scale (ADKS), the Dementia Knowledge Assessment Tool Version 2 (DKAT2), the Dementia Knowledge 20 (DK-20), and the University of Jaén Alzheimer’s Care Scale (UJA ACS) [7-9].

The ADKS is one of the most widely used of these instruments [3]. It is a 30-item questionnaire that measures what people know about AD across seven critical knowledge domains: risk factors (six items), symptoms (4 items), assessment and diagnosis (4 items), disease trajectory (4 items), life impact (3 items), treatment and management (4 items), and caregiving (5 items) [10]. The ADKS has adequate psychometric properties and is designed to be administered to the general public, patients, caregivers, and healthcare professionals [8, 10].

We aimed to achieve a greater understanding of the psychometric characteristics of the ADKS by applying nonparametric item response theory (IRT) procedures to the responses obtained from an anonymous survey. To our knowledge, this is the first time that a study has focused on ADKS item characteristics instead of using classical approaches that rely on the reliability and validity of the total ADKS score.

Methods

Study Design and Participants

A noninterventional, cross-sectional, self-completed, web-based study (the CONOCE study) was conducted among employees of a pharmaceutical company in Spain (Roche Farma SA). This study was conducted in accordance with the Good Clinical Practice Guidelines of the International Conference on Harmonisation and with the ethical principles of the Declaration of Helsinki. It was approved by the investigational review board of the Hospital Universitario Ramón y Cajal, Madrid, Spain (reference code: 372). Informed consent was obtained from all subjects. Participants were invited to participate from December 2019 to February 2020.

Study Procedures

Participants answered questions regarding demographic data and completed the ADKS. Each ADKS item is a statement that can be either false (value 0) or true (value 1) [10]. “True” is the correct response for 18 items, whereas the remaining 12 items are reverse scored, so “false” is the correct response for those items. A higher score indicates better knowledge regarding AD.

Statistical Analyses

Continuous variables were described using means and standard deviations, and categorical variables using frequencies and percentages. A nonparametric IRT procedure—Mokken analysis—was used to explore the underlying measurement structure of the ADKS and how well it fits to the monotone homogeneity (MH) model, which allows people to be ranked on AD knowledge based on their total ADKS scores. Among other criteria, the MH model assumes unidimensionality of the latent construct measured with the scale, as assessed with Loevinger’s scalability coefficients [11]. Scalability coefficients describe the degree to which individual items (H), pairs of items (H), and the overall set of items (H) form a scale that can be used to rank people on their ability (knowledge) regarding the latent trait being measured (AD knowledge). Scalability coefficients reflect the ratio of the observed Guttman error frequency (inadequacy with which the achievement level is matched to the difficulty of the item being measured) to the expected Guttman error frequency (i.e., based on chance alone) for a pair of items [12, 13]. When data fit well to the MH model, H values are positive and range from zero to 1, where a value of 1 indicates no Guttman errors and values closer to zero indicate many Guttman errors. To define an appropriate unidimensional scale, each one of the 30 ADKS items was required to have a scalability coefficient (H) of ≥ 0.30 and an overall scale scalability index (H) of ≥ 0.30 [14]. We extracted the published effect sizes of the ADKS from previous studies (mean values, standard deviations, and sample sizes) and combined them with our own results using a random-effects meta-analysis to arrive at an estimate of the overall AD knowledge and how this knowledge differed between population subgroups [15].

We performed the statistical analyses with R version 4.0 (https://cran.r-project.org/) using the libraries “mokken” for IRT and “metafor” for the random-effects meta-analysis [11, 16, 17].

Results

A total of 447 subjects participated in the study. Most participants were aged between 18 and 50 years (78%), female (65%), and had a bachelor’s or master’s degree (89%). Forty-two (9.4%) participants reported having a first-degree relative with AD. Demographic characteristics of the sample are shown in Table 1.

Table 1

Demographic characteristics of the study sample (n = 447)

	n	%
Gender
Female	292	65.3
Age, years
≤ 30	66	14.8
31–40	113	25.3
41–50	170	38.0
51–60	93	20.8
> 60	5	4.7
Education
Secondary	23	5.1
Vocational training	26	5.8
Bachelor's degree or equivalent	167	37.4
Master's degree or equivalent	203	45.4
PhD or equivalent	28	6.3
AD caregiver	42	9.4

AD Alzheimer’s disease

Overall AD Knowledge

The mean ADKS score was 21.2 (SD 2.8; 95% CI 20.9–21.4). For four items (caregiving domain: “informed decisions” and “difficulties with self-care;” risk factors domain: “at risk for high blood pressure” and “at risk for high cholesterol”), less than 50% of the answers were correct. Figure 1 shows the ADKS items ranked by percentage of correct answers (“percent correct”) after reversing the scores for inverse items.

Fig. 1

Alzheimer’s Disease Knowledge Scale: correct answers

Figure 2 shows the frequency distribution of scores along the observed range (13–29 points); 50% of the observations occur between the score values of 19 and 23 (which therefore correspond to the 25% and 75% percentiles of the ADKS score distribution, respectively).

Fig. 2

Alzheimer’s Disease Knowledge Scale: distribution of scores

Underlying Structure of the ADKS and Item Characteristics

Most of the ADKS items (22 of 30) do not fit to any scale and can be considered to be scale independent. Two items (#1 and #20) fit to a knowledge domain related to depression, another two items ( #2 and #8) are related to AD prevention and recovery, and four items (#12, #18, #26, and #27) fit to a knowledge domain related to risk factors. Table 2 shows the results of the automated item selection procedure (AISP) that was implemented to ascertain the knowledge domains covered by the ADKS in the current survey.

Table 2

Correct answers, results from the Mokken automated item selection procedure, scalability coefficients, and monotone homogeneity model violation counts for the Alzheimer’s Disease Knowledge Scale (n = 447)

Item no. and wording	Correct answers n (%)	AISP	Scalability coefficient	No. of significant violations
1. People with AD are particularly prone to depression	327 (73)	3	0.057	0
2. It has been scientifically proven that mental exercise can prevent a person from getting AD	334 (75)	2	0.047	1
3. After symptoms of AD appear, the average life expectancy is 6–12 years	249 (56)	0	0.015	0
4. When a person with AD becomes agitated, a medical examination might reveal other health problems that caused the agitation	277 (62)	0	− 0.006	1
5. People with AD do best with simple instructions, giving one step at a time	432 (97)	0	0.003	1
6. When people with AD begin to have difficulty taking care of themselves, caregivers should take over right away	156 (35)	0	0.045	1
7. If a person with AD becomes alert and agitated at night, a good strategy is to try to make sure that the person gets plenty of physical activity during the day	274 (61)	0	− 0.004	1
8. In rare cases, people have recovered from AD	398 (89)	2	− 0.002	2
9. People whose AD is not yet severe can benefit from psychotherapy for depression and anxiety	398 (89)	0	0.054	1
10. If trouble with memory and confused thinking appears suddenly, it is likely due to AD	283 (63)	0	0.019	0
11. Most people with AD live in nursing homes	334 (75)	0	− 0.008	1
12. Poor nutrition can make the symptoms of AD worse	385 (86)	1	0.073	0
13. People in their 30s can have AD	341 (76)	0	0.039	0
14. A person with AD becomes increasingly likely to fall down as the disease gets worse	377 (84)	0	0.031	0
15. When people with AD repeat the same question or story several times, it is helpful to remind them that they are repeating themselves	343 (77)	0	0.034	0
16. Once people have AD, they are no longer capable of making informed decisions about their own care	203 (45)	0	0.063	0
17. Eventually, a person with AD will need 24 h supervision	365 (82)	0	0.006	1
18. Having high cholesterol may increase a person’s risk of developing AD	149 (33)	1	0.092	0
19. Tremor or shaking of the hands or arms is a common symptom in people with AD	340 (76)	0	− 0.006	3
20. Symptoms of severe depression can be mistaken for symptoms of AD	288 (64)	3	0.060	0
21. AD is one type of dementia	297 (66)	0	− 0.002	0
22. Trouble handling money or paying bills is a common early symptom of AD	335 (75)	0	0.048	0
23. One symptom that can occur with AD is believing that other people are stealing one’s things	334 (75)	0	0.075	0
24. When a person has AD, using reminder notes is a crutch that can contribute to decline	301 (67)	0	0.038	0
25. Prescription drugs that prevent AD are available	410 (92)	0	0.015	0
26. Having high blood pressure may increase a person’s risk of developing AD	149 (33)	1	0.064	0
27. Genes can only partially account for the development of AD	409 (92)	1	0.098	0
28. It is safe for people with AD to drive as long as they have a companion in the car at all times	412 (92)	0	0.005	1
29. AD cannot be cured	424 (95)	0	0.052	0
30. Most people with AD remember recent events better than things that happened in the past	354 (79)	0	0.033	0

An AISP value of zero denotes an item that does not fit to any scale

AD Alzheimer’s disease, AISP automated item selection procedure, MH monotone homogeneity

As expected from those results, neither the overall scale (H = 0.033) nor its items showed appropriate scalability indices. No item had a scalability index H ≥ 0.30, and indices were in fact quite low, suggesting that the ADKS does not fit a MH model. Moreover, 11 items violated assumptions of the MH model (Table 2).

Meta-analysis of the ADKS Scores

Table 3 shows the main characteristics and effect sizes of the studies that have reported ADKS mean scores (including the present study). Figure 3 displays a forest plot with individual and combined ADKS scores. The combined result has a relatively high mean score (mean 21.78; 95% CI 20.67–22.90) that differs significantly among subgroups (Q test = 11.35 on 4 df; p value = 0.02), with health professionals presenting the highest mean score (11 data points, mean ADKS = 22.9), followed closely by caregivers (3 data points, mean ADKS = 21.6). A study conducted by Baral et al. found that students obtained the lowest scores (ADKS score = 15.4), whereas the subgroup of health professionals yielded the highest scores in a study by Carpenter et al. (ADKS score = 27.4) [10, 18].

Table 3

Average Alzheimer’s Disease Knowledge Scale scores for different populations

Study and year	Country	Population	n	Mean ADKS (SD)
Carpenter [10]	USA	Students Older adults with no cognitive impairment Senior center staff Dementia caregivers Healthcare professionals	484 89 61 54 75	20.19 (3.59) 24.10 (2.95) 20.15 (4.10) 22.70 (4.27) 27.40 (1.89)
Nordhus [19]	Norway	Psychologists	956	24.10 (2.50)
Smyth [22]	Australia	Healthcare professionals	360	23.60 (3.26)
Hattink [20]	Netherlands and UK	Dementia caregivers	83	24.28 (3.35)
Hughes [28]	USA	Alzheimer’s Association website visitors	552	23.11 (3.36)
Kim [21]	South Korea	Students Healthcare professionals Dementia caregivers	422 64 89	19.29 (2.93) 21.56 (2.61) 17.93 (2.88)
Mat Nuri [23]	Malaysia	Pharmacists	445	19.05 (3.69)
Sullivan [8]	Australia	Healthcare professionals	55	22.80 (2.89)
Amado [26]	Brazil	Laypeople Healthcare professionals	899 515	20.50 (3.33) 23.46 (3.51)
Alacreu [24]	Spain	Pharmacists General practitioners	578 104	22.95 (2.50) 24.40 (2.30)
Strom [25]	India	Nurses	15	19.50 (3.00)
Baral [18]	Nepal	Students	385	15.45 (2.95)
Wang [27]	China	General practitioners	341	21.42 (2.73)
This study 2020	Spain	Roche employees	447	21.16 (2.83)

ADKS Alzheimer’s Disease Knowledge Scale, SD standard deviation

Fig. 3

Meta-analysis of Alzheimer’s Disease Knowledge Scale scores

Discussion

The ADKS belongs to a group of psychometric scales that aim to assess knowledge regarding AD, as this can assist with the development of psychoeducational curricula and interventions for dementia care [7]. The original validation of the ADKS included different populations in the USA (college students, older adults with no cognitive impairment, dementia caregivers, and healthcare professionals), and the scale was also applied to or validated by studying college students in South Korea and Nepal, caregivers in the Netherlands, the United Kingdom, and South Korea, and healthcare professionals in Australia, Brazil, China, India, Malaysia, and Spain [8, 10, 18–27]. It was also administered to laypeople in Brazil, and was included as a knowledge resource on the website of the Alzheimer’s Association in the USA [26, 28].

Despite some criticisms of the ADKS that are linked to dichotomous item assessment, reverse-scored items, and a likely ceiling effect for some items, this is currently one of the few validated scales to be used either at the intervention planning stage or as a signal or outcome measure when evaluating interventions or the overall knowledge of dementia [7, 8, 29]. Because the ADKS was designed to estimate the overall knowledge regarding AD, not an underlying construct or dimension, several common psychometric statistics such as internal reliability or analyses of dimensional structure are not as relevant as they might be to other scales that are used to evaluate the effectiveness of interventions [7, 8].

Our study shows that the range of percent correct values for ADKS items and the near-symmetric distribution of overall scores may support its usefulness for relatives of people with cognitive impairment who are seeking a neurological evaluation, and to guide the psychoeducational efforts of dementia support groups [10]. However, the results of the IRT approach do not support scale unidimensionality or the ranking of subjects across a continuum of AD knowledge. The AISP indicates that the underlying structure of the ADKS does not fit the originally theorized domains. It instead suggests that items are independent, which is relevant for a general knowledge scale that is used to detect knowledge gaps, but not for ranking individuals.

Carpenter et al. [10] developed the ADKS in 2009 to incorporate new scientific understanding about AD into the ADKT. All measures require periodic updates to keep pace with developments and new information emerging from the rapidly evolving field of cognitive disorders. In their systematic review of AD knowledge outcome measures, Spector et al. recommend the development of a contemporary instrument that incorporates items relating to biopsychosocial and patient-centered models of AD care [7]. Our results do not support the use of the total ADKS score to meaningfully rank people according to their AD knowledge, as the data did not fit with the MH model. However, even though the IRT results indicate that the ADKS is not suitable for evaluating people at the individual level, it does not directly follow that group mean scores based on knowledge accrued from multiple independent items are not suitable for comparing population subgroups in terms of their AD knowledge, as our meta-analytical results show. In fact, what might be argued is that the ADKS is a useful outcome measure for evaluating the effectiveness of interventions. When Hattink et al. evaluated an e-learning course for dementia caregivers, they did not find a significant difference between subjects randomized to the experimental psychoeducational intervention (n = 37, mean ADKS score = 24.37, SD = 2.94) and those randomized to the control intervention (n = 46, mean ADKS score = 24.39, SD = 2.90) [20]. The mean difference (MD) was not significantly different between groups (MD − 0.02, 95% CI − 1.30 to 1.26) and did not show a significant change at four months from baseline (n = 83, mean ADKS score = 24.28, SD = 3.35). However, even given the serious doubts regarding the use of ADKS as an outcome measure to evaluate interventions, knowledge of the correctness of responses may be useful when designing and developing interventions aimed at improving AD knowledge among caregivers and health professionals—aims that also guided the development of the ADKS [10].

This study has several limitations. There is participant self-selection bias; it is possible that the survey tended to attract the most motivated respondents or those who were most knowledgeable about AD. In addition, carrying out the study in only one company may have limited the generalizability of the findings to other healthcare communities, or even the general population.

Conclusion

Lack of information and misconceptions about AD are still critical problems. Whereas the ADKS does not show the characteristics and dimensionality expected of a scale whose aim is to evaluate a nonobservable latent factor or construct, it does present a set of items that together contribute to the overall knowledge of AD, and so it might be useful for evaluating knowledge gaps regarding AD. As originally described, the ADKS can be used with laypeople or caregivers to determine what they know about AD, and by healthcare professionals and support groups to guide psychoeducational efforts.

Why carry out this study?

Lack of knowledge and misconceptions about Alzheimer’s disease are critical problems around the world.

Improving the public’s understanding of Alzheimer’s disease may facilitate early diagnosis, reduce stigma about the disease, and prompt a discussion of the needs of patients and their families.

What was learned from the study?

The Alzheimer’s Disease Knowledge Scale is a self-rated questionnaire that assesses what people know about Alzheimer’s disease in different key domains. In addition to its good psychometric properties, it is an easy-to-implement and reliable tool for evaluating knowledge gaps in laypersons, caregivers, and healthcare professionals.