Italian version of the short 10/66 dementia diagnostic schedule: a validation study.

OBJECTIVES: To determine the criterion and concurrent validity of the Italian version of the short 10/66 Dementia Diagnostic Schedule and algorithm in a sample of Italian native speakers, older adults.
DESIGN: A cross-sectional, validation study.
SETTING: The study was conducted with older adults living in the community and in nursing homes in the Canton of Ticino, Switzerland, and the Piedmont region in Italy between March and August 2019. PARTICIPANTS: A convenience sample of 229 participants (69% females) were recruited. The eligibility criteria were being ≥60 years old and having an informant. The final sample included 74 participants (32%) with a previous clinical diagnosis of dementia and 155 (68%) cognitively healthy older adults. PRIMARY AND SECONDARY OUTCOME MEASURES: The short version of 10/66 Dementia Diagnostic Schedule consists of the Community Screening Instrument for Dementia, the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) 10-word list learning task with delayed recall and the depression scale, Euro-Depression (EURO-D) scale. Disability was measured using the WHO Disability Assessment Schedule (WHO-DAS II).
RESULTS: The Italian version of the short 10/66 Dementia Diagnostic Schedule showed fair sensitivity (87%), specificity (61%) and agreement with the clinical diagnosis of dementia (kappa=0.40, area under the receiver operating characteristics curve=0.74). Older adults with dementia living in nursing homes had higher disability scores (WHO-DAS II mean=23.14, SE=1.29) than those living in the community (WHO-DAS II mean=7.08, SE=0.66). WHO-DAS II was positively correlated with the short version of the 10/66 dementia diagnosis (β=5.23, 95% CI 2.05 to 8.41).
CONCLUSIONS: In settings where lengthy diagnostic procedures are not feasible, the short 10/66 is a practical tool to identify dementia in older adults. Our findings extend evidence on the validity of the 10/66 dementia diagnostic algorithm to high-income countries, where epidemiological evidence on dementia and its impact is outdated. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Our study is the first to validate the short version of the 10/66 dementia diagnostic schedule and algorithm to detect dementia in high-income settings and in older adults living in the community and in nursing homes.

We developed, piloted, tested and used innovative data capturing methods on mobile devices fully implemented in the electronic data collection system Research Electronic Data Capture.

We did not perform second-level assessment for the recruited participants who had a diagnosis of dementia that may have introduced differential verification bias.

The specificity of the diagnostic algorithm in our study was lower than in previous studies. We used the same cut-offs for the sensitivity and specificity analysis of previous 10/66 studies. Adjusting the cut-offs for future epidemiological studies may lead to a better balance in the sensitivity and specificity of the short 10/66 dementia diagnostic algorithm, at the detriment of standardisation of and reducing comparability across studies.

Information bias cannot be excluded because the interviewers could not always be blind to the clinical diagnosis of the participants.

Introduction

Dementia is recognised by the WHO as a global public health priority,1 and because its occurrence increases exponentially with age, steep surges in the number of cases are expected in ‘greying’ populations.2 The Italian-speaking regions in southern Switzerland and Italy have already world’s high life expectancies of 853 and 83 years,4 respectively. In these regions and worldwide, the population level needs associated with dementia are high and remain largely unmet. The WHO public health approach to dementia emphasises the importance of increasing healthcare coverage, which is low at the community level, where up to 50% of people with dementia live,5 and in nursing homes,6 and in both low-income and high-income countries.

High-quality epidemiological studies are indispensable to measure the prevalence and impact of dementia and to monitor progress in the reduction of the diagnostic and healthcare coverage gaps7 and can greatly contribute to advance our knowledge and understanding of dementia.8 However, traditional epidemiological studies into dementia have stagnated in the past 20 years in Europe.9 Mobile technologies (ie, tablets and smartphones) can be used to engage with and recruit community-based samples, for data collection and management and can contribute to making dementia ascertainment at the population level less time and resource consuming and to make participation easier, more feasible and sustainable.8

The 10/66 Dementia Research Group (DRG) has conducted extensive cross-country validation studies that confirmed the accuracy of the purposely developed algorithm for dementia diagnosis10–12 and completed numerous surveys on dementia impact in several low-income and middle-income countries (LMICs).13 The 10/66 DRG has recently developed and validated a short dementia assessment schedule,14 which was successfully applied in the Trinidad national survey of ageing and cognition.15 The short-form schedule takes about 15 min with the participant and 10 min with an informant, and it provides the opportunity to conduct dementia studies in nationally representative samples in high-income countries as well. The aim of this study was to conduct an independent validation study of the short 10/66 diagnostic schedule and algorithm and to assess the acceptability and feasibility of an all-electronic, web-based and multilingual data collection platform fully consistent with the 10/66 instruments and data collection procedures.

Methods

We used the Standards for Reporting Diagnostic Accuracy (STARD) guidelines to report our study.16

Study design

We identified two groups of older adults (≥60 years old) with and without a previously established diagnosis of dementia (the reference standard) in two separate sites. In both groups, we used an identical neuropsychological battery (the index test) to assess cognitive functions and to assign a diagnosis of dementia based on a probabilistic algorithm (described further).

Eligibility criteria

Eligible participants were older adults aged 60 years and above, living either in the community or in a nursing home, who also had an informant. An informant was defined as the person who is closest to and knows the participant best (eg, spouse, relative or a carer of community-dwelling older adults). In the context of nursing homes, the informants were identified from the clinical staff, that is, the staff member that was caring for the participant. People with dementia were identified based on previous tests and clinical diagnosis made by a local specialist. We also included people with a clinical diagnosis of mild cognitive impairment (MCI).

Setting, location and dates

We conducted the study in two main settings: the community and nursing homes where older adults lived and in two main Italian-speaking locations, southern Switzerland (Ticino canton) and in northern Italy (Asti, Piedmont region), between March and August 2019.

Participants’ recruitment and sampling strategy

This was a convenience sample, and participation was on voluntary basis. Based on previous evidence,14 17 we calculated that a target sample size of 100 participants (50 people with dementia and 50 controls) was needed to attain a ±5% precision in the psychometric parameters estimations of the new measure. In the Swiss site, we recruited dementia patients from local memory clinics as well as geriatric, neurological and psychiatric services for older adults. Dementia diagnosis was established by specialists in the memory clinics and the other services, independently from the research team and was based on the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) diagnostic criteria.18 Diagnosis of MCI due to Alzheimer’s disease (AD) was also established in line with the National Institute on Aging and Alzheimer's Association (NIA-AA) criteria (AD-MCI).19 The diagnostic procedure did not differ between centres. We matched the sample of clinically diagnosed cases with cognitively healthy controls who volunteered to participate. Cognitively healthy participants and their informants were recruited through standard advertisement and word of mouth through local older adults’ organisations and association, and interested participants contacted us directly.

In the Italian site, we followed a similar recruitment strategy for cognitively healthy, community-dwelling older adults and their informants through older adults’ associations. However, we recruited all people with dementia from two nursing homes who had a diagnosis of dementia in their existing medical records. To facilitate and optimise both recruitment and data collection, we selected nursing homes in which our research team had conducted previous studies and interventions. The clinical evaluation and diagnosis of dementia of nursing homes’ participants was initially established by a general practitioner and was revised and confirmed by attending medical doctor in each nursing home before entering the study. Dementia was diagnosed based on the DSM-IV diagnostic criteria.18

The study interviewers received standard instructions and training to evaluate the eligibility of participants before the conduction of the cognitive assessment interview. Participants in both sites were not asked to bring copies of their medical records to the interview when the 10/66 neuropsychological assessment was conducted. The local research teams had independent access to the clinical diagnosis, to all available and relevant clinical records, and could confirm diagnosis with a next of kin of the participant as needed. For all participants, we recorded their sociodemographic characteristics including age, gender, educational level and marital status. In participants with dementia, we confirmed information with the informant, and we recorded further information on dementia subtypes when available using the accessible records and clinical documentation.

Measurements

The index test was the previously developed and validated 10/66 short dementia diagnostic schedule and algorithm,14 which draws on the output scores of a composite neuropsychological assessment and a brief depressive symptoms scale, based on the following instruments:

The Community Screening Instrument for Dementia (CSI-D) consists of two parts, a participant (32 items) and an informant interview (26 items).20 The CSI-D is a widely used dementia screening instrument based on a culturally unbiased, education-fair comprehensive cognitive assessment, combined with an information questionnaire about objective decline in cognitive functions and functional abilities.21 The total cognitive assessment score (COGSCORE) ranges between 0 (cognitively impaired) and 32 (no cognitive impairment), while the informant’s total score (RELSCORE) ranges between 12 (cognitive impairment) and 0 (no cognitive impairment).

The Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) 10-word list learning task with delayed recall22 consists of asking the participants to recall 10 words that are read aloud at 1 s pace per word. The number of words remembered gives a total score out up to 10 per trial. The immediate recall is the sum of three consecutive trials, and the delayed recall is the sum of number of words recalled after 5 min.

The Euro-Depression (EURO-D) is a 12-item depression screening scale derived from the Geriatric Mental State examination (GMS) for mental disorders specific of older people.23 Each EURO-D item is scored 0 (symptom not present) or 1 (symptom present), and the total score ranges between 0 and 12.

The Italian version of the above-mentioned measures is available in the online supplemental material. The English version of the questionnaires is publicly available on the 10/66 DRG website (https://1066.alzint.org/resources.php). In addition, we used the short, 12-item version of the World Health Organisation Disability Assessment Schedule (WHO-DAS II) to assess disability in all participants. Participants are asked to rate any difficulties associated with health problems on a 1 (none) to 5 (extreme/cannot do) Likert scale, with higher scores indicating higher disability.24

The short 10/66 dementia diagnostic algorithm

The 10/66 dementia case ascertainment methodology has been previously described and validated.12 25 A short version of the 10/66 dementia diagnostic schedule was developed to allow dementia diagnosis in epidemiological studies in which the GMS interview is not possible and has been validated14 using data from the 10/66 survey samples26 and from a population-based study in Singapore.17 Furthermore, it has been successfully used in nationwide surveys.15 We used the same procedures, cut-offs, regression coefficients and statistical computations to assign a probabilistic dementia diagnosis to all participants, using the coefficients from the CSI-D, the modified CERAD 10-word list learning delayed recall score and EURO-D scale.14 The data processing algorithm is publicly available on the 10/66 DRG website (https://1066.alzint.org/resources.php).

Translation

We followed the WHO protocol27 for the Italian translation of the English version of the 10/66 data collection instruments. Two Italian mother tongue, experienced clinical neuropsychologists and fluent in English translated and independently back-translated all materials favouring conciseness and conceptual equivalence rather than literal translation. An expert panel formed of a geriatrician, a neurologist and a psychiatrist, and three clinical neuropsychologists all working in local memory clinics, outpatients old age psychiatric services and nursing homes discussed and resolved discrepancies and inadequate wording. The clinical neuropsychologists administered the instruments to five cognitively healthy older adults for pretesting and discussed any potential difficulties with comprehension debriefing the interviews with the interviewees. Few minor translational improvements were made based on the summary of the problems encountered and were further discussed by the panel members to reach consensus.

Data collection and management

We imported the English original and Italian translated measures to Research Electronic Data Capture (REDCap) and conducted thorough checks and pilot testing that confirmed accuracy and seamless functioning. All study data were collected and managed using REDCap tools hosted in a secure server at Università della Svizzera italiana (USI).28 REDCap is a secure, web-based application designed to support data capture and management for research studies, providing: (1) an intuitive interface for reliable and consistent data entry; (2) audit trails for tracking data manipulation and export procedures; and (3) automated export procedures for data downloads to common statistical packages. The interviewers used mobile devices (ie, tablets and smartphones) to collect data in-person with both participants and informants, either online or with the dedicated REDCap app for offline data collection when internet connection was absent, weak or unstable. REDCap enables secure data collection and storage of data and personal information of participants on separate, remote servers (both hosted at USI). Before the beginning of the study, we confirmed that data collected in the field was seamlessly sent via the internet after encryption to and safely stored in the USI servers.

Finally, at the end of each interview, we asked a set of questions to both participants and informants to explore the acceptability of the all-electronic data collection using mobile devices and the feasibility of the interview if it was perceived as too long and/or tiresome.

Training

We trained all interviewers for both sites using a standard training module based on the original 10/66 manual, which was developed specifically for the short version of the 10/66 dementia diagnostic schedule14 and which was previously used in community settings.15 The manual covers the procedures to administer the cognitive tests (CSI-D and 10-word list learning task) and the Euro-D. The first session of the training aimed at introducing the interviewers to the cognitive assessment instruments and the theory behind them. The second session included practical activities to train the interviewers on administering the cognitive tests using REDCap on mobile devices. The practical session was conducted by two experienced neuropsychologists and the principle investigator (EA), an experienced neuroepidemiologist and member of the 10/66 DRG since 2006. The practical training included a session with a simulated patient. A purposely trained professional actor played the role of the older adult with and without cognitive impairment. The practical training was followed by dedicated sessions of mock interviews between the interviewers that consisted of simulating the entire interview procedure, starting from obtaining signatures on paper copies of the informed consent to using mobile devices for data collection. The final session was dedicated to questions and answers and to the standardisation of data collection using study devised standard operating procedure documents.

Interviews

In the Swiss site, interviews were conducted by four psychology postgraduate students and six junior psychologists from the local neurology and psychiatric services. In the Italian site, two postgraduate students in health sciences from the University of Turin conducted the interviews under the supervision of an experienced psychologist. In both sites, interviews with community-dwelling older adults (with or without dementia) and their informants took place at the participant’s home. Prior to the conduction of the neuropsychological assessments, interviewers did not receive explicit information about the clinical diagnosis of the participant but were not blind to it, neither did they have access to the dementia diagnosis outcome based on the short 10/66 diagnostic algorithm during and after the data collection phase of the study.

Statistical analyses

We carried out descriptive statistics to explore clinical dementia diagnosis across sociodemographic characteristics. Similar to previous validation studies,17 we established the criterion validity of the short 10/66 dementia diagnostic algorithm and calculated its sensitivity, specificity, false positive value, false negative value (FNV), positive predictive value (PPV) and negative predictive value (NPV). We calculated all diagnostic accuracy statistic at 95% CI. In line with a previous study that validated the short 10/66 algorithm in a high-income setting,17 we test the agreement between the gold standard clinical diagnosis and the 10/66 dementia diagnosis by calculating Cohen’s kappa, percentage agreement and area under the receiver operating characteristics curve (AUC) (see online supplemental material for statistical analysis). We repeated this analysis stratified by place of residence to explore differences in the accuracy of the short 10/66 diagnosis between community and nursing homes settings. Moreover, we explored the potential differential effect on diagnoses of age, gender and education, comparing their distributions according to the clinical and 10/66 algorithmic dementia diagnosis. In the main analysis, we considered participants with clinically diagnosed MCI to be not cognitively healthy and excluded participants with MCI in a sensitivity analysis.

In addition, we examined the concurrent validity of the short 10/66 dementia diagnosis with the WHO-DAS II, entering disability scores as the dependent variable in regression models adjusted for age, sex, educational level and place of residence. We used Stata V.15 for all statistical analyses (Stata Corp LP).

Patient and public involvement statement

We involved participants in the piloting phase of the study to explore potential difficulties in comprehending the translated questionnaires and to enquire about the acceptability of the data collection procedures.

Results

Participants

Between March and August 2019, 244 eligible older adults completed the full set of instruments. Fifteen participants (6%) were excluded from the analysis because of missing values on at least one item across instruments. These participants did not differ from the rest of the sample in terms of age, gender, previous diagnosis of dementia or residency. The final analytic sample comprised of 229 participants.

Participants characteristics

Table 1 reports the sample sociodemographic characteristics by clinical diagnosis of dementia. There were 74 (32.31%) previously diagnosed dementia cases, of which 24 cases were AD dementia, 25 were vascular dementia and 25 were of unspecified cause. We included 155 (67.69%) older adults who were classified as cognitively healthy based on the combination of clinical records and self-report of both participants and informants. The sample also included 22 participants with a clinical diagnosis of MCI. Figures 1 and 2 provide a graphical representation of the frequency distributions to illustrate the performance of participants with and without dementia on the word-list recall and the CSI-D. Frequency distributions were provided according to the clinical diagnoses as well as the 10/66 short diagnostic schedule. The histograms of both tests suggest that within the group of people with dementia, some participants have better performance than others which could, to some extent, be a proxy of the severity of dementia.

Table 1

Sociodemographic characteristics of 229 participants across the previous clinical dementia diagnosis

	No dementian=155 (67.7%)	Dementia*n=74 (32.3%)	P value†
Study site			<0.001
Italy (Asti)	41 (26.5)	40 (54.1)
Switzerland (Ticino)	114 (73.6)	34 (46.0)
Living conditions			<0.001
Community dwelling	124 (80.0)	25 (33.8)
Nursing home	31 (20.0)	49 (66.2)
Age group (years)			<0.001
60–74	70 (45.2)	11 (14.9)
75–84	51 (33.0)	28 (37.9)
85+	34 (21.9)	35 (47.3)
Gender			0.369
Men	51 (32.9)	20 (27.0)
Women	104 (67.1)	54 (73.0)
Marital status			<0.001
Never married	11 (7.2)	10 (13.5)
Married	83 (54.6)	21 (28.4)
Widowed	15 (9.9)	4 (5.4)
Divorced/separated	43 (28.3)	39 (52.7)
Educational level			<0.001
None	8 (5.4)	7 (9.9)
Primary	19 (12.9)	33 (47.9)
Secondary	52 (35.4)	21 (29.6)
Tertiary	68 (46.3)	9 (12.7)

Values are frequencies (percentages).

*Clinical diagnosis (excluding MCI participants).

†P value based on χ2 test.

MCI, mild cognitive impairment.

Distribution of word-list recall scores across trials by dementia diagnosis according to clinical or short 10/66 diagnosis.

Distribution of CSI-D scores by dementia diagnosis according to clinical or short 10/66 diagnosis. CSI-D, Community Screening Instrument for Dementia.

Validity of the short 10/66 dementia diagnostic schedule against clinical diagnosis and concurrent validity with WHO-DAS II

The diagnostic accuracy of the short 10/66 dementia diagnostic schedule and algorithm is summarised in table 2, against the clinical diagnosis. We explored the diagnostic accuracy statistics in cognitively impaired participants (dementia and MCI group) compared with the dementia group only. While the sensitivity was higher (87%), specificity was lower when people with MCI were excluded (61%), corresponding to a slightly lower proportion of false negatives, and a slightly higher proportion of false positives, respectively. Overall, the short 10/66 dementia diagnosis showed fair agreement with the clinical diagnosis (kappa=0.40). The algorithm also shows acceptable discriminatory ability (AUC=0.74).

Table 2

Diagnostic accuracy of the short 10/66 dementia diagnostic schedule and algorithm against clinical diagnosis of dementia, including and excluding MCI

	Clinical diagnosis (dementia and MCI group)(n=96)	Clinical diagnosis (dementia group only)(n=74)
Sensitivity	82% (73% to 89%)	87% (77% to 93%)
Specificity	65% (57% to 73%)	61% (53% to 68%)
FPV	35%	39%
FNV	18%	14%
PPV	63% (54% to 72%)	51% (42% to 60%)
NPV	84% (75% to 90%)	90% (83% to 95%)
% agreement	72 (67 to 78)	69 (63 to 75)
Kappa	0.458 (0.346 to 0.569)	0.399 (0.293 to 0.505)
AUC	0.74 (0.68 to 0.79)	0.74 (0.68 to 0.79)

Kappa values: <0=less than chance agreement, 0.01–0.20=slight agreement, 0.21–0.40=fair agreement, 0.41–0.60=moderate agreement, 0.61–0.80=substantial agreement, 0.81–0.99=almost perfect agreement.

95% CIs are reported in parentheses.

AUC, area under the receiver operating characteristics curve; FNV, false negative values; FPV, false positive values; MCI, mild cognitive impairment; NPV, negative predictive values; PPV, positive predictive values.

In the sensitivity analysis, after exclusion of participants with MCI, the short 10/66 dementia diagnosis showed better accuracy in the community setting compared with nursing homes. Sensitivity was 96% and 81%, and specificity was 66% and 39%, respectively, in the former and latter setting (online supplemental table S1) in the supplementary material). Cross-tabulation of the 10/66 algorithm diagnosis by the previous clinical diagnosis is shown in online supplemental table S2 in the supplementary material.

Those with dementia were older, with less education and more likely men compared with those without dementia, and these distributions were non-differential between the clinical and the 10/66 algorithmic diagnostic approach, as shown in figure 3.

Figure 3

Dementia prevalence across age, gender and educational level according to the short version of the 10/66 diagnostic algorithm and the clinical diagnosis (note: prevalence and 95% CIs are from Poisson regressions with robust SEs, adjusted for age, gender and educational level).

Compared with older adults who lived in the community (WHO-DAS II mean=7.08, SE=0.66), disability scores were higher in those who lived in nursing homes (WHO-DAS II mean=23.14, SE=1.29). We found a positive correlation between the short 10/66 dementia diagnosis and the WHO-DAS II disability score, accounting for age, sex, educational level and place of residence (ie, community vs nursing home) (β=5.23, 95% CI 2.05 to 8.41).

Acceptability and feasibility of electronic data collection

Data collection using mobile devices was well accepted by the majority of participants (77%). Twenty-one per cent found it to be excellent and a significant improvement to traditional data collection. Only 2% preferred a traditional pen and paper questionnaire. Data collection, transmission, storage and management in REDCap worked seamlessly and proved to be feasible and efficient. Overall, the mean duration of the interview was 35.6 min (SD=15.4). More specifically, on average, the interview lasted for 18.7 (SD=6.5) min for the cognitive assessments, 8.7 (SD=6.2) min for the Euro-D and 8.8 (SD=6.9) min for the WHO-DAS II questionnaire. Most participants (91 %) found the duration of the interview acceptable and not fatiguing, and only 4% of participants complained about the setting of the interview (ie, too noisy or distracting, or lacking privacy).

Discussion

In this study, we investigated the criterion and concurrent validity of the short version of the 10/66 dementia diagnostic schedule against the clinical diagnosis in an independent sample of older adults living in the community or in nursing homes. Our findings suggest that the short 10/66 schedule retains its criterion validity to identify dementia cases among older adults living in the community as well as in nursing homes, in two high-income countries. We also found that our innovative all-electronic data collection system implemented on portable devices was efficient, reliable and highly accepted by older adults.

The 10/66 DRG has conducted numerous population-based studies into dementia, mainly in LMICs,13 with few notable exceptions including in Portugal29 and Singapore.17 The 10/66 original diagnostic algorithm was validated in 15 countries for use in international epidemiological research, providing strong support for the robustness and comparability of the epidemiological findings of the 10/66 surveys across continents.12 However, the 10/66 diagnostic algorithm required assessments were deemed too long to allow wide use, particularly in national censuses. Moreover, the duration of the original 10/66 interviews may pose constraints on the conduction of epidemiological studies in dementia, including in high-income countries due to the high costs of data collection. With this in mind, a short version of the 10/66 schedule was developed and validated using data from the cross-sectional phase of the original 10/66 surveys.14 A similar approach was used in other settings, where epidemiological data on dementia prevalence had been previously collected.15 17 Nevertheless, only one study from Singapore has been so far purposely designed and conducted to test the criterion validity of the short 10/66 schedule against a clinical diagnosis of dementia.17 This approach is standard and less prone to bias due to circularity, and it was used in the original validation of the full 10/66 dementia diagnostic algorithm.12 Neither the full nor the short version of the 10/66 diagnostic schedule has been validated or used in nursing homes, where up to 50% of residents may have dementia30 and where a consensus diagnosis based on existing medical records is typically used to adjudicate dementia status.31 In addition, although electronic data collection with laptops was used in the 10/66 Cuban site by local physicians for the prevalence and incidence surveys,32 an all-electronic, online data collection and management system using mobile devices was not previously used across the 10/66 sites. Our findings on the high acceptability, feasibility and efficiency of this all-electronic, 10/66 fully compliant data collection system provides evidence on the robustness of the 10/66 DRG procedures using new technologies.

Our study extends evidence on the validity of the 10/66 methods and probabilistic dementia diagnostic approach and algorithm from LMICs to high-income countries (ie, Switzerland and Italy) and to Italian language. Furthermore, we found that the short 10/66 schedule has very good convergent validity with a standard measure of disability (WHO-DAS II) and acceptable criterion validity when compared with clinical diagnosis of dementia in older adults who live both in the community and in nursing homes. This may be important for epidemiological research and for monitoring and screening purposes, because dementia occurrence and impact in residential care facilities are significantly underestimated due to the lack of a standard, practical and yet fairly valid approach to diagnosis. Multiphase designs, in which screening tools are combined with in-depth assessments, have been used on the ground of their apparent efficiency.33 Previous studies in Italy used a combination of routinely collected data complemented with assessments of cognitive function, functional activities and depressive symptoms.34 35 Some studies included costly case finding procedures that combined neuroimaging, blood and urine testing to generate a probabilistic dementia diagnosis.36 37 By using the short 10/66 schedule, we retain the assessment of cognitive functions, accounting for depressive symptoms and functional ability by combining instruments in a parsimonious ascertainment schedule.34–36 Moreover, the short 10/66 schedule, unlike previous approaches,35 36 includes a structured informant interview as part of the assessment and case identification, which has shown to improve both its sensitivity and specificity and further reduces education bias in diagnosis in participant’s with or without dementia.12 14 Importantly, compared with DSM diagnostic criteria,18 38 the 10/66 dementia diagnosis is significantly less prone to under-reporting of social impairment and cognitive decline by informants, which tends to be high where dementia awareness is low.10 Although age and education differences between participants may have differentially impacted dementia ascertainment, both the short and the standard version of the 10/66 algorithm have similar sensitivity (94% in both) and specificity in low (93%, 94%) and high education (97%, 97%) groups, respectively.14 In addition, as mentioned above, the informant interview further reduces education bias.

Some limitations of our study are worth noting. Although the sensitivity was comparable and adequate (86.5%), the specificity was lower (60.6%) in our study compared with previous validation studies of the short 10/66 diagnostic algorithm.14 17 We included older adults who live in nursing homes, and the ability of the short 10/66 algorithm to correctly identify people without dementia was somewhat lower than expected. However, the 10/66 algorithm was designed, validated for and has been used in community samples.12 14 15 Moreover, in our study, most dementia diagnoses against which we compared the accuracy of the short 10/66 diagnosis were made by specialists and in highly specialised memory clinics. In these settings, differential diagnosis is often integrated with and informed by various kinds of biomarkers and structural and functional neuroimaging assessments. Diagnosis is then refined, and dementia may be excluded despite overt and objective cognitive and functional decline. Nonetheless, the algorithm’s low specificity might imply that people without dementia could be erroneously identified as dementia cases. However, it is important to underline that the participants do not receive the results of the cognitive assessment at the end of the interview, and a diagnosis of dementia should only be carried out by a trained medical professional based collectively on medical history and physical examination. Therefore, careful precautions should be applied in future studies, and communication of individual results to participants may be disclosed only on approval of a competent research ethic body. In cases of a positive dementia ascertainment by the algorithm, and in order to improve the diagnostic procedure for future application of the algorithm, we propose to provide a predefined protocol to refer participants for further investigation by a clinician in case of being identified as a dementia case by the algorithm. Finally, the reported positive predictive value of the index test was not solely affected by specificity but also by the base rate of dementia in the present study sample.

Because the 10/66 diagnosis is syndromic and purposely symptoms and needs centred, we maintain that our results provide empirical support for its construct validity despite the lower specificity. In the current study, we follow the same procedure and cut-offs for the specificity and sensitivity analysis of the 10/66 algorithm to allow for direct comparison with previous studies and findings.12 14 17 However, an adjustment to the original 10/66 cut-offs may be considered for a better balance in the sensitivity and specificity analyses of future studies, in which the test performance may be accounted for in prevalence calculations.

One of the strengths of our study is the inclusion of people with dementia whose severity of symptoms ranged from very mild to moderate based on their performance on the cognitive assessment of the 10/66 schedule (figures 2–3). Indeed, it is important to differentiate people with MCI from those with dementia in both the community and nursing home settings. In the sensitvitiy analysis in our study, the algorithm showed better accuracy when people with MCI were excluded. This adds confidence in our results on the relatively high sensitivity of the short 10/66 diagnostic algorithm because it is likely that the performance on cognitive tests of numerous participants with an existing clinical dementia diagnosis was only slightly below normative values. This is important also because the onset of dementia with, for example, psychological symptoms (including apathy) may precede cognitive disturbances and objective memory decline.39 Because dementia has an insidious progressive nature, and the diagnostic gap of dementia is likely high in Switzerland6 and Italy,40 we cannot exclude with certainty that those whom we included as ‘controls’ may in fact already have dementia. That we did not perform a second-level clinical assessment for the recruited participants is another important limitation of our study. However, although this approach is standard and seemingly more robust, it may be affected by spectrum bias.41 42 In other words, it is prone to an overestimation of the true performance of the ‘index test’ (ie, the short 10/66 schedule) because certain cases are compared with certain non-cases. Studies performed on a population that lacks diagnostic uncertainty may produce a biased estimate of the ‘new’ test’s performance relative to a study restricted to people for whom the test (our diagnostic algorithm) would be indicated. The ideal population should include only people with true diagnostic uncertainty, which was the case in our study. While our design and approach reduced the likelihood of spectrum bias, it is prone to differential verification bias due to the use of two different reference tests for at least some of our cases and controls.43 In fact, differential verification bias may explain why specificity was lower compared with previous studies. As said, while cases were more strictly defined, some of those classified and used in the analysis as ‘controls’ might have been already affected by dementia. The true number of false positives could be much lower (and thus specificity higher) than what we found.

Training of interviewers and the relatively short duration of interviews with both older adults and informants suggest that the ‘short’ 10/66 schedule and diagnostic algorithm can be used in large scale, population-wide, nationally representative samples of older adults to ascertain dementia prevalence in the community and nursing homes in high-income countries, where epidemiological research on dementia stagnate. Moreover, our results suggest that an electronic data collection system may facilitate the standardisation and quality monitoring of data collection, without requiring data entry, and simplifying data cleaning and management. Because this could be integrated in routine electronic medical records systems, using the short 10/66 diagnostic approach may be promising beyond research purposes. Research is warranted, though, to explore whether this innovative data collection approach can contribute to reducing the current dementia diagnostic gap through an integration of its use at the primary care level and in general practitioners’ clinics by purposely trained non-specialist health workers and in nursing homes.

Conclusion

The short 10/66 diagnostic schedule is a valid tool that is also practical, cost-effective, short to administer and highly acceptable also in a high-income setting, where epidemiological evidence on dementia is lacking or outdated. Our findings on the validity of the short 10/66 diagnostic schedule and the feasibility of electronic data collection may have positive implications for epidemiological research in comparable settings. Moreover, they can contribute to conduct studies aimed at measuring the impact of dementia and contextually the gap in dementia diagnosis and care, and thus reduce the burden that dementia poses on those who are affected, their family, communities and society at large.