Reliability and validity of the severe impairment battery in Taiwanese patients with moderate to severe Alzheimer's disease.

BACKGROUND: The severe impairment battery (SIB) was developed to evaluate cognitive functions in moderate to severe dementia patients. We aimed to examine the reliability and validity of the Taiwanese version of the SIB (T-SIB) in patients with moderate to severe Alzheimer's disease (AD).
METHODS: AD patients with clinical dementia rating (CDR) stage 2 (n = 79) or 3 (n = 21) and scores <15 on the Taiwanese version of mini mental state examination (T-MMSE) were recruited from six hospitals in Taiwan. Cronbach's alpha was used to evaluate the internal consistency of the T-SIB. The CDR and functional assessment staging (FAST) scores were used to assess dementia severity.
RESULTS: We recruited 100 AD patients (73 women and 27 men; mean T-SIB score, 56.4 ± 24.8). The mean T-SIB total score for patients with CDR 2 and 3 were 60.3 ± 23.3 and 41.2 ± 24.9, respectively. The internal consistency of the T-SIB was 0.96. The T-SIB was moderately correlated with the T-MMSE (Pearson's correlation coefficient = 0.76). The areas under the curve for discriminating between CDR 2 and CDR 3 were 0.81 (95% CI = 0.91-0.71) and 0.72 (95% CI = 0.84-0.61), respectively. Using a cut-off score of 59, the T-SIB had a sensitivity of 80% and specificity of 61% for discriminating between CDR 2 and CDR 3. Using a cut-off score of 45, the T-SIB had a sensitivity of 83.3% and specificity of 73.1% for discriminating between the FAST stage 7c.
CONCLUSION: T-SIB is a reliable and valid instrument for measuring cognition of severely demented Taiwanese AD patients.

1. INTRODUCTION

Alzheimer’s disease (AD) is the most common cause of dementia in the elderly, affecting >46 million people worldwide.[1] Tracking preserved abilities and functions can provide essential information for patient care as many patients with AD rely heavily on assisted care.[2] Accurate diagnosis of late stage AD has become increasingly important as drug treatments for this group of patients may become available.[3] However, several cognitive tests, such as the mini mental state evaluation (MMSE),[4] the neurobehavioral cognitive status examination,[5] and the 7-min screening test,[6] were designed to assess patients with mild to moderate cognitive impairment. Most patients with dementia of moderate to severe degree performed poorly on these assessment tools due to the impaired expression and language skills associated with moderate to severe AD.[7] It is therefore challenging to evaluate cognition in patients with moderate to severe dementia.

Several instruments have been developed to assess the cognitive function of patients with moderate to severe dementia, including the Gottfries-Brane-Steen scale,[8] the test for severe impairment,[9] the severe cognitive impairment profile,[10] the preliminary neuropsychological battery,[11] and the severe impairment battery (SIB).[7] Due to the limited comprehension and language skills, patients with moderate to severe AD may have difficulty completing more challenging standard neuropsychological assessments. The SIB was specifically designed and validated to assess cognition in this group of patients.[7] In addition to using simple verbal commands with gestures, nonverbal and partially correct responses from the patients are taken into account in the SIB, allowing for a finer assessment of cognitive function.[12] It was also more apt for identifying differences in performances resulting in scores in the 5 to 10 range in the MMSE, thus, avoiding floor effects. The SIB is now a standard and commonly used assessment tool in clinical trials involving patients with moderate to severe AD.[13]

French, Italian, Norwegian, Korean, and Greek versions of the SIB (both the full version and revised forms) have been validated in a number of studies.[14-18] These studies have demonstrated different versions of SIB to be reliable and valid in differentiating patients with moderate to severe dementia. A revised short-form of the SIB had also being validated for a Korean population, with a clear definition of recruited participants as clinical dementia rating (CDR) stages 2 or 3 and scores of <15 on the MMSE.[16] In addition, a French study using factor analysis for the SIB domains in relation to other measures of functioning found that impairments in cognition and function may be interrelated in patients with moderate to severe AD.[14] However, the inclusion criteria for the participants was diverse in the previous studies, and no study has been conducted to validate the Taiwanese version of the SIB (T-SIB) in a Taiwanese patient population. According to the Taiwan Ministry of Health and Welfare and the Taiwan Ministry of the Interior, there were about 269 725 demented patients in Taiwan by the end of 2018, with 31% severely demented patients.[19] A standardized Chinese neuropsychological tool that can assess cognitive functions in severely demented AD patients is needed. The aim of this study is to examine the reliability and validity of the SIB in Taiwanese patients with moderate to severe AD for the purpose of adapting this test for AD patients in Taiwan.

2. METHODS

2.1. Subjects

Participants were enrolled in this multi-center, nonintervention observational study. Six hospitals participated in this study: Taipei Veterans General Hospital, Taipei Medical University Shuang-Ho hospital, Kaohsiung Veterans General Hospital, Kaohsiung Municipal Ta-Tung hospital, Changhua Christian Hospital, and Taichung Veterans General Hospital. The clinical history of participants was examined. They were diagnosed as AD according to the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA).[20] Participants also underwent a physical and neurological examination and neuroimaging (computed tomography [CT] or magnetic resonance imaging [MRI]) to exclude non-AD types of central nervous system pathology. Additional investigations were carried out when necessary (e.g. complete blood count, serum B12 or folic acid, thyroid hormone levels, and routine biochemical tests). All participants were required to have a Taiwanese version of mini mental state examination (T-MMSE) score of 0 to 14 and a CDR score of 2 to 3, and they had to undergo T-MMSE and CDR tests during the screening stage to ensure eligibility for participation in the study. Also, participants had to have the ability to read and write traditional Chinese before the onset of dementia. Participants with other neurologic disorders, psychiatric disorders, a history of head trauma, alcohol use, other illicit drug use or clinically significant medical conditions were excluded.

2.2. Ethics statement

The Ethics Committee and Institutional Review Board of all participants’ hospital approved the study. All dementia patients and their proxies provided written informed consent.

2.3. Assessment instruments

2.3.1. MMSE

The MMSE[4,21] is an assessment instrument that measures cognitive functions such as orientation, registration, attention and calculation, memory, and language abilities. The MMSE contains 11 questions and can be administered in 5 to 10 minutes, making it a quick and easy method for assessing mental functions.

2.3.2. CDR

The CDR[22] evaluates six cognitive categories (memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care) through a semi-structured interview with both the patient and a reliable informant. Cognitive impairment levels of patients are assessed on a five-point scale from 0 to 3 (0 = no dementia, 0.5 = questionable, 1 = mild dementia, 2 = moderate dementia, and 3 = severe dementia).

2.3.3. SIB

The SIB is a tool for assessing cognitive impairment through low-level tasks in severely demented patients who may be unable to complete conventional neuropsychological tests.[23] Apart from conventional problem-solving questions and tasks, one of the strengths of the SIB is that it also scores behavioral observations of effortful and automatic patient responses.[24] SIB question items may be presented with gestures, and questions can be repeated if necessary to aid comprehension. Hints may be provided if the patient is unable to answer the question. Partially correct or nonverbal answers are credited. The T-SIB can thus avoid the floor effect that was frequently encountered in conventional neuropsychological tests. The current version of the SIB consists of 40 questions and tasks. The SIB takes 20 minutes to administer and scores range from 0 to 100. The SIB contains nine subscales, namely social interaction (0-6 points), memory (0-14 points), orientation (0-6 points), language (0-46 points), attention (0-6 points), praxis (0-8 points), visuospatial ability (0-8 points), construction (0-4 points), and responding to name (0-2 points). The English version SIB was translated into complex Chinese/Taiwanese and independently translated back into English by a board-certificated psychologist and a neurologist. Under bilingual expert panel evaluation, items on the Taiwanese version were identical to the English version.

2.3.4. Functional assessment staging

Functional assessment staging (FAST) is a well-validated measure and is used to assess the stage of dementia of a person with AD. It was designed to evaluate participants at the moderate-to-severe stages of dementia. It focuses more on an individual’s level of functioning and the activities of daily living than on cognitive decline. The FAST has seven stages (7 = severe dementia) and its scores can be assigned by interviewing a caregiver who is an accurate reporter, the patient (if at a high cognitive stage), and/or by watching the patient engage in activity.[25]

2.4. Statistical analysis

To test for differences between the patients’ characteristics, we used the t test for independent groups and the chi square test. The Cronbach’s alpha and item-total correlation coefficients were generated to examine the internal consistency of the T-SIB. The Mann-Whitney U test was used to assess the differences in the mean T-SIB total score and its subscales for the two CDR groups and various FAST stages. The receiver operating characteristic (ROC) curve was used to determine the degree to which the T-SIB allows the dementia severity to be discriminated (CDR 2 vs CDR 3), and to find the cut-off values between different CDR groups. The sensitivity and specificity levels of the T-SIB were also compared with those of the T-MMSE. The ROC curves for patients at different FAST stages were also generated to examine the discriminatory ability of T-SIB and T-MMSE. Statistical significance was set at p < 0.05. All data processing and statistical analyses were performed with SPSS version 17 (SPSS Inc.) and Statistical Analysis Software (SAS) version 9.1 (SAS Institute, Cary, NC).

3. RESULTS

3.1. Demographic characteristics

A total of 100 patients (73 women and 27 men) with AD were enrolled in our study, with an average age of 81.2 ± 7.0 years (Table 1). Among them, 30 patients had hypertension and five had type 2 diabetes mellitus. Their mean years of education were 5.6 ± 5.0 years. The mean score for the T-SIB was 56.4 ± 24.8 and that for the T-MMSE was 6.8 ± 3.8. We found no significant differences in test scores between demographic groups in terms of gender, level of education, and age (Table 2).

Table 1

Demographic characteristics and mean scores on the T-SIB and T-MMSE for the total sample

	n (%) or mean ± SD	Range
Age	81.2 (7.0)	65-96
Male	27 (27%)
Education (year)	5.6 (5.0)	0-16
Duration of AD (month)	37.1 (28.1)	0-146
T-SIB	56.4 (24.8)	0-94
T-MMSE	6.8 (3.8)	0-14

AD= Alzheimer’s disease; T-MMSE = Taiwanese version of mini mental state examination; T-SIB = Taiwanese version of severe impairment battery.

Table 2

Total scores of T-SIB (mean, SD and p-value) stratified by gender, education and age

	N	Mean SIB score (SD)	p-value
Male	27	59.4 (27.8)	0.520
Female	73	55.8 (23.4)
≤6 years of education	65	58.1 (23.2)	0.350
>6 years of education	35	53.2 (27.5)
Age < 85 years	67	54.9 (25.2)	0.426
Age ≥ 85 years	33	59.9 (24.0)

T-SIB = Taiwanese version of Severe Impairment Battery.

3.2. Reliability

The Cronbach’s alpha coefficient for the T-SIB was 0.96. The item-total correlation ranged from 0.18 (question number 14—examiner’s name delayed) to 0.76 (question number 34a—shape identification ‘circle’) and was statistically significant (p < 0.01).

3.3. Validity

The Spearman correlation coefficients between the T-SIB and T-MMSE, T-SIB and CDR, and T-SIB and FAST were 0.76 (p < 0.001), –0.51 (p < 0.001), and –0.50 (p < 0.001), respectively.

3.4. T-SIB scores according to dementia severity

To compare T-SIB scores of participants with differing T-MMSE scores, we separated participants into two groups according to their T-MMSE scores: Group 1 included participants scoring from 7 to 14 on the T-MMSE (n = 59) and Group 2 included participants scoring from 0 to 6 on the T-MMSE (n = 41) (Table 3). The mean T-SIB score for Group 1 was 68.5 ± 18.0, and the mean T-SIB score for Group 2 was 38.5 ± 22.6. The results of paired comparisons between the two groups showed differences in total T-SIB scores and all T-SIB subscale scores. All participants were also separated into two groups according to their dementia severity using the CDR with the aim of determining the total T-SIB score and subscale scores (Table 4). As a result, the difference between the total T-SIB score and its subscale scores in the CDR 2 and CDR 3 groups was significant, except for the visuospatial ability and orienting to name subscales.

Table 3

SIB subscales mean scores (±SD) for all patients and results of paired comparisons between two different MMSE severity group

	Total patients (N = 100)	MMSE 7-14 (N = 59)	MMSE 0-6 (N = 41)
Social interaction**	4.4 ± 1.9	4.9 ± 1.3	3.7 ± 2.3
Memory***	5.1 ± 3.8	6.9 ± 3.3	2.5 ± 2.7
Orientation***	2.4 ± 1.4	2.9 ± 1.1	1.6 ± 1.3
Language***	27.3 ± 12.4	33.4 ± 7.9	18.5 ± 12.5
Attention***	3.6 ± 2.1	4.5 ± 1.6	2.3 ± 2.1
Praxis***	4.6 ± 2.8	5.7 ± 2.3	3.0 ± 2.6
Visuospatial ability***	4.6 ± 2.9	5.5 ± 2.5	3.2 ± 2.8
Construction***	2.4 ± 1.6	2.9 ± 1.5	1.6 ± 1.4
Orienting to name**	1.5 ± 0.8	1.6 ± 0.6	1.2 ± 0.8
Total SIB scores***	56.4 ± 24.8	68.5 ± 18.0	38.5 ± 22.6

T-SIB = Taiwanese version of severe impairment battery; T-MMSE = Taiwanese version of mini mental state examination.

**p < 0.01, ***p < 0.001.

Table 4

SIB subscales mean scores (±SD) for all patients and results of paired comparisons between CDR 2 and 3

	Total patients (N = 100)	CDR 2 (N = 79)	CDR3 (N = 21)
Social interaction**	4.4 ± 1.9	4.7 ± 1.7	3.2 ± 2.1
Memory***	5.1 ± 3.8	5.8 ± 3.7	2.4 ± 2.5
Orientation*	2.4 ± 1.4	2.5 ± 1.3	1.7 ± 1.4
Language**	27.3 ± 12.4	29.4 ± 11.3	19.4 ± 13.4
Attention*	3.6 ± 2.1	3.8 ± 2.0	2.6 ± 2.1
Praxis*	4.6 ± 2.8	5.0 ± 2.7	3.3 ± 2.7
Visuospatial ability	4.6 ± 2.9	4.8 ± 2.7	3.6 ± 3.2
Construction*	2.4 ± 1.6	2.6 ± 1.6	1.6 ± 1.5
Orienting to name	1.5 ± 0.8	1.5 ± 0.7	1.2 ± 0.8
Total SIB scores**	56.4 ± 24.8	60.3 ± 23.3	41.2 ± 24.9

CDR = clinical dementia rating; T-SIB = Taiwanese version of severe impairment battery.

*p < 0.05, **p < 0.01, ***p < 0.001.

3.5. Discriminating power of T-SIB

ROC curves show the relationship between 1-specificity and sensitivity. As shown in Fig. 1, ROC curves can be used to determine the cut-off points between the two groups. It identifies the coordinates of the ROC curve, which gives the best sensitivity and specificity. The area under the curve (AUC) for discriminating between CDR 2 and CDR 3 following the T-MMSE was 0.81 (95% CI = 0.91–0.71) and the AUC following the T-SIB was 0.72 (95% CI = 0.84–0.61). The p-value for the difference between the AUC of the two tools was 0.24. Using an optimal cut-off score of 59, the T-SIB had a sensitivity of 80% and specificity of 61% for discrimination between CDR 2 and CDR 3. The AUC revealed that the T-SIB (AUC = 0.85, 95% CI = 0.99–0.72) was more accurate than the T-MMSE (AUC = 0.73, 95% CI = 0.98–0.49) in differentiating FAST stage ≤7c (cannot walk without personal assistance) and FAST stage >7c (cannot sit up without assistance) groups, with a p-value of 0.39. Using an optimal cut-off score of 45, the T-SIB had a sensitivity of 83.3% and specificity of 73.1% for discriminating between FAST stages ≤7c and FAST stages >7c.

Fig. 1

ROC curves of the T-SIB and T-MMSE for discriminating between CDR 2 and CDR 3, and between FAST ≤ 7c and FAST > 7c. (a) ROC curve for CDR 2 vs CDR 3, identifying the coordinates X = 0.177 and Y = 0.7 giving the best cut-off points of T-MMSE with a sensitivity of 0.7 and a specificity of 0.823. Coordinates X = 0.392 and Y = 0.8 giving the best cut-off points of T-SIB with a sensitivity of 0.8 and a specificity of 0.608. (b) ROC curve for FAST ≤ 7c vs FAST > 7c, identifying the coordinates X = 0.269 and Y = 0.833 giving the best cut-off points of T-SIB with a sensitivity of 0.833 and a specificity of 0.731. CDR = clinical dementia rating; FAST = functional assessment staging; MMSE = mini mental state examination; ROC = receiver operating characteristic; SIB = severe impairment battery.

Subanalysis that stratified patients into different education level subgroups was performed to investigate whether education will influence the discriminatory ability of T-SIB. As shown in Fig. 2, the AUC for discriminating between CDR 2 and CDR 3 following the T-MMSE was 0.85 (95% CI = 0.70-0.99) and the AUC following the T-SIB was 0.84 (95% CI = 0.71-0.94) among 63 patients with <6 years of education. The AUC for discriminating between CDR 2 and CDR 3 following the T-MMSE was 0.82 (95% CI = 0.66-0.98) and the AUC following the T-SIB was 0.82 (95% CI = 0.71-0.93) among 56 patients with <3 years of education. There were no significant differences between the AUC of the two tools.

Fig. 2

ROC curve of the T-SIB and T-MMSE for discriminating between CDR 2 and CDR 3. (a) Among patients with <6 years of education, the area under the curve (AUC) for discriminating between CDR 2 and CDR 3 following the T-MMSE was 0.85 (95% CI = 0.70-0.99) and the AUC following the T-SIB was 0.84 (95% CI = 0.71-0.94). (b) Among patients with <3 years of education, the AUC for discriminating between CDR 2 and CDR 3 following the T-MMSE was 0.82 (95% CI = 0.66-0.98) and the AUC following the T-SIB was 0.82 (95% CI = 0.71-0.93). AUC = area under the curve; CDR = clinical dementia rating; ROC = receiver operating characteristic; T-MMSE = Taiwanese version of mini mental state examination; T-SIB = Taiwanese version of severe impairment battery.

4. DISCUSSION

The reliability and validity of the T-SIB were examined in this study, and its clinical utility for discriminating dementia severity was confirmed using the ROC curve. Our results indicated that the T-SIB is a reliable and valid instrument for assessing the cognitive functions of severely demented Taiwanese AD patients. Additionally, the T-SIB can be useful for measuring cognitive impairment in a clinical setting.

The Cronbach’s alpha coefficient for the total T-SIB score was 0.96, suggesting that the T-SIB has an excellent level of reliability. Our Cronbach’s alpha coefficients were in agreement with those reported in previous studies including the Korean, Norwegian, and Greek validation study (0.98, 097, and 0.904, respectively).[15,17,18] The item-total correlation for T-SIB showed significance, indicating that T-SIB has good internal consistency.

The correlation between the T-SIB and T-MMSE suggested that the T-SIB has appropriate construct validity. Other studies have compared the SIB to the MMSE, and reported correlation between the two tests as 0.73,[26] 0.82,[27] and 0.87,[18] which were consistent with our findings. In particular, previous validation studies for MMSE conducted in the community have shown that education can influence the test scores, emphasizing the need for education-adjusted cut-offs.[28] In our study, we found that T-SIB and T-MMSE had no differences in discriminating between moderate-stage (CDR = 2) and advanced-stage (CDR = 3) patients of different education level by using the ROC curve. The AUC following T-SIB and T-MMSE all showed excellent discriminating power among patients with <6 years or <3 years of education.[29] These results suggest that the T-SIB, which is independent of educational influence, would be a useful tool in countries where elderly participants have relatively low levels of education. A previous validation study in Korea also compared the CDR to the SIB and reported a Spearman correlation coefficient of –0.67 for the total SIB score.[16] We found a Spearman correlation coefficient of –0.55, which is more comparable to the aforementioned Spearman correlation coefficient than that reported by Bergh et al.[15]

Comparisons of the T-SIB and the T-MMSE revealed that the T-SIB is valid and capable of distinguishing between participants who scored between 0 to 6 and 7 to 14 for all subscales of the T-SIB. This finding indicated that the T-SIB can be used in severely impaired patients who score <15 on the MMSE. This suggests the T-SIB has a strong power for differentiating moderate to severe demented patients compared to what was reported in a previous study.[27] Comparisons of the T-SIB and the CDR showed that the T-SIB is capable of distinguishing between CDR 2 and CDR 3 patients, although significant differences between groups could only be found for some of the subscales during this comparison (Table 4). Similar to a study that validated the short version of the SIB,[16] our study only included patients with moderate and advanced stage dementia (CDR = 2 and CDR = 3). However, while the aforementioned study showed significant differences between all subscales apart from orienting to name, our study showed significant differences in all subscales except for visuospatial ability and orienting to name. It is possible that the difference in the orienting to name subscale score was not obvious because recognition of self is relatively maintained until the advanced stages of dementia. David et al[30] demonstrated that visuospatial function is a relatively underreported symptom, which relies on parietal lobes structures that are damaged in early-stage AD, followed by a decline in global cognitive function during the subsequent year. In addition, CDR contains no items for examining visuospatial skills. Comparisons of T-SIB between patients with moderate dementia (CDR = 2) and those with advanced dementia (CDR = 3) showed no significant difference. These aspects of the comparative decline in various cognitive domains are consistent with the clinical course of AD.

Using the ROC curve, we found that T-SIB and T-MMSE had similar sensitivity and specificity for discriminating between moderate-stage (CDR = 2) and advanced-stage (CDR = 3) patients, even though T-SIB might be more useful for discriminating between FAST ≤7c and FAST >7c. However, this trend did not reach statistical significance. The FAST scale relatively correlated with activity of daily living scales, and it is capable of describing the entire course of brain-ageing-associated instrumental and functional capacities and subsequent AD.[31] In a previous study involving patients with advanced AD with MMSE score <10, it was shown that FAST staging procedures but not MMSE scores could demonstrate progressive change in these patients.[32] Moreover, the FAST scale identifies substages 6d to 7, which correspond to CDR stage 3, providing a more detailed staging for severe AD.[33] These results suggest that the T-SIB might be useful for evaluating AD patients with severely impaired functioning.

This study has several limitations. First, the design of our study was cross-sectional. In future, longitudinal studies should be conducted to examine the ability of the T-SIB to measure cognitive decline as AD progresses. Although we did not assess the traditional Chinese version of the SIB over the long term, Schmitt et al[7] reported that the SIB is highly reliable and valid in the long-term assessment of patients with AD. Second, we recruited a small number of male patients. However, there were no differences in T-SIB scores between different gender groups. The T-SIB also showed only a minimal change in correlation with FAST staging when it was adjusted for gender.

We conclude that the Taiwanese version of the SIB is reliable and can be used to evaluate cognitive function in Taiwanese patients with moderate to severe AD, and to discriminate between patients with dementia of moderate and severe degrees.

ACKNOWLEDGMENTS

The study was supported in part by grants from the Ministry of Science and Technology of Taiwan (108-2321-B-075-001) and Taipei Veterans General Hospital (V109C-061 and VGHUST109-V1-5-1).