Preliminary findings on the reliability and validity of the Cantonese Birmingham Cognitive Screen in patients with acute ischemic stroke.

BACKGROUND: There are no currently effective cognitive assessment tools for patients who have suffered stroke in the People's Republic of China. The Birmingham Cognitive Screen (BCoS) has been shown to be a promising tool for revealing patients' poststroke cognitive deficits in specific domains, which facilitates more individually designed rehabilitation in the long run. Hence we examined the reliability and validity of a Cantonese version BCoS in patients with acute ischemic stroke, in Guangzhou.
METHOD: A total of 98 patients with acute ischemic stroke were assessed with the Cantonese version of the BCoS, and an additional 133 healthy individuals were recruited as controls. Apart from the BCoS, the patients also completed a number of external cognitive tests, including the Montreal Cognitive Assessment Test (MoCA), Mini Mental State Examination (MMSE), Albert's cancellation test, the Rey-Osterrieth Complex Figure Test, and six gesture matching tasks. Cutoff scores for failing each subtest, ie, deficits, were computed based on the performance of the controls. The validity and reliability of the Cantonese BCoS were examined, as well as interrater and test-retest reliability. We also compared the proportions of cases being classified as deficits in controlled attention, memory, character writing, and praxis, between patients with and without spoken language impairment.
RESULTS: Analyses showed high test-retest reliability and agreement across independent raters on the qualitative aspects of measurement. Significant correlations were observed between the subtests of the Cantonese BCoS and the other external cognitive tests, providing evidence for convergent validity of the Cantonese BCoS. The screen was also able to generate measures of cognitive functions that were relatively uncontaminated by the presence of aphasia.
CONCLUSION: This study suggests good reliability and validity of the Cantonese version of the BCoS. The Cantonese BCoS is a very promising tool for the detection of cognitive problems in Cantonese speakers.

Introduction

There is a high incidence of stroke in the People’s Republic of China, with the age-standardized annual incidence of first stroke among Chinese individuals aged 45–74 years falling between 205 and 584 per 100,000.1 Stroke is a common cause of cognitive impairment.2 Prior studies have shown that between 44% and 74% of patients present with some degree of cognitive disturbance when tested 6 months after their stroke – with the incidence dependent on the type of cognitive problem and the form of measurement.3 Since early intervention can improve outcome,4,5 it is important to test stroke patients and to detect cognitive impairments in the subacute stage.

The Mini Mental State Examination (MMSE)6 and the Montreal Cognitive Assessment Test (MoCA)7 are probably the most commonly used cognitive screening tests in the People’s Republic of China, if not worldwide. However, the MMSE lacks sensitivity to detect patients with mild symptoms, especially those with visuospatial and executive function impairments.8 The MoCA, though more sensitive than the MMSE,9 remains limited. For example, being developed for dementia, the MoCA is not “tuned” to detect some of the common cognitive problems after stroke – notably visuospatial neglect and apraxia.10–13 Moreover, both the MMSE and the MoCA are highly verbal screens, with the majority of tests requiring good verbal abilities. This means that patients with aphasia found after stroke can fail the tests for reasons to do with language impairments and not the putative factors being tested (eg, memory or attention). There is also a lack of specificity in assigning a problem to a specific cognitive function. These screens typically return an overall score, which cannot be easily used to direct treatment, as treatment needs to target the specific cognitive problem (eg, therapy for spatial neglect differs from that for apraxia).

The Birmingham Cognitive Screen (BCoS) battery14 is a new instrument that was developed to screen individuals for cognitive problems after a stroke and to generate a cognitive profile across a range of cognitive domains (Attention and Executive Function; Language; Memory; Number Skills; and Praxis). Importantly, the test is designed to maximize inclusion of stroke patients, while generating test results that are uncontaminated by the co-occurrence of language or spatial attention problems, which can otherwise have a contaminating impact on performance. The test is made “aphasia friendly” by including short, high-frequency words, whenever possible, when language is not directly assessed and by incorporating forced-choice response options (so aphasic patients can point even when they cannot make a verbal response). The tests are “neglect friendly” in their spatial design (the stimuli are vertically aligned in the mid-line, in order to avoid interference of neglect) and the use of multimodal exposures.

Method

Subjects

The participants were 98 patients with ischemic stroke, from the Neurological Department of Guangzhou First People’s Hospital, and 133 healthy controls. The control participants were recruited from the physical examination center in the hospital, from among relatives of the patients, and from among residents of several old peoples’ communities in Guangzhou. All the participants were native Cantonese speakers.

All the control participants were aged 50 years or above, without a history of brain lesion and without complaint of memory loss. Participants who had a MoCA score less than 22 were excluded from the control group.15,16

Patients were recruited between July 2013 and March 2014.

The inclusion criteria for the patient group were as follows:

Aged 50 years and above

Within 2 weeks of a confirmed stroke and without a prior history of brain lesion; the patient was judged by the clinical team to be able to concentrate for at least 45 minutes, to enable the tests to be aministered

The patient consented to testing and signed the consent document.

The exclusion criteria were:

Past history of cognitive impairment and brain lesion including a history of stroke

The presence of chronic heart failure, anemia, or other diseases that may lead to cognitive impairment

Inability to concentrate for at least 45 minutes.

The study was approved by the local research ethics committee, and participants gave informed, written consent.

Cultural and linguistic modifications of BCoS

The validity and reliability of the BCoS has previously been established in English (in the UK).17 In the present paper, we report on the validity and reliability of the Cantonese version of the screen. For this version, four specific cultural and linguistic modifications were made.

Picture naming

Some of the items were identified as being culturally inappropriate in the People’s Republic of China, and these were replaced (eg, the picture of a colander was rejected, as colanders are not standardly used in Chinese cooking; this was replaced with a picture of a spatula).

Sentence reading and nonword reading

Here, we define a “regular” character as one being “regular-consistent” while an “exception” character as one being “irregular-inconsistent”. Both ‘regularity’ and ‘consistency’ are often used to describe mapping between orthography and phonology.18 Broadly speaking, a written word (ie, a character in Chinese writing) is regular if its pronunciation follows the grapheme-to-phoneme correspondence rules of the written language (eg, “doctor” in English);19 and a word/character is an exception if its pronunciation deviates from those rules (eg, “daughter” in English). Consistency is described with respect to the orthographic body and the phonological rime. A consistent word/character (eg, wade in English) is one that has a word-body (-ade) pronounced in the same way for the entire set of orthographic neighbors (eg, spade, jade, made). An inconsistent word/character (e.g., wave in English) has among its neighbors at least one exception word (eg, have).

In the Cantonese sentence reading task, we included only ‘regular’ characters which are also ‘consistent’, that is, those having a consistency value of >80% (ie, more than 80% of its orthographic neighbors are pronounced in the same way as the chosen word). With regard to the ‘exception’ characters, we included only ‘irregular’ characters which are also ‘inconsistent’, that is, those having a consistent value of <20% (ie, less than 20% of its orthographic neighbors are pronounced in the same way as it is). On the whole, all items chosen have a frequency of >200 (in millions) documented in Leung et al (unpublished database, 2010).

Gesture production and gesture recognition

Some of the gestures were replaced to fit with Chinese culture and daily habits (eg, the gesture of hitchhiking was rejected because this activity is not common in the People’s Republic of China; this item was then replaced with a gesture of rubbing one’s thumb and forefinger together to indicate money).

Word writing

Chinese character writing is an equivalent task to (real) word writing in the original English version. During the task, a common, familiar word (eg, “眼睛”, the word for “eye” in Chinese) consisting of the target character (eg, “眼”,/ngaan5/) and another character that often coappears (eg, “睛”,/zing1/) is voiced out, although the participant is only required to produce the target character in the written response. There are four items in this test, half of which are concrete characters (representing real objects), and the other half are abstract (representing concepts). Within each category, one item is a regular character, while the other is an exception character. The characters were taken from The Hong Kong, Mainland China and Taiwan: Chinese Character Frequency list (http://humanum.arts.cuhk.edu.hk/Lexis/chifreq/), with a word frequency ranging between 5000/663461 and 11115/663461. The 15 characters are controlled for the six tones in Cantonese, part-of-speech, number of strokes, and phonetic structures (“CV”, “CVV”, and “CVC”),20 with minimal repetition of the same onsets and rimes. Disyllabic and trisyllabic word items were created by randomly combining the 15 characters. Character combination was subsequently verified to ensure that characters do not exist together in any known words in Cantonese. Regarding nonword writing, a test of writing Chinese pseudocharacters (or nonwords) is not available in the Cantonese version, as it is impossible to achieve in the logographic Chinese writing system.

Tests

A forward–backward translation of the BCoS was conducted. The battery was first translated from English into Cantonese by author AK. Author JL then translated the Cantonese version back into English. Any disagreements (<5% out of all translated items) were subsequently resolved to maintain the integrity, consistency, and accuracy of the translation. Note that some items in subtests were replaced by stimuli with cultural and linguistic modifications (as described above); all other items were directly adopted and translated from the English version.

All the patients were tested on the Cantonese version of the BCoS, the MoCA, the MMSE, Albert’s test of neglect,21 the Rey–Osterrieth Complex Figure Test,22 and six gesture matching tasks from Goldenberg.23 The MoCA and MMSE were used to validate assessments of language, number processing, and memory; the Rey–Osterrieth Complex Figure Test was used to validate complex figure copying; Albert’s test was used to validate the BCoS measure of neglect (Apple cancellation); and Goldenberg’s tasks were employed to validate the praxis tests in the BCoS. All the participants were randomly assessed by one of four testers in our study. All the testers were doctors in the Neurological Department, and all were trained in test administration by the research teams at Birmingham and Oxford Universities (UK). The interrater reliability was shown in the results.

Subsets of the controls and the patients were retested on a second occasion to establish test–retest reliability. The subject’s identity (as a control or patient) was not blind to the testers, as it is almost impossible to blind a tester to the presence of a hemiplegia or a major cognitive deficit in a stroke patient. Also, scores requiring qualitative judgments of performance were assessed by two other testers in order to evaluate interrater reliability. The testers were blind to each other’s score.

Statistical analysis

Demographic data were compared between the control and patient groups using an independent t-test. Cutoffs were set at the 5th percentile, for scores indicating high ability, and at the 95th percentile, for scores indicating difficulties, and were smoothed across age groups. Pearson’s correlation tests were used to test the test–retest reliability and convergent validity. The interrater reliability was addressed by calculating interclass correlations among the scores of three parallel ratings of the Cantonese version BCoS.

Results

General condition

A total of 133 healthy participants were assessed, with 51 males and 82 females. The average age of these controls was 65.70±8.99 years, and the average years of education was 9.12±4.20. Ninety-eight stroke patients participated, including 57 males and 41 females. Considering the decay of cognitive ability along with age growth, participants were divided into two age groups. Within each of the age groups, the patients and controls did not differ in either their age or the level of education (both P>0.05) (Table S1).

There were 98 patients in our study, including 39 patients with left hemispheric lesion, 36 with right hemispheric lesion, five with bilateral hemispheric lesion, and 15 with brainstem and cerebellum lesion. According to the TOAST classification,24 there were 30 patients with large-artery atherosclerosis, seven with cardioembolism, 57 with small-artery occlusion, and four with undetermined etiology. The average National Institutes of Health Stroke Scale (NIHSS) score of the patient group was 2.355±2.45 (ranged from 0 to 15).

Normative data

The standardization sample for BCOS was divided into two age groups: 50–69 years and ≥70 years. The mean score and 5th and 95th percentile cutoffs are presented in Table 1. The test labels presented in the table were all taken from the BCoS.24 Subtests were divided into their respective domains.

Table 1

Mean scores, and 5th and 95th percentiles across the two age groups of healthy participants

	50–69 years N=94			≥70 years N=39
	Score range	Mean	5th or 95th percentile	Score range	Mean	5th or 95th percentile
Attention-spatial
Apple cancellation (accuracy)	0–50	46.6	42	0–50	45.79	39
Asymmetry (allocentric) (R, L)	0–20	0.02	0, 1	0–20	−0.21	−2, 1
Asymmetry (egocentric) (R, L)	0–20	0.29	−3, 3	0–20	−0.13	−3, 4
Left visual unilateral	0–4	3.97	4	0–4	4	4
Right visual unilateral	0–4	3.98	4	0–4	4	4
Left visual bilateral	0–8	7.99	8	0–8	8	8
Right visual bilateral	0–8	8	8	0–8	7.97	8
Left tactile unilateral	0–4	4	4	0–4	3.95	3
Right tactile unilateral	0–4	3.98	4	0–4	3.97	3
Left tactile bilateral	0–8	7.96	8	0–8	7.97	8
Right tactile bilateral	0–8	7.97	8	0–8	7.97	8
Attention-controlled
Auditory attention accuracy	0–54	50.82	39	0–54	48.92	33
Auditory attention practice	1–3	1.09	1	1–3	1.29	1
Auditory attention word recall	0–3	2.83	2	0–3	2.68	2
Sustained attention index		0.50	−2, 4		0.08	−4, 3
Rule finding and set shifting accuracy	0–18	11.96	6	0–18	11.13	3
Rule finding-rules found	0–3	2.21	0	0–3	2.03	0
Language-spoken
Instruction comprehension	1–3	3	3	1–3	2.84	2
Picture naming	1–14	13.29	11	1–14	12.03	8
Sentence construction	0–8	7.80	7	0–8	7.82	6
Language-written
Nonword reading-accuracy	0–6	5.84	5	0–6	5.89	5
Sentence reading-accuracy	0–40	39.27	37	0–40	39.26	34
Character writing	0–4	3.38	1	0–4	3.19	1
Memory-orientation
Personal	0–8	7.94	7	0–8	7.90	7
Time and space	0–6	5.99	6	0–6	5.87	5
Memory-episodic
Story free-recall 1	0–15	7.03	4	0–15	5.64	2.5
Story recognition 1	0–15	12.45	9	0–15	11.62	8
Story free-recall 2	0–15	9.46	5.5	0–15	7.55	3
Story recognition 2	0–15	14.22	13	0–15	13.46	9
Task-recognition	0–10	9.43	8	0–10	8.92	7
Number
Reading	0–9	8.78	8	0–9	8.42	7
Writing	0–5	4.68	3	0–5	4.46	2
Calculation	0–4	3.71	2	0–4	3.47	2
Praxis-action
Figure copy	0–47	43.24	37	0–47	41.92	34
Multistep object use	0–12	11.86	11	0–12	11.76	11
Gesture production	0–12	11.20	9	0–12	10.45	7
Gesture recognition	0–6	5.96	6	0–6	5.92	6
Imitation	0–12	10.30	6	0–12	9.47	5

Abbreviations: R, right; L, left.

Cutoff scores were set at the 5th percentile for scores indicating high ability and the 95th percentile for scores indicating difficulties, smoothed across age groups. The cutoffs were set at the 2.5th and 97.5th percentiles for subtests that had two tails (eg, the spatial asymmetry scores, which could be positive or negative).

Table 2 presents the numbers of stroke patients who were impaired on the tests, based on the cutoffs given in Table 1.

Table 2

Numbers of patients who were impaired on BCoS subtests

	50–69 years		≥70 years
	Frequency	%	Frequency	%
Attention and executive function
Spatial
Apple cancellation (accuracy)	10/49	20.4	13/47	27.7
Asymmetry (allocentric) (R, L)	10/49	20.4	6/47	12.8
Asymmetry (egocentric) (R, L)	6/49	12.2	5/47	10.6
Left visual unilateral	1/48	2.1	1/48	2.1
Right visual unilateral	1/48	2.1	0/48	0
Left visual bilateral	2/48	4.2	3/48	6.3
Right visual bilateral	1/48	2.1	4/48	8.3
Left tactile unilateral	5/48	10.4	0/48	0
Right tactile unilateral	2/48	4.2	0/48	0
Left tactile bilateral	4/48	8.3	5/48	10.4
Right tactile bilateral	1/48	2.1	7/48	14.6
Controlled
Auditory attention accuracy	6/48	12.5	9/46	19.6
Auditory attention practice	11/48	22.9	19/46	41.3
Auditory attention word recall	4/48	8.3	8/46	17.4
Sustained attention index	5/45	11.1	4/37	10.8
Rule finding and set shifting accuracy	9/48	18.8	9/46	19.6
Language
Spoken
Instruction comprehension	20/47	42.6	7/47	14.9
Picture naming	5/49	10.2	2/48	4.2
Sentence construction	12/49	24.5	10/48	20.8
Written
Nonword reading-accuracy	5/49	10.2	9/48	18.8
Sentence reading-accuracy	10/49	20.4	10/48	20.8
Character writing	7/41	17.1	11/33	33.3
Memory
Orientation
Personal	5/50	10	3/48	6.3
Time and space	11/50	22	9/48	18.8
Episodic
Story free-recall 1	19/50	38	9/48	18.8
Story recognition 1	15/50	30	10/48	20.8
Story free-recall 2	12/49	24.5	11/47	23.4
Story recognition 2	18/49	36.7	6/47	12.8
Task-recognition	7/46	15.2	5/47	10.6
Number skills
Reading
Reading	13/47	27.7	12/47	25.5
Writing
Writing	7/46	15.2	10/45	22.2
Calculation
Calculation	5/47	10.6	7/46	15.2
Praxis
Figure copy	16/46	34.8	18/45	40
Multistep object use	6/47	12.8	13/47	27.7
Gesture production	10/47	21.3	9/47	19.1
Gesture recognition	10/47	21.3	13/47	27.7
Imitation	5/47	10.6	7/47	14.9

Notes: Values are expressed as n (%). Some of the scores were missing due to the fatigue and physical challenges.

Abbreviations: BCoS, Birmingham Cognitive Screen; R, right; L, left.

The results are consistent with considerable numbers of patients being impaired relative to our cutoffs – up to 40% of the patients had some impairment in complex figure copy, and about 20% had problems in aspects of controlled attention (eg, auditory attention accuracy). Note that these particular tests are relatively unbound by culture.

Reliability studies

Test–retest reliability

The test–retest assessments were performed on the eleven controls, who were examined with a test–retest interval ranging from 122 to 330 days (205±95.93 days). The average age of these participants was 63.8 years (SD =7.99 years), and they had received an average of 10.5 years of education (SD =3.39 years). There were five males and six females. In addition, test–retest resampling was also carried out on 15 stroke patients. The average age of the stroke patients was 64.5 years (SD =6.66 years). There were 12 males and three females, and they had received an average 9 years of education (SD =3.17 years). The test–retest interval in this case ranged from 102 to 191 days (144.2±25.61 days).

The data were analyzed in two ways. First, correlations were conducted between the test and retest scores, pooling the data across patients and controls to maximize power. Tests were not included here, where the controls were uniformly at ceiling. This meant, too, that for the measure of spatial attention, we took only the overall score and not the asymmetry scores (since controls showed no spatial asymmetries). The correlation results are shown in Table 3.

Table 3

Correlations between the initial test scores and the retest scores

	Pearson correlation	P-value	Pearson correlation	P-value	Patients alone	P-value
	Controls + Patients		Controls alone
Attention-spatial
Apple cancellation (overall score)	0.551	0.004	0.799	0.003	0.276	0.320
Controlled
Auditory attention accuracy	0.756	<0.001	0.749	0.008	0.711	0.003
Rule finding and set shifting accuracy	0.639	<0.001	0.690	0.019	0.706	0.003
Language-spoken
Picture naming	0.399	0.044	0.593	0.055	0.341	0.214
Sentence construction	0.077	0.709	0.671	0.024	NA	NA
Written
Sentence reading-accuracy	0.843	<0.001	0.464	0.150	0.907	<0.001
Nonword reading accuracy	0.483	0.013	0.467	0.148	0.535	0.040
Memory-episodic
Story free-recall 1	0.485	0.012	0.483	0.133	0.416	0.123
Story recognition 1	0.645	<0.001	0.596	0.053	0.655	0.008
Story free-recall 2	0.598	0.001	0.720	0.013	0.500	0.058
Story recognition 2	0.320	0.111	0.313	0.349	0.406	0.133
Number skills
Number reading	0.793	<0.001	0.391	0.234	0.947	<0.001
Number writing	0.615	0.001	0.671	0.024	NA	NA
Calculation	0.603	0.001	0.608	0.047	0.639	0.010
Praxis
Complex figure copy	0.559	0.001	0.516	0.104	0.491	0.063
Gesture production	0.652	<0.001	0.443	0.172	0.723	0.002
Imitation	0.528	0.006	0.934	0.000	0.374	0.170

Notes: NA signifies tests where correlations could not be performed due to lack of variance in the data.

There were reliable correlations across all the subtests that remained after filtering for ceiling effects. These correlations were not solely due to the contrast between patients and controls, and were apparent even when each group was considered alone.

In the second analysis, we assessed performance differences at times 1 and 2, using the Wilcoxon signed rank test. As for the correlation analyses, the assessments were performed only for tests where the controls did not score at ceiling and then, pooling the data across the patients and the controls (Tables S2, S3, and S4). In nearly all cases, there were no differences between performance on the two test occasions. The exceptions to this were for the recognition test and the second recall task, where performance improved on the second test occasion. This held for the data for the patients when they were considered alone too (P=0.026 [recognition test] and P=0.064 [second recall test]).

Interrater reliability

There are five tests in the BCoS where scoring requires qualitative judgments of performance: the sentence construction task, multistep object use tasks, gesture production, imitation, and figure copying. To assess interrater reliability, three independent raters scored the performance of ten ischemic stroke patients, and interclass correlations (ICC) between the scores of the three parallel ratings of the BCoS subtests were calculated. Results showed that there was no significant difference between the raters’ judgments in respect of these qualitative assessments (Table S5).

Convergent validity

The different subtests of the BCOS were validated against existing measures chosen to index the same underlying cognitive function. The standard tests selected for each BCoS subtest are listed in Table 4, along with the number of participants (patients and controls) for which data were collected and correlated. The data reported in Table 4 cover both patients and controls (pooled results) and indicate reliable correlations across all the selected tasks. This did not reflect purely the difference between the patients and the controls, and similar results occurred for just the patients considered alone (Table S6).

Table 4

Correlations with comparable standardized tasks

Domain	Task name	Validation task	N	R	P-value
Attention-spatial	Apple cancellation accuracy	Total number of lines crossed on Albert’s test	70	0.630	0.000
	Asymmetry (allocentric) (R, L)	Albert cancellation test-asymmetry	70	0.304	0.010
	Asymmetry (egocentric) (R, L)	Albert cancellation test-asymmetry	70	0.429	0.000
Attention-controlled	Auditory attention accuracy	MoCA-attention (read and subjects tap)	191	0.385	0.000
		MoCA attention (total score)	191	0.518	0.000
		MoCA attention (digit span)	191	0.224	0.002
	Rule finding and set shifting accuracy	MoCA-attention (total score)	191	0.406	0.000
	Rule finding and set shifting accuracy	MoCA visuospatial/executive	191	0.508	0.000
Language-spoken	Instruction comprehension	MoCA-language (total score)	192	0.436	0.000
	Picture naming	MoCA-naming	191	0.254	0.000
	Sentence construction	MoCA-language (total score)	192	0.416	0.000
Language reading	Nonwords reading	MoCA-language (total score)	191	0.322	0.000
		MMSE-reading	102	0.388	0.000
	Sentence reading	MoCA-language (total score)	191	0.331	0.000
		MMSE-reading	102	0.365	0.000
	Character writing	MMSE-writing	102	0.444	0.000
Memory-orientation	Time and space	MoCA-orientation	190	0.568	0.000
		MMSE-orientation	102	0.452	0.000
	Time and space (MCQ)	MoCA-orientation	190	0.730	0.000
		MMSE-orientation	102	0.665	0.000
Memory episodic	Story free-recall1 (immediate recall)	MoCA-delayed recall and MMSE delayed recall	191	0.397	0.000
		MMSE-delay recall	103	0.390	0.000
	Story recognition (immediate recognition)	MoCA-delay recall	191	0.335	0.000
		MMSE-delay recall	103	0.314	0.001
	Story free-recall 2 (delayed free-recall)	MoCA-delay recall	191	0.495	0.000
		MMSE-delay recall	103	0.556	0.000
	Story recognition 2	MoCA-delay recall	191	0.346	0.000
		MMSE-delay recall	103	0.385	0.000
	Task-recognition	MoCA-delay recall	191	0.327	0.000
		MMSE-delay recall	103	0.219	0.026
Number skills	Reading	MoCA-calculation (serial 7)	191	0.482	0.000
	Writing	MoCA-calculation (serial 7)	191	0.303	0.000
	Calculation	MoCA-calculation (serial 7)	191	0.454	0.000
		MMSE-calculation	103	0.555	0.000
Praxis	Complex figure copy	Rey–Osterrieth Complex Figure Test	64	0.650	0.000
	Multistep object use	Gestural imitation score (Goldenberg)23	71	0.474	0.000
	Gesture production	Gestural imitation score (Goldenberg)23	71	0.583	0.000
	Gesture recognition	Gestural imitation score (Goldenberg)23	71	0.479	0.000
	Imitation	Gestural imitation score (Goldenberg)23	71	0.626	0.000

Abbreviations: MCQ, multi choice question; MMSE, Mini Mental State Examination; MoCA, Montreal Cognitive Assessment test; R, right; L, left.

The results indicate reliable correlations between the subtests of the BCoS and the standard tests chosen to measure selective cognitive functions.

Performance of patients with aphasia

Due to its design (ie, the use of short, high-frequency words and forced-choice testing), the BCoS is able to gain measures on many functions for patients with language problems. This is less apparent for other screens, such as the MMSE and the MoCA, where measures depend on the patients having good spoken language. To assess the effectiveness of the BCoS for these patients, we examined the performance of patients with and without a problem on the sentence construction test of the tests of controlled attention (auditory attention and executive function), memory (story free- recall 1 and story recognition 1), character writing, and praxis (figure copy and multistep object use). Patients were defined as having a spoken language problem if they fell beneath the cutoff on the sentence construction test. We divided patients into two subgroups based on their sentence construction scores. Table 5 presents the numbers of patients in each group who were impaired on the tests mentioned above, according to the cutoffs given in Table 1. The comparisons of incidence rate of disability in controlled attention, memory, character writing, and praxis, between patients with and without spoken problem were presented in Table 5 as well.

Table 5

Numbers of patients in each group who were impaired on the tests

	Patients with problem in sentence construction		Patients without problem in sentence construction		χ2	P-value
	N	%	N	%
Auditory attention	8/20	40	7/74	9.5	10.950	0.001
Rule finding and set shifting accuracy	11/20	55	7/74	9.5	21.091	0.000
character writing	8/15	53.3	10/59	16.9	8.601	0.003
Story free-recall 1	9/22	40.9	18/75	24	2.421	0.120
Story recognition 1	13/22	59.1	12/75	16	16.510	0.000
Figure copy	15/19	78.9	19/72	26.4	15.570	0.000
Multistep object use	8/20	40	11/74	14.9	6.618	0.013

The table indicates that the aphasic patients largely performed worse than the nonaphasic patients in most of the subtests. Nevertheless, in very many cases, at least half the aphasic patients were able to generate normal scores on other aspects of the BCoS.

Discussion

There is currently no screen for Chinese individuals (Cantonese speakers) that has been specially designed (ie, being “aphasia and neglect friendly”) to maximize the inclusion of patients after stroke while being sensitive to common cognitive problems after stroke (such as spatial neglect and apraxia). To rectify this, we constructed a Cantonese version of the BCoS, a screen designed specifically to address these problems in English. We showed that the Cantonese version BCoS has high convergent validity; there were reliable correlations between each of the subtests and standard tests in the literature chosen to assess a particular aspect of cognition. This demonstrates that the Cantonese version BCoS does indeed assess the subfunctions being proposed. The Cantonese version BCoS also had good test–retest reliability, and there was good reliability across trained testers in the scoring of the qualitative aspects of the screen. Moreover, by using the screen, we were able to generate scores on non-language-based tests in patients with poor language (eg, the measures of executive function in patients with poor picture naming). Thus the Cantonese version BCoS can be very useful for generating cognitive profiles that are (relatively) uncontaminated by poor language, in stroke patients.

The screen did indicate that there were relatively high numbers of patients with cognitive impairments after their stroke. Around 40% of the patients showed poor figure copy, and 30% were impaired on the BCoS measures of sustained attention – these tests either use nonverbal or (at most) very simple linguistic material, so it is interesting that high incidence rates were noted. In addition to this, there were high incidence rates for apraxia and impaired spatial attention (~27% of the sample were impaired on the multistep object use task and on gesture recognition, and ~28% of patients had an impairment on the test of spatial attention). Given that these impairments are not measured in standard screens currently applied in the People’s Republic of China (ie, the MoCA and the MMSE), the Cantonese version BCoS certainly has distinct advantages. Notably, the majority of subjects recruited in our patient group had lacunar infarction (57 out of 98 patients in our group suffered from small-artery occlusion). According to previous studies, mild cognitive impairment was present in up to 50% of patients with lacunar stroke, even in those with minimal or without physical disabilities.25,26 Other research has shown that more than half of patients with a first-ever lacunar stroke and without cognitive impairment show minor neuropsychological alterations, particularly in the performance of executive functions (eg, semantic fluency) and short-delayed verbal memory. Therefore it is very important to include a “neuropsychological study” in future clinical trials in patients with lacunar stroke.27

Another advantage of the Cantonese version BCoS compared with other current screens relates to the procedures for reporting BCoS scores. The BCoS uses a “wheel of cognition”, which enables deficits (or strengths) in particular domains of cognition to be grasped at a glance (Figure S1). This facilitates the reporting of domain-specific deficits (rather than reporting back a single score, as is done for the MoCA and the MMSE), which we suggest is important in order to target rehabilitation at the subdomains that are affected in the patients. We suggest that the combined use of the Cantonese version BCoS and of its reporting scheme will substantially improve the early detection and rehabilitation of cognitive problems after stroke in the People’s Republic of China.

However, there were still some limitation in our study. Firstly, the stroke patients recruited in the study generally had relatively low NIHSS scores, indicating that their neurological functions were mildly to moderately impaired. Hence it is unclear whether the Cantonese version BCoS is effective enough for stroke patients with severe neurological impairments. Secondly, our sample size was relatively small. Therefore a larger sample for reassessment is needed in the future study, to make the data more representative. Finally, since the time required for completing the questionnaire is about 1 hour, some patient data were missing because of fatigue or inability to concentrate for such a lengthy period.

Example “visual snapshot” of the cognitive profile of a patient given by the BCoS.

Notes: Each colored area represents a different cognitive domain. In the reporting system, a black rim indicates intact performance (within normal limits). The rim is colored white when a deficit occurs relative to the norm. The rim is omitted if the test has not been administered. After a little experience, clinical teams learned to “read” the profile efficiently, identifying the strengths (preserved cognitive abilities) and weaknesses (impairements relative to the controls) of particular patients.

Abbreviation: BCoS, Birmingham Cognitive Screen.

Table S1

The comparison in age and education years between controls and patients within each age group (mean ± standard deviation)

	50–69 years			≥70 years
	Patient group	Control group	P-value	Patient group	Control group	P-value
Age (years)	60.78±5.80	60.86±4.98	0.93	76.35±4.43	77.36±4.80	0.31
Years of education	8.47±2.95	9.18±3.62	0.24	6.89±4.64	8.97±5.39	0.06

Table S2

Comparisons of test–retest scores (pooled across patients and controls)

Tasks–patient and control group	Range	1st test		2nd test		Wilcoxon signed rank	% exact score agreement
		Mean	SD	Mean	SD	P-value
Attention-spatial
Apple cancellation (overall score)	0–50	47.04	3.04	48.08	1.65	0.063	34.6
Controlled
Auditory attention accuracy	0–54	50.38	5.29	50.73	4.30	0.269	30.8
Rule finding accuracy	0–18	12.85	2.96	13.23	3.13	0.313	30.8
Language-spoken
Picture naming	1–14	13.35	0.80	13.38	0.70	0.776	57.7
Sentence construction	0–8	7.69	0.74	7.92	0.27	0.131	80.8
Written
Sentence reading-accuracy	0–40	39.00	1.88	38.96	2.05	0.763	65.4
Nonword reading accuracy	0–6	5.77	0.59	5.81	0.40	0.705	73.1
Memory-episodic
Story free-recall 1	0–15	6.21	2.20	7.44	2.04	0.014	15.4
Story recognition 1	0–15	11.92	1.83	12.92	1.38	0.003	15.4
Story free-recall 2	0–15	8.35	2.51	9.88	2.37	0.004	7.7
Story recognition 2	0–15	13.73	1.12	14.38	0.64	0.007	30.8
Number skills
Number reading	0–9	8.73	0.67	8.77	0.71	0.655	80.8
Number writing	0–5	4.77	0.59	4.88	0.59	0.257	84.6
Calculation	0–4	3.69	0.62	3.65	0.56	0.705	73.1
Praxis
Complex figure copy	0–47	42.92	2.61	43.85	3.02	0.034	15.4
Gesture production	0–12	11.04	1.28	11.35	0.85	0.082	57.7
Gesture recognition	0–6	5.88	0.33	6.00	0.00	0.083	88.5
Imitation	0–12	11.12	1.18	11.12	1.18	0.683	57.7

Abbreviation: SD, standard deviation.

Table S3

Comparisons of test–retest scores (control group)

Tasks (control group)	Range	1st test		2nd test		Wilcoxon signed rank	% exact score agreement
		Mean	SD	Mean	SD	P-value
Attention-spatial
Apple cancellation overall score	0–50	45.64	4.06	48.00	1.79	0.027	45.5
Controlled
Auditory attention accuracy	0–54	52.55	2.34	52.82	1.89	0.454	18.2
Rule finding accuracy	0–18	12.18	2.86	13.64	1.96	0.039	36.4
Language-spoken
Picture naming	1–14	13.45	0.69	13.18	0.60	0.317	63.6
Sentence construction	0–8	7.91	0.30	7.91	0.30	0.317	90.1
Written
Sentence reading-accuracy	0–40	39.18	1.17	39.27	1.19	0.854	63.6
Nonword reading accuracy	0–6	5.64	0.67	5.82	0.41	0.317	63.6
Memory-episodic
Story free-recall 1	0–15	6.73	2.79	8.23	2.21	0.102	9.1
Story recognition 1	0–15	12.36	1.91	13.45	1.51	0.046	18.2
Story free-recall 2	0–15	8.86	3.16	10.73	2.10	0.029	0
Story recognition 2	0–15	14.27	0.91	14.36	0.65	0.739	54.5
Number skills
Number reading	0–9	8.73	0.47	8.73	0.65	1.000	63.6
Number writing	0–5	4.64	0.81	4.73	0.91	0.655	81.8
Calculation	0–4	3.64	0.67	3.45	0.69	0.317	63.6
Praxis
Complex figure copy	0–47	44.27	1.90	45.45	1.70	0.058	18.2
Gesture production	0–12	11.27	1.10	11.55	0.69	0.334	54.5
Gesture recognition	0–6	5.91	0.30	6.00	0.00	0.317	90.9
Imitation	0–12	11.18	1.17	11.55	0.69	1.000	81.8

Abbreviation: SD, standard deviation.

Table S4

Comparisons of test–retest scores (patient group)

Tasks (patient group)	Range	1st test		2nd test		Wilcoxon signed rank	% exact score agreement
		Mean	SD	Mean	SD	P-value
Attention-spatial
Apple cancellation	0–50	48.07	1.44	48.13	1.60	0.893	26.7
Controlled
Auditory attention accuracy	0–54	48.80	6.29	49.20	4.95	0.511	40
Rule finding accuracy	0–18	13.33	3.04	12.93	3.81	0.823	26.7
Language-spoken
Picture naming	1–14	13.27	0.88	13.47	0.74	0.380	53.3
Sentence construction	0–8	7.53	0.92	8.00	0.00	0.066	73.3
Written
Sentence reading-accuracy	0–40	38.87	2.30	38.73	2.520	0.492	66.7
Nonword reading accuracy	0–6	5.87	0.52	5.80	0.41	0.564	80
Memory-episodic
Story free-recall 1	0–15	5.83	1.64	6.87	1.77	0.059	20
Story recognition 1	0–15	11.60	1.77	12.53	1.19	0.026	13.3
Story free-recall 2	0–15	7.967	1.93	9.27	2.43	0.064	13.3
Story recognition 2	0–15	13.33	1.11	14.40	0.63	0.005	20
Number skills
Number reading	0–9	8.73	0.80	8.80	0.78	0.317	93.3
Numberwriting	0–5	4.87	0.35	5.00	0.00	0.157	86.7
Calculation	0–4	3.73	0.59	3.80	0.41	0.564	80
Praxis
Complex figure copy	0–47	41.93	2.66	42.67	3.27	0.216	13.3
Gesture production	0–12	10.87	1.41	11.20	0.94	0.163	60
Gesture recognition	0–6	5.87	0.35	6.00	0.00	0.157	86.7
Imitation	0–12	10.87	1.99	11.07	1.16	0.675	40

Abbreviation: SD, standard deviation.

Table S5

Interrater reliability for the sentence construction tasks, multistep object use, gesture production, gesture imitation, and figure copying

Task	ICC (interclass correlations)	95% CI	% exact score agreement
Sentence construction	0.92	0.80–0.98
Multistep object use	–	–	100%
Gesture production	–	–	100%
Imitation	0.987	0.96–0.99
Figure copy	0.993	0.98–0.99

Abbreviations: CI, confidence interval; ICC, the mean correlation coefficient across the three raters.

Table S6

Subtests of BCoS compared with the validation task in the patient group

Domain	Task name	Validation task	N	Correl (R)	P-value
Attention-spatial	Apple cancellation accuracy	Total number of crossed lines (Albert’s test)	56	0.631	0.000
	Asymmetry (allocentric) (R, L)	Albert cancellation test-asymmetry	56	0.311	0.019
	Asymmetry (egocentric) (R, L)	Albert cancellation test-asymmetry	56	0.486	0.000
Attention-controlled	Auditory attention accuracy	MoCA-attention (read and subjects tap)	74	0.385	0.001
		MoCA attention (total score)	74	0.592	0.000
		MoCA attention (digit span)	74	0.331	0.004
	Rule finding accuracy	MoCA-attention (total score)	74	0.500	0.000
Language-spoken	Instruction comprehension	MoCA-language (total score)	74	0.524	0.000
	Picture naming	MoCA-naming	74	0.230	0.049
	Sentence construction	MoCA-language (total score)	74	0.488	0.000
Language reading	Nonwords reading	MoCA-language (total score)	74	0.336	0.000
		MMSE-reading	65	0.440	0.000
	Sentence reading	MoCA-language (total score)	74	0.361	0.002
		MMSE-reading	65	0.406	0.001
	Character writing	MMSE-writing	65	0.549	0.000
Memory-orientation	Time and space	MoCA-orientation	73	0.584	0.000
		MMSE-orientation	64	0.424	0.000
	Time and space (MCQ)	MoCA-orientation	73	0.790	0.000
		MMSE-orientation	64	0.675	0.000
Memory episodic	Story free-recall1 (Immediate recall)	MoCA-delay recall	74	0.337	0.003
		MMSE-delay recall	65	0.458	0.000
	Story recognition (immediate recognition)	MoCA-delay recal	74	0.370	0.001
		MMSE-delay recall and	65	0.307	0.013
		MoCA-delayed recall
	Story free-recall 2 (delayed free recall)	MoCA-delay recall	74	0.428	0.000
		MMSE-delay recall	65	0.524	0.000
	Story recognition 2	MoCA-delay recall	74	0.319	0.006
		MMSE-delay recall	65	0.352	0.004
	Task recognition	MoCA-delay recall	74	0.315	0.006
		MMSE-delay recall	74	0.084	0.506
Number skills	Reading	MoCA-calculation (serial 7)	74	0.600	0.000
	Writing	MoCA-calculation (serial 7)	74	0.336	0.003
	Calculation	MoCA-calculation (serial 7)	74	0.570	0.000
		MMSE-calculation	74	0.615	0.000
Praxis	Complex figure copy	Rey–Osterrieth Complex Figure Test	49	0.621	0.000
	Multistep object use	–	56
	Gesture production	Gestural imitation score (Goldenberg)	57	0.552	0.000
	Gesture recognition	Gestural imitation score (Goldenberg)	57	0.451	0.000
	Imitation	Gestural imitation score (Goldenberg)	57	0.610	0.000

Abbreviations: BCoS, Birmingham Cognitive Screen; MCQ, multi choice questions; MMSE, Mini Mental State Examination; MoCA, Montreal Cognitive Assessment test; L, left; R, right.