The validation of the Beijing version of the Montreal Cognitive Assessment in Chinese patients undergoing hemodialysis.

OBJECTIVE: Cognitive impairment is common among hemodialysis patient, but still lack adequate screening in clinical settings. The Montreal Cognitive Assessment (MoCA) is reportedly to be a sensitive screening tool for cognitive impairment, but its clinical value in patients undergoing hemodialysis is not well established. We aimed to validate the utility of the Beijing version of the MoCA (MoCA-BJ) for detecting cognitive impairment in comparison to a detailed neuropsychological battery as the gold standard.
METHODS: We assessed 613 patients undergoing hemodialysis using the MoCA-BJ, the Mini-Mental State Examination (MMSE), and a comprehensive neuropsychological battery. Cognitive dysfunction was defined by the fifth version of the Diagnostic and Statistical Manual of Mental Disorders (DSM-V). Spearman's correlation and linear regression were used to estimate the performance of the MoCA-BJ and MMSE in predicting cognitive impairment. A receiver operating characteristic (ROC) curve analysis was used to evaluate the utility of various cutoffs of the MoCA-BJ and MMSE for predicting cognitive impairment.
RESULTS: Cognitive impairment was diagnosed in 80.91% (496/613), 75.69% (464/613), and 61.34% (376 /613) of the patients using the DSM-V, MoCA-BJ, and MMSE, respectively. Spearman's rank correlation analysis indicated that the MoCA-BJ was significantly correlated with the neuropsychological battery (rs = 0.639, p<0.001), whereas the MMSE had a weaker correlation with the battery. The area under the ROC curve for cognitive impairment diagnosis using the MoCA-BJ was 0.891 (95% confidence interval: 0.859-0.924) while using the MMSE was 0.823 (95% confidence interval: 0.786-0.860). The optimal MoCA-BJ cutoff score in discriminating patients with and without cognitive impairment was 24 points with a sensitivity of 0.877 and specificity of 0.752.
CONCLUSION: The MoCA-BJ offers good sensitivity and specificity levels in detecting cognitive impairment in hemodialysis patients. These findings support the utility of the MoCA-BJ as a screening tool for cognitive impairment in Chinese patients undergoing hemodialysis.

Introduction

The prevalence of cognitive impairment in patients undergoing hemodialysis (HD) is approximately 30–80%, which is two to four-fold higher as much as that in the general population [1-3]. In addition to being associated with the occurrence of cerebrovascular diseases and other adverse clinical outcomes, cognitive impairment may also influence adherence to medication and dietary management in hemodialysis patients [4, 5]. Under such circumstances, it seems urgent to find a rapid yet sensitive tool in the early detection of cognitive impairment among those patients.

Although comprehensive neuropsychological testing remains the gold standard for assessing cognitive function, it is not always logistically or economically feasible in clinical settings, the Montreal Cognitive Assessment (MoCA) is a brief screening test for cognitive impairment that covers major cognitive domains including memory, language, attention, orientation, visuospatial ability, and executive functions[6]. Several validation studies regarding the discrimination of different degrees of cognitive impairment were published in the last 10 years, they showed great variability in terms of sample sizes, normative scores and cut-off points for detecting people with Mild Cognitive Impairment (MCI) and Alzheimer’s disease (AD) [7, 8]. Bosco A et al [9]compared a group of people with probable Alzheimer’s Disease (AD) with healthy counterparts, the results showed that the optimal cutoff for a diagnosis of probable AD was a MoCA less than 14 in those Italian population. The Beijing version of MoCA (MoCA-BJ) has been widely used in mainland China for the screening of cognitive function [10]. Huang L et al [11] validated the Chinese version of MoCA and found that the MoCA was an effective cognitive test to distinguish between NC, MCI, mild and moderate AD among the Chinese elderly with various levels of education. Other studies have assessed the ability of the MoCA in the detection of cognitive impairment in a variety of diseases including Alzheimer’s disease, cerebral small vessel disease, and stroke [7, 12].

Data concerning the predictive value of MoCA in hemodialysis patients were mostly from other ethnic groups and the sample size of the previous studies was relatively small [13, 14]. Therefore, we aimed to validate the sensitivity, specificity and predictive value of the MoCA-BJ as a screening tool in detecting cognitive impairment which was determined by a comprehensive neuropsychological battery in a group of Chinese hemodialysis patients.

Methods

Participants

The study was performed using the data repository of the cohort study of CI in Chinese patients undergoing hemodialysis (CODE) (ClinicalTrials.gov ID: NCT03251573), which including 613 patients recruited from 11 HD centers in Beijing between April 2017 and June 2017. The mean age of participants was 63.7±7.8 years old, female participants were 258 (42.1%), the ratio of education level over 12 years was 72.4%. The eligibility criteria for participants were as follows: (1) aged 50–80 years, (2) had end-stage kidney disease, (3) were treated with long-term outpatient HD for the previous ≥3 months, (4) their HD team agreed to join the investigation and patients were willing to provide written informed consent, (5) ability to complete a 90 min cognitive and physical function battery, and (6) first language is Chinese.

The exclusion criteria for all participants were as follows: (1) were unable to participate for reasons such as sensory (e.g., visual and hearing) or motor impairment, (2) had a life expectancy of <6 months according to the evaluation from the physicians, (3) experienced disturbance of consciousness or were recently diagnosed with psychosis, and (4) had a planned kidney transplantation within 6 months of baseline.

At the enrollment, sociodemographic information, clinical history, and HD vintage were obtained by participant reports and patients’ electronic or paper charts. Pre-dialysis blood tests included measurement of the serum levels of hemoglobin, albumin, calcium, phosphate, and intact parathyroid hormone; the single-pool Kt/V were obtained from all the subjects.

Neuropsychological assessment

Neuropsychological assessment was conducted individually in a separate room on the second day after dialysis. Testing required approximately 90 min on average. A comprehensive battery of neuropsychological tests was designed to assess five cognitive domains: (1) attention using the Chinese modified version of the Trail Making Test A[12], Symbol Digit Modalities Test [15], (2) executive function using the Chinese modified version of the Trail Making Test B and A and modified version of the Stroop Color-Word Test [16]; (3) memory using the Chinese version of the Auditory Verbal Learning Test for short-delay and long-delay free recall [17] and Rey-Osterrieth Complex Figure (delayed recall test; Chinese version) [18]; (4) language using the Boston Naming Test (the 30-item version) and Category Verbal Fluency Test [19]; and (5) visuospatial function using the Rey-Osterrieth Complex Figure (copy test) [20].

Global cognition was assessed using the Chinese version of the MoCA-BJ and the Mini-Mental State Examination (MMSE) which is a 10 min screening test including questions to assess spatial and temporal orientation, immediate and delayed recall, language ability and oral command comprehension, and serial subtraction, and tasks of visuospatial ability [21, 22]. The MoCA-BJ requires educational adjustment; one point was added to the total score for those with ≤12 years’ education [23]. We also evaluated depression using the Hamilton Depression Scale. Scores range from 0 to 63, with a score of ≥7 suggested as the optimal cutoff for suspected depression [24].

Classification of cognitive impairment

We classified subjects as cognitive unimpaired or cognitive impairment based on criteria from the fifth version of the Diagnostic and Statistical Manual of Mental Disorders (DSM-V). Specifically, if the age- and education-adjusted scores were lower than1.5 SDs of the published norms in one test of one domain, the patient could be diagnosed as cognitive impairment [25]. The norms were based on a normative study of healthy, cognitively normal, community-dwelling adults in China [26].

Ethics statement

The research project was done in accordance with the latest version of the Declaration of Helsinki and the study protocol was approved by the institutional ethical review board of Beijing Shijitan Hospital affiliated to Capital Medical University (approval no. SJT2016-18). All participants provided written informed consent before participating.

Statistical analysis

All analyses were performed using SPSS software (version 19.0) (SPSS Inc, Chicago, IL, USA). The distribution of the data was analyzed by employing the Kolmogorov-Smirnov test. Data are presented as the mean ± SD for continuous variables with a normal distribution. Medians and interquartile ranges were used for continuous variables without a normal distribution and categorical variables.

To test the differences in the demographic and clinical characteristics and cognition scores between the cognitive impairment and cognitively unimpaired groups, normally distributed variables were compared using the Student’s t-test, and non-normal distributed variables were compared using the Mann-Whitney U test. Categorical variables are presented as percentages and were compared using the Chi-square test.

To demonstrate the different impairment features of each cognitive domain, the raw scores for each neuropsychological test were T-transformed. The T-scores for each domain were then generated by averaging the T-scores of their respective tests; composite T-scores were computed by averaging the T-scores of the five cognitive domains, the Mann-Whitney U test was applied to compare the difference between cognitive impairment and cognitively unimpaired groups.

To examine the relationships between neurocognitive domains performance and MoCA and MMSE scores, a correlational analysis was performed using Spearman’s rank correlation coefficient (rs). To examine the validity between the total scores of MoCA-BJ and the test battery overall composite scores, a bivariate linear regression model was also calculated using the MoCA-BJ as the dependent variable and the cognitive composite score as the independent variable. The demographic variables including age, sex, and education were also added as covariates [13].

To further evaluate the discriminate predictive power of the MoCA-BJ and the MMSE against the DSM-V standards, a receiver operating characteristic (ROC) curve analysis was performed. The area under the curve (95% confidence interval, 95%CI), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the MoCA-BJ and MMSE at various cutoffs were calculated. Optimal cutoff points were determined using the maximum value of Youden’s index (calculated by sensitivity+ specificity-1) [9]. Statistical significance for all analyses was set at P value < 0.05.

Results

Demographic and clinical characteristics

Patients’ demographic and clinical characteristics and the MoCA-BJ, and MMSE scores stratified by the cognitive function are presented in Table 1. Patients with cognitive impairment were more likely to be older and have a lower education level; a longer HD vintage; and comorbidities of diabetes, hypertension, and stroke. The MoCA-BJ score and MMSE score were significantly lower in the cognitive impairment group than the cognitively unimpaired group (Table 1).

Table 1

Demographics and Clinical data for studied participants.

Characteristics	Total	Cognitively unimpaired	Cognitive impairment	P Value§
	n = 613	n = 117	n = 496
Age (years)	63.7±7.8	59.3±7.7	64.7±7.4	< 0.001
Female sex	258(42.1%)	47(40.2%)	211(42.5%)	0.641
Education level				0.001
≤12 years	444(72.4%)	69(59.0%)	375(75.6%)
>12 years	169(27.6%)	48(41.0%)	121(24.4%)
Medical history
Smoking	270(44.0%)	51(43.6%)	219(44.2%)	0.925
Alcohol intake	261(42.6%)	51(43.6%)	210(42.3%)	0.936
Diabetes	231(37.7%)	33(28.2%)	198(39.9%)	0.019
Hypertension	545(88.9%)	95(81.2%)	450(90.7%)	0.003
Stroke	100(16.3%)	6(5.1%)	94(19.0%)	<0.001
CHD	193(31.5%)	36(30.8%)	157(31.7%)	0.853
BMI (kg/m2)	23.6±4.1	24.2±5.8	23.5±3.6	0.202
Dialysis vintage (months)	57.0(24.0,101.5)	42.0(12.0,78.0)	60.0(29.0,104.8)	<0.001
spKt/V	1.5±0.5	1.4±0.3	1.5±0.5	0.246
Lab. examination
Hb (g/L)	111.1±14.6	110.7±15.3	111.2±14.5	0.737
Alb (g/L)	39.9±3.2	40.3±2.5	39.9±3.3	0.218
Calcium (mmol/L)	2.2±0.2	2.2±0.2	2.2±0.3	0.127
Phosphate (mmol/L)	1.7±0.7	1.8±0.7	1.7±0.6	0.342
iPTH (pg/mL)	187.6(103.5,358.2)	167.6(107.0,311.5)	201.8(103.4,372.2)	0.058
Depression scores	4.0(1.0,7.0)	4.0(2.0,7.0)	3.0(1.0,7.0)	0.441
MoCA-BJ scores	21.0(19.0,23.0)	25.0(23.5,26.0)	21.0(18.0,22.0)	<0.001
MMSE scores	25.0(23.0,27.0)	27.0(26.0,29.0)	25.0(23.0,26.0)	<0.001

Values are presented as the mean ± SD for normally distributed continuous variables, median (interquartile range) for non-normally distributed continuous variables, and n (%) for categorical variables.

§ Normally distributed variables were compared using the Student’s t-test, and non-normal distributed variables were compared using the Mann-Whitney U test. Categorical variables are presented as percentages and were compared using the Chi-square test.

Abbreviations: CHD, coronary heart disease; BMI, body mass index; spKt/V, single-pool Kt/V; Hb, hemoglobin; Alb, albumin; iPTH, intact parathyroid hormone; MoCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination.

Characteristics of cognitive impairment

Of the 613 subjects, cognitive impairment was diagnosed in 496 (80.91%) using the DSM-V, and 117 (19.09%) were determined to have normal cognition. The T-scores in each cognitive domain in the cognitive impairment group were significantly lower than those in the cognitively unimpaired group (Table 2). Specifically, 75.69% (464/613) of patients were determined to have cognitive impairment according to the MoCA-BJ score, only 61.34% (376/613) of patients were determined to have CI according to the MMSE score.

Table 2

T-scores of the standard neuropsychological assessments of five cognitive domains in the cognitively unimpaired and cognitive impairment groups.

Cognitive domains	Total	Cognitively impaired	Cognitive impairment	Z value	P value
	n = 613	n = 117	n = 496
Global cognition
MoCA	21.0(19.0,24.0)	26.0(24.0,27.0)	21.0(18.0,23.0)	-13.389	<0.001
MMSE	25.0(23.0,27.0)	27.0(26.0,29.0)	25.0(23.0,26.0)	-10.944	<0.001
Memory	49.9(45.6,54.7)	54.7(51.5,58.3)	48.5(44.8,52.9)	-9.588	<0.001
AVLT 5	50.3(43.4,57.1)	53.7(46.8,60.6)	46.8(39.9,57.1)	-4.920	<0.001
AVLT1-5	49.0(43.1,56.5)	53.2(48.1,59.0)	47.3(40.8,55.7)	-5.805	<0.001
CFT- memory	48.3(42.3,57.6)	57.9(50.7,63.9)	47.1(41.1,53.1)	-8.940	<0.001
Attention	50.2(45.5,55.0)	56.6(53.0,59.9)	48.7(44.1,53.1)	-11.373	<0.001
SDMT	52.3(46.6,56.1)	56.6(53.4,60.6)	50.2(44.2,54.9)	-9.679	<0.001
TMT-A	49.4(44.6,55.1)	57.6(52.7,62.0)	47.0(42.1,52.7)	-11.427	<0.001
Executive function	50.9(46.6,54.7)	55.6(53.2,58.1)	49.7(45.8,53.6)	-10.851	<0.001
TMT-B	50.1(43.6,57.2)	57.9(53.2,62.8)	48.2(40.8,54.7)	-9.844	<0.001
SCWT-C	53.3(46.7,56.0)	54.7(52.0,57.3)	53.3(45.3,56.0)	-5.526	<0.001
SCWT-T	51.4(47.6,56.0)	55.8(52.0,57.9)	50.6(47.0,55.0)	-7.781	<0.001
Language	50.4(45.9,54.8)	53.9(50.4,58.6)	49.7(45.2,53.9)	-6.736	<0.001
AFT	49.7(42.3,55.4)	55.4(47.9,61.0)	47.9(42.3,55.4)	-6.412	<0.001
BNT	54.8(48.2,58.1)	54.8(51.5,58.1)	51.5(44.9,54.8)	-3.608	<0.001
Visual space	54.7(45.5,57.0)	55.9(54.7,58.1)	52.5(41.2,56.9)	-8.374	<0.001
CFT-copy	54.7(45.5,57.0)	55.9(54.7,58.1)	52.5(41.2,56.9)	-8.374	<0.001

Abbreviations: MoCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination; AVLT, Auditory-Verbal Learning Test; CFT, Complex Figure Test; SDMT, Symbol Digit Modalities Test; TMT, Trail Making Test; SCWT, Stroop Color, and Word Test; AFT, Animal verbal fluency test; BNT, Boston Naming Test

Associations between the MoCA-BJ, MMSE, and the standard neuropsychological battery of DSM-V

The Spearman’s rank correlation analysis indicated that the MoCA-BJ was correlated with the composite scores of the five cognitive domains (rs = 0.639, p<0.001), specially in the domains of executive function (rs = 0.775, p<0.001), attention (rs = 0.564, p<0.001), visual space (rs = 0.457, p<0.001), memory (rs = 0.378, p<0.001) and language (rs = 0.237, p<0.001), whereas the MMSE had a weaker correlation with the standard neuropsychological tests (Table 3). The linear regression analysis revealed that the composite score explained 37.1% (β = 0.998, R2 = 0.371, p<0.001) of the variance in the MoCA-BJ and the covariates including age (β = 0.18, R2 = 0.062, p<0.05), sex (β = 0.12 R2 = 0.051, p<0.05) and education (β = 0.23, R2 = 0.084, p<0.05) only marginally affected variance in the performance.

Table 3

The correlations of the Beijing version of the Montreal Cognitive Assessment and the Mini-Mental State Examination with composite T-scores and each cognitive domain.

Cognitive domains	MoCA		MMSE
	rs	P-value	rs	P-value
Memory	0.378	<0.001	0.192	<0.001
Attention	0.564	<0.001	0.272	<0.001
Executive function	0.775	<0.001	0.353	<0.001
Language	0.237	<0.001	0.043	0.288
Visual space	0.457	<0.001	0.211	<0.001
Composite	0.639	<0.001	0.296	<0.001

Abbreviations: MoCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination

ROC curve analysis

The ROC analysis identified an optimal cut-off for the MoCA at ≤ 24 points (Table 4), with a sensitivity of 87.7% (95% CI, 0.851–0.903), a specificity of 75.2% (95% CI, 0.716–0.788) and an area under the curve (AUC) of 0.891 (95% CI, 0.859–0.924). In comparison, the MMSE only had a sensitivity of 70.4% (95% CI, 0.668–0.740), and a specificity of 76.9% (95% CI, 0.736–0.802, and AUC of 0.823 (95% CI, 0.786–0.860) for an optimal cut-off of ≤ 26 points (Table 4, Fig 1).

Table 4

Discriminant validity of the Beijing version of the Montreal Cognitive Assessment and the Mini-Mental State Examination for detecting cognitive impairment.

MoCA					MMSE
Cutoff ≤	Sen	Spe	PPV	NPV	Cutoff ≤	Sen	Spe	PPV	NPV
20	0.345	0.983	0.988	0.261	20	0.101	1.000	1.000	0.208
21	0.492	0.983	0.992	0.313	21	0.111	1.000	1.000	0.210
22	0.657	0.880	0.959	0.377	22	0.169	1.000	1.000	0.221
23	0.806	0.803	0.946	0.495	23	0.234	1.00	1.000	0.235
24†	0.877	0.752	0.938	0.591	24	0.371	1.000	1.000	0.273
25	0.942	0.598	0.909	0.707	25	0.498	0.897	0.954	0.297
26	0.988	0.393	0.873	0.885	26†	0.704	0.769	0.928	0.380
27	1.000	0.222	0.845	1.000	27	0.792	0.701	0.918	0.443
28	1.000	0.103	0.825	1.000	28	0.869	0.470	0.874	0.458
29	1.000	0.103	0.825	1.000	29	0.946	0.299	0.851	0.565

Abbreviations: MoCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination; Sen, Sensitivity; Spe, Specificity; PPV, positive predictive value; NPV, negative predictive value.

† Optimal cutoff score determined using Youden’s index.

Fig 1

Receiver operating characteristic curve analysis of the Mini-Mental State Examination and the Montreal Cognitive Assessment.

Discussion

In this multicenter observational study, we validated that the MoCA-BJ was an effective and sensitive screening tool for the early detection of cognitive impairment identified by the DSM-V in patients undergoing HD. Furthermore, the refined cutoff point of 24 for the MoCA-BJ could provide optimal sensitivity and specificity in screening for cognitive impairment in these patients. However, the MMSE only had a lower power of discrimination of cognitive impairment in hemodialysis patients.

Previous validation studies have evaluated the efficacy of the MoCA in screening cognitive impairment in a variety of populations[9, 27, 28], although the original data collected in a group of patients with AD and mild cognitive impairment (MCI) showed the optimal cut-off value should be less than 26[20], several international validations showed wide differences in thresholds, sensitivity and specificity values [29]. Damian AM et al [30] evaluated the predictive value of the MoCA in the detection of cognitive impairment in 135 American subjects, the results showed that the MoCA threshold of 26 appeared to be optimal in primary care. Luis CA et al [31] also found that Using the recommended cut-off score of 26, the MoCA detected 97% of those with cognitive impairment but specificity was fair (35%). Using a lower cut-off score of 23, the MoCA exhibited excellent sensitivity (96%) and specificity (95%) in 118 English-speaking older American adults. All these validation analyses remind us to make further examination about the application of the MoCA in different kinds of patients. However, the reliability and validity of the MoCA in patients undergoing HD have not been fully explored [32, 33]. Recently, Tiffin-Richards FE et al [13] demonstrated that the MoCA had good levels of sensitivity and specificity for screening cognitive impairment in a moderately sized sample of HD patients. In another validation study, Lee SH et al [34] compared the Korean version of the MoCA to the MMSE in 30 patients undergoing HD with a matched control group; the results indicated that the Korean version of the MoCA seemed to be more sensitive than the MMSE in HD patients. Our results revealed that the MoCA-BJ was more sensitive and had more predictive power than the MMSE in the detection of mild CI in the Chinese HD population (Table 4, Fig 1), which was consistent with the findings of the abovementioned studies and validated these results in a large group of patients undergoing HD. This is also the first validation analysis of MoCA-BJ among Chinese patients undergoing HD.

Exploring the reason why the MoCA-BJ is superior to MMSE in patients undergoing HD, the features of cognitive impairment in patients undergoing HD and the contents of the MoCA-BJ might be related. First, both those of previous studies and our data indicated that cognitive impairment in patients undergoing HD was more related to the injuries in the domains of executive function and visuospatial abilities [35]. In addition, patients undergoing HD often have a high prevalence of cerebrovascular risk factors, such as older age, hypertension, diabetes, dyslipidemia, high total plasma homocysteine level, oxidant stress, and inflammation [36, 37]. Secondly, the MoCA-BJ incorporated subtests assessing executive functions and psychomotor speed, which are frequently impaired in vascular cognitive impairment. In these regards, the focus of the MoCA on executive functions was believed to be one of the main reasons why it was superior to the MMSE in HD populations. The MMSE was designed for the detection of cognitive deficits observed in the process of dementia, particularly regarding attention, memory, and language functions. It lacks the ability to assess executive domains; thus, its sensitivity for detecting cognitive impairment in patients undergoing HD is lower than that of the MoCA.

Another important result of our study was that the optimal MoCA-BJ cutoff score was 24, as determined by Youden’s index, which was lower than the commonly recommended cutoff score of 26 in the general population. Although the general MoCA-BJ cutoff (≤26) had excellent sensitivity (0.942), the specificity was poor (0.598) in this group of patients undergoing HD. When we applied the optimal cutoff (i.e., ≤24), the specificity improved to 0.752, which was nearly equivalent to that of the MMSE (0.795), but the sensitivity was reduced to 0.877, although this was still higher than that of the MMSE (0.704). Several other studies have determined lower values in different populations e.g., a cutoff of 23.5 in a population with mild cognitive impairment [38] and of 21/22 in a population with cerebral small vessel disease [39]. The possible reason for this difference might be attributed to the difference between the original MoCA study and our study about age, race, education level, clinical history, and the diagnostic criteria for cognitive impairment. It seems that the optimal cutoff value should be applied when the MoCA-BJ is used to assess Chinese patients undergoing HD.

Some strengths of our study include the detailed neuropsychological testing with published norms on all tests in all cognitive domains, and the diagnosis of cognitive impairment based on the criteria of the DSM-V, which enabled a comprehensive and standard evaluation of cognition and the interpretation of concurrent validity. In addition, to improve the quality and reliability of our study, all neuropsychological tests were performed by research staff, who were trained centrally by the study neuropsychologist and received certificates before the study commencement. However, there were still several limitations associated with our study. Given that the present data were obtained from Chinese patients undergoing HD in Beijing, these results may not be generalizable to other areas in China, because there is much geographic and cultural variability across China, and specific regional characteristics may influence the performance of the residents on the neuropsychological tests. Future studies should include patients undergoing HD from more regions of the country.

In conclusion, the MoCA-BJ appears to be a good sensitivity and specificity levels in detecting cognitive impairment in hemodialysis patients. Our data support that the MoCA-BJ can be applied for the screening of cognitive impairment in routine practice in these patients.

STROBE statement.

(DOCX)

Click here for additional data file.

Basic characteristics of the participants.

(XLSX)

Click here for additional data file.

Neuropsychological tests of five cognitive domains of the participant.

(XLSX)

Click here for additional data file.

30 Oct 2019

PONE-D-19-21883

The validation of the Beijing version of the Montreal Cognitive Assessment in Chinese patients undergoing hemodialysis

PLOS ONE

Dear Dr. Luo,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Dec 14 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Jeannie-Marie Sheppard Leoutsakos, PhD

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

3. There are several cases where P-values are reported as being equal to 0. Please correct and clarify.

4.  We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The ms. is focused on a contribution to the validation of the Beijing version of the MoCA (MoCA-BJ) for detecting Cognitive Impairments among a group of haemodialysis patients.

I have some concerns with the ms. In particular:

• Review the Abstract

Please revise the English

• Review the Introduction:

I suggest to improve this section, it is very gaunt, providing scarce information. First of all, it is not clear which kind of cognitive impairment the authors are talking about. Moreover, before starting to investigate the prevalence of the disorder in the hemodialysis population, it’d be good to do a general overview of the disorder, also in other populations, citing recent articles published on the same topic such as Bosco et al., 2017; Siciliano et al., 2019; Huang et al., 2018.

• Review the Subjects and Methods section:

Rename the section “method”;

Move the subsection “ethics statement” before the “statistical analysis”;

Rename the subsection “Study participants” in “participants”. Add in the text some demographic information such as means and standard deviation about age and level of education, and gender composition, as well. the paragraph needs to be rewrite taking into account English grammar and syntactic rules.

Au.s should clarify in the subsection “Classification of cognitive function” what they intend for “cognitive impairment”, specifying better also the criteria of DSM-5 used to diagnose the impairment. Reading is not easy.

Add references after Roc Curves and Youden Index in the statistical analysis section.

• Review the Results section and Discussion

In table 1 substitute the label p-value with Test

Add references after this sentence: “Previous studies have evaluated the efficacy of the MoCA and the MMSE in screening cognitive impairment in a variety of populations”

It is necessary to check some typos and errors, and, again, to revise the English in order to make the ms. easily legible.

• Notwithstanding the scarce literature about haemodialysis patients affected by “cognitive impairment” it is necessary to integrate the bibliographical apparatus with more recent research.

Reviewer #2: This study details the use of the MoCA-Beijing (BJ) in hemodialysis patients. The authors report on the MoCA-BJ’s correlation with the MMSE as well as with neuropsychological tests. Diagnostic accuracy of the MoCA-BJ is also assessed and the authors find that the optimal cutoff for cognitive impairment is 24, which is consistent with several previously published studies of the English version of the MoCA. Below are my comments.

General Comments

1. Although the English is generally good, there are several instances where the text is grammatically incorrect.

2. Throughout the manuscript, the authors refer to their subjects as cognitive impairment and non-cognitive impairment. The preferred term for non-cognitive impairment is “cognitively unimpaired” (Jack CR, et al. NIA-AA Research Framework: toward a biological definition of Alzheimer’s disease. Alzheimers Dementia. 2018;14.4:535–62.)

Statistical Analysis

1. When describing the analysis of categorical variables, please spell out Chi-square and only use the Greek symbol for Chi when reporting the results from statistical tests.

Results

1. In the bivariate linear regression model, were any demographic variables such as age, sex, and education adjusted for? This is particularly relevant on page 13 where the authors report the amount of variance explained. Perhaps they could report R-squared values from adjusted and unadjusted models.

2. Throughout the text and in the tables, p-values of 0.000 are reported. Although this is the default output for small p-values in SPSS, this is not the way they should be reported. For all cases where the p-value is 0.000, please replace with <0.001.

3. For the measures of diagnostic accuracy, the authors report 95% CIs for the AUC values but not for sensitivity and specificity. 95% CIs for sensitivity and specificity should be reported as well. If these are not given by SPSS, the equation for their calculation is relatively simple and is easily available on a number of reputable websites.

Discussion

1. The authors’ finding that 24 is the optimal cutoff for the identifying cognitive impairment is similar to other findings and these studies should be cited and briefly discussed:

Damian, A.M., Jacobson, S.A., Hentz, J.G., Belden, C.M., Shill, H.A., Sabbagh, M.N., et. al (2011). The Montreal Cognitive Assessment and the Mini-Mental State Examination as screening instruments for cognitive impairment: Item analyses and threshold scores. Dementia and Geriatric Cognitive Disorders, 31, 126-131.

Coen, R.F., Cahill, R., & Lawlor, B.A. (2011). Things to watch out for when using the Montreal Cognitive Assessment (MoCA). International Journal of Geriatric Psychiatry, 26, 106- 108.

Luis CA, Keegan PA, & Mullan M. (2009). Cross validation of the Montreal Cognitive

Assessment in community dwelling older adults residing in the Southeastern US. International Journal of Geriatric Psychiatry, 24, 197-201.

Gluhm, S., Goldstein, J., Loc, K., Colt, A., Van Liew, C., & Corey-Bloom, J. (2013). Cognitive performance on the Mini-Mental State Examination and the Montreal Cognitive Assessment across the healthy adult lifespan. Cognitive and Behavioral Neurology, 26, 1-5.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Prof. Andrea Bosco PhD

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

23 Nov 2019

Nov 22th 2019

PONE-D-19-21883 (The validation of the Beijing version of the Montreal Cognitive Assessment in Chinese patients undergoing hemodialysis)

PLOS ONE

Dear Editor Jeannie-Marie Sheppard Leoutsakos

Here is the response letter from the authors of PONE-D-19-21883, we would like to express our appreciation to you and reviewers for your kind work and valuable suggestions. Now we have read the comments carefully and revised our manuscript accordingly. Here below are some of answers to the questions of the reviewers.

At the same time, we understand that our language ability is limited, so we have sent our manuscript to the editage for language editing, and we have uploaded the certificate of that editing to you. We also uploaded the anonymized data set of our study as supplementary documents for replicating our study founding.

If there are still other limitations, please let us know, and we are looking forward to hearing from you soon.

Sincerely

Yang Luo.

Reviewer #1: The ms. is focused on a contribution to the validation of the Beijing version of the MoCA (MoCA-BJ) for detecting Cognitive Impairments among a group of haemodialysis patients.

I have some concerns with the ms. In particular:

• Review the Abstract

Please revise the English

We have asked the Editage, one of renown English language editing workshops, to improve our expression in the manuscript. We have uploaded the certificate of the editing by the Editage as a supplementary document.

• Review the Introduction:

I suggest to improve this section, it is very gaunt, providing scarce information. First of all, it is not clear which kind of cognitive impairment the authors are talking about. Moreover, before starting to investigate the prevalence of the disorder in the hemodialysis population, it’d be good to do a general overview of the disorder, also in other populations, citing recent articles published on the same topic such as Bosco et al., 2017; Siciliano et al., 2019; Huang et al., 2018.

We fully agree with your opinion that the cognitive impairment in the hemodialysis patient includes mild and major cognitive impairment. We have already learned the papers that you recommended to us and try our best to add information about the related cognitive impairment in other populations in our introduction.

• Review the Subjects and Methods section:

Rename the section “method”; we have changed accordingly.

Move the subsection “ethics statement” before the “statistical analysis”. We have moved the ethics statement to the right place.

Rename the subsection “Study participants” in “participants”. Add in the text some demographic information such as means and standard deviation about age and level of education, and gender composition, as well. the paragraph needs to be rewrite taking into account English grammar and syntactic rules. We have renamed this subsection as “participants” under your suggestion, and provide some demographic information.

Au.s should clarify in the subsection “Classification of cognitive function” what they intend for “cognitive impairment”, specifying better also the criteria of DSM-5 used to diagnose the impairment. Reading is not easy. We have revised this section, try to make it clear to express the diagnosis criteria of cognitive impairment.

Add references after Roc Curves and Youden Index in the statistical analysis section. We have added references in this section.

• Review the Results section and Discussion

In table 1 substitute the label p-value with Test. We have added notes in Table 1 to indicated the tests used in the analysis.

Add references after this sentence: “Previous studies have evaluated the efficacy of the MoCA and the MMSE in screening cognitive impairment in a variety of populations”. We have added related references and introduced some of the important results from previous studies.

It is necessary to check some typos and errors, and, again, to revise the English in order to make the ms. easily legible. As we have mentioned above, we provided the certificate of language editing from the Editage.

• Notwithstanding the scarce literature about haemodialysis patients affected by “cognitive impairment” it is necessary to integrate the bibliographical apparatus with more recent research. We agree with your suggestion and searched the related papers from Pubmed and revised our manuscript accordingly.

Reviewer #2: This study details the use of the MoCA-Beijing (BJ) in hemodialysis patients. The authors report on the MoCA-BJ’s correlation with the MMSE as well as with neuropsychological tests. Diagnostic accuracy of the MoCA-BJ is also assessed and the authors find that the optimal cutoff for cognitive impairment is 24, which is consistent with several previously published studies of the English version of the MoCA. Below are my comments.

General Comments

1. Although the English is generally good, there are several instances where the text is grammatically incorrect. Thank you for the suggestions. In order to improve the quality of our manuscript, we have asked the Editage, one of renown English language editing workshops, to improve our expression in the manuscript. We have uploaded the certificate of the editing by the Editage as supplementary document.

2. Throughout the manuscript, the authors refer to their subjects as cognitive impairment and non-cognitive impairment. The preferred term for non-cognitive impairment is “cognitively unimpaired” (Jack CR, et al. NIA-AA Research Framework: toward a biological definition of Alzheimer’s disease. Alzheimers Dementia. 2018;14.4: 535–62.)

We have read the article and changed the old terms in our manuscript, thank you for your suggestion.

Statistical Analysis

1. When describing the analysis of categorical variables, please spell out Chi-square and only use the Greek symbol for Chi when reporting the results from statistical tests.

We have changed Greek symbols into normal spelling words, thank you again.

Results

1. In the bivariate linear regression model, were any demographic variables such as age, sex, and education adjusted for? This is particularly relevant on page 13 where the authors report the amount of variance explained. Perhaps they could report R-squared values from adjusted and unadjusted models.

After reading some references, we understood the importance to add some demographic variables like age, sex, and education in our linear regression analysis, we have provided the related R-squared in the related section of our results.

2. Throughout the text and in the tables, p-values of 0.000 are reported. Although this is the default output for small p-values in SPSS, this is not the way they should be reported. For all cases where the p-value is 0.000, please replace with <0.001.

We have replaced the old expression and changed in the style as you suggested.

3. For the measures of diagnostic accuracy, the authors report 95% CIs for the AUC values but not for sensitivity and specificity. 95% CIs for sensitivity and specificity should be reported as well. If these are not given by SPSS, the equation for their calculation is relatively simple and is easily available on a number of reputable websites.

We have added the 95% CIs for the sensitivity and specificity in the results of ROC analysis.

Discussion

1. The authors’ finding that 24 is the optimal cutoff for the identifying cognitive impairment is similar to other findings and these studies should be cited and briefly discussed:

We have downloaded these papers and cited them in our discussion. The contents of these papers provide a lot of important information in this area, we would like to thank you again.

Damian, A.M., Jacobson, S.A., Hentz, J.G., Belden, C.M., Shill, H.A., Sabbagh, M.N., et. al (2011). The Montreal Cognitive Assessment and the Mini-Mental State Examination as screening instruments for cognitive impairment: Item analyses and threshold scores. Dementia and Geriatric Cognitive Disorders, 31, 126-131.

Coen, R.F., Cahill, R., & Lawlor, B.A. (2011). Things to watch out for when using the Montreal Cognitive Assessment (MoCA). International Journal of Geriatric Psychiatry, 26, 106- 108.

Luis CA, Keegan PA, & Mullan M. (2009). Cross validation of the Montreal Cognitive

Assessment in community dwelling older adults residing in the Southeastern US. International Journal of Geriatric Psychiatry, 24, 197-201.

Gluhm, S., Goldstein, J., Loc, K., Colt, A., Van Liew, C., & Corey-Bloom, J. (2013). Cognitive performance on the Mini-Mental State Examination and the Montreal Cognitive Assessment across the healthy adult lifespan. Cognitive and Behavioral Neurology, 26, 1-5.

Submitted filename: Respose to reviewers.docx

Click here for additional data file.

12 Dec 2019

The validation of the Beijing version of the Montreal Cognitive Assessment in Chinese patients undergoing hemodialysis

PONE-D-19-21883R1

Dear Dr. Luo,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Jeannie-Marie Sheppard Leoutsakos, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: It seems to me that authors have made all the requested revisions.

Minor concers

In the list of references I found after some authors' names, the fragment "AUID- Oho", please identify it and delete.

Reviewer #2: The authors have responded sufficiently to my comments. No additional revisions are needed for this manuscript.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Prof. Andrea Bosco

Reviewer #2: Yes: Michael Malek-Ahmadi

19 Dec 2019

PONE-D-19-21883R1

The validation of the Beijing version of the Montreal Cognitive Assessment in Chinese patients undergoing hemodialysis

Dear Dr. Luo:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Jeannie-Marie Sheppard Leoutsakos

Academic Editor

PLOS ONE