Internal consistency and test-retest reliability of the Chinese version of the 5-item Duke University Religion Index.

BACKGROUND: The Duke University Religion Index (DUREL) is a widely-used 5-item scale assessing religiosity. AIM: Assess the internal consistency, reliability, and factor structure of the revised Chinese version of DUREL.
METHODS: Using probability proportionate to size (PPS) methods we randomly identified 3981 households with eligible occupants in 20 primary sampling sites in Ningxia Hui Autonomous Region, a province in northwest China in which 34% of the population are Muslims of the Hui ethnic group. In 3054 households a screening interview was completed and an adult family member was randomly selected; 2425 respondents completed the survey (including the DUREL) and 188 randomly selected individuals repeated the survey an average of 2.5 days later.
RESULTS: The internal consistency (Cronbach's α) of the 5 items in the full sample was 0.90; it ranged from 0.70 to 0.90 in various subgroups of subjects stratified by ethnicity, urban versus rural residence, and above versus below median education. The test-retest reliability (intraclass correlation coefficient) for the total score in the full sample was 0.87; it ranged from 0.63 to 0.90 in the different subgroups of subjects. Exploratory factor analysis in a random half of the sample identified a single factor (eigen value=4.21) that explained 84% of the total variance. Confirmatory factor analysis in the second half of the sample confirmed the unidimensional model; the model fit measures of the one-factor model using the 5 item scores as observed variables were acceptable (comparative fit index [CFI] and Tucker-Lewis index [TLI]>0.99; root mean square error of approximation [RMSEA]=0.105; χ (2) =70.49, df=5), but the model fit improved after adding the correlation between items 1 and 2 (that assess organized and personal religious activities, respectively) as a sixth observed variable(CFI and TLI>0.99; RMSEA=0.046; χ (2) =14.32, df=4).
CONCLUSION: The Chinese version of the DUREL is a reliable and valid measure of religiosity that can be used to assess the relationship of religiosity/spirituality to physical and psychological wellbeing in Chinese respondents. As suggested by other authors, our factor analysis results indicate that the overall score is the best measure derived from the scale, not the three dimensional scores recommended by the original authors.

Introduction

Religiosity and spirituality are closely associated with psychological wellbeing and the occurrence and recovery of mental disorders.[1],[2],[3],[4],[5] Ningxia Hui Autonomous Region (hereafter, Ningxia) is a relatively poor, sparsely populated province in northwestern China in which 34% of the population are Muslims of the Hui ethnic group and the remaining 66% are almost all atheists of Han ethnicity (the main ethnic group in China), so it is an ideal setting in which to assess the relationship of religion and psychological wellbeing.[6] In these types of settings, accurately measuring the relationship between religiosity (spirituality) and mental health plays an important role in developing targeted mental health promotion programs. Thus, developing instruments that can validly and reliably assess individuals’ level of religiosity is an essential first step in providing ethnic group-appropriate mental health prevention and treatment services.

The Duke University Religion Index (DUREL) is a 5-item scale that assesses organizational religious activity (1 item), nonorganizational religious activity (1 item), and intrinsic religiosity (3 items).[7] Most authors follow the recommendations of the originators and report separate scores for these three subscales rather than reporting a total summary score for all five items,[8],[9],[10],[11],[12] but some researchers employing confirmatory factor analysis only identify a single dimension (factor), which suggests that a total score would be more appropriate.[13],[14]

Two previous studies have assessed our adapted Chinese version of this scale (see Appendix 1 for translation) in select samples. A study[8] with 1410 Ningxia university students reported a Cronbach alpha (internal consistency) for the five items of 0.85 and a repeat assessment with 105 of the students found intraclass correlation coefficients of the three subscale scores ranging from 0.27 to 0.88. Another study[9] in 1039 women 18 to 34 years of age from rural Hebei Province also reported good internal consistency of the five items of (alpha=0.78).

The reliability and validity of this scale in the general Chinese population has not yet been assessed. It is also uncertain whether or not the Chinese version of the scale has the three dimensions reported in the original version of the scale – organizational, nonorganzational, and intrinsic religiosity. The current study assesses the reliability and validity of the Chinese version of DUREL in a large randomly selected sample of community dwelling residents of Ningxia and uses the results of the survey to assess the factor structure of the questionnaire in Chinese respondents.

Methods

This report is part of the Epidemiological Investigation and Health System Interventions for Mental Health in rural Western China conducted by the School of Public Health at Ningxia Medical University in collaboration with the Shanghai Mental Health Center.

Sampling

The flowchart for the identification and enrollment of participants is show in Figure 1. Based on the 2010 census, the 6.3 million residents of Ningxia live in 7 urban and 15 rural counties. The 22 counties were stratified into 4 groups of counties based on the proportion of residents who lived in urban communities and the proportion of residents who were of the Hui ethnic group: one urban county had a majority of Hui residents, 6 urban counties had a majority of Han residents, 7 rural counties had a majority of Hui residents and 8 rural counties had a majority of Han residents. Based on the population proportions of these four types of counties, the total target sample of 2500 adult community-resident adults included 104 from the single urban county with predominantly Hui residents, 805 residents from urban counties with predominantly Han residents, 841 residents from rural counties with predominantly Hui residents and 750 residents from rural counties with predominantly Han residents. To achieve this sample a total of 10 counties were randomly selected from the 4 classes of counties using probability proportionate to size (PPS) methods. The ten counties included the single urban-Hui county, 3 of the 6 urban-Han counties, 3 of the 7 rural-Hui counties and 3 of the 8 rural-Han counties. Based on a pilot study in 3 neighborhoods and 2 villages (in May 2013), rural residents were easier to locate and enroll than urban residents, so in rural counties the number of randomly selected households (from which the randomly selected adult respondents would be selected) was twice the target sample size for the county while in urban counties the number of randomly selected households was 4-fold the target sample size for the county. At the next step 9 urban neighborhoods and 11 rural villages (that is, a total of 20 primary sampling units [PSUs]) were randomly selected using PPS methods from the 148 neighborhoods and villages in the 10 selected counties. From 1 to 3 PSUs were selected from each of the 10 counties based on the estimated number of households that would need to be visited (with a minimum of 50 households and a maximum of 300 households at each of the PSU).

Four supervisors visited each PSU prior to the survey to randomly select the target households. The households were selected from the residence registry maintained by the local community officials, but these lists were often inaccurate so the researchers had to discuss the list with local residents and (occasionally) survey the number of residential buildings in the community to update the list prior to selecting households. After all households were enumerated (giving each household a unique ID number starting from 1, 2, 3, etc.) a selection interval was estimated (i.e., the rounded whole number of the result of dividing the total number of households in PSU by the number of households that needed to be selected in the PSU) and a random starting number smaller than the interval was determined (using Excel). The household with the ID number the same as the selected random number and households with ID numbers equal to the random number plus multiples of the interval number were then selected.

Conduct of the survey

Four teams were involved in completing the surveys at the 20 PSUs, which were conducted two to three days after the sampling was completed. Each team consisted of a supervisor, a coordinator, 6 to 12 investigators (students at the School of Public Health of Ningxia Medical University) and 1 to 5 local guides (local administrators, doctors, women’s cadres, etc.). The supervisors and coordinators were trained by the study PIs (ZW and MP) in a 3-day course. They subsequently trained the investigators in a day-long session that discussed the specific procedures in completing both the household screening questionnaire and the main survey; this training of investigators included role-play exams to ensure their compliance to the protocol.

After the team arrived at each PSU the supervisor identified appropriate guides (with the help of a local health official) and spent a couple of hours with them explaining the aims of the project and the method they were expected to use when introducing the project to prospective participants. The guides helped locate the selected households and gave persons living in the households a general introduction to the project, after which they introduced the investigator who then conducted the household screening questionnaire. Based on the result of the household questionnaire (which typically took 10-15 minutes to complete) the investigator randomly selected one adult household member as the target participant for the main survey. If the selected individual was present at the time, he or she was asked to sign the consent form and the main survey was completed immediately (which took to a mean [sd] of 44.4 (18.3) minutes to complete). If the identified individual was not present, an appointment was made for the investigator to return and complete the survey later. Both the household screening questionnaire and the main survey were read to participants by the investigator. After finishing the survey, the subjects were given a gift worth 20 Renminbi (about $3 US) for their time. The teams spent an average of 6.5 days at each PSU.

The main investigation was conducted from 18 July 2013 to 26 October 2013. As shown in Figure 1, 7032 households were selected but in 3051 (43%) of these households residents were not located or not eligible. This was primarily because the households were not occupied, a common situation in both urban and rural China. In 927 of the 3981 households (23%) in which investigators had contact with eligible residents, the interviewed residents refused to complete the household screening questionnaire. The household screening questionnaire listed the characteristics of all residents of the selected household who had lived in the household for at least 50% of the time in the previous three months (regardless of where their formal residence permit was located). The respondent for the survey was randomly selected from among the adult residents in the household (i.e., 18 years of age or older) listed in the family questionnaire. Of the 3054 target subjects identified in the household questionnaires, 629 (21%) did not complete the questionnaire, primarily because the identified individual was not present in the household over the time the survey was conducted in the PSU. A total of 2425 individuals successfully completed the survey.

Among the 299 individuals randomly selected to repeat the survey (to assess test-retest reliability), 188 (63%) successfully completed the survey a second time a mean of 2.5 [1.4] days after the first administration of the survey. The administrator of the repeat survey was blind to the result of the first administration of the survey.

Assessments

The DUREL was revised for use in China. The back translation of the Chinese version of the scale is shown in Table 1 and the Chinese version of the scale is shown in Appendix 1. Based on pilot testing with Chinese respondents there have been three main changes in the Chinese version of the scale: (1) the response set is reversed with the options going from low frequency to high frequency rather than the reverse; (2) items 3, 4 and 5 are posed as questions (like items 1 and 2) rather than being presented as statements in the first person (e.g., item 3 in the Chinese version is “In your life, do you experience the presence of the Divine?” but in the original English version it is “In my life, I experience the presence of the Divine.” ). (3) The focus of item 5 has changed to emphasize the active promotion of religious beliefs to others (i.e., “Do you try hard to promote your religious beliefs to people around you?” in the Chinese version versus “I try hard to carry my religion over into all other dealings in life.” in the original English version).

The first item in the 5-items scale assesses organizational religious activity on a 6-point Likert scale. The second item assesses nonorganizational religious activity on a 6-point Likert scale. And items 3 to 5 assess intrinsic religiosity on three 5-point Likert scales. Taken together these five items have a theoretical range of scores from 5 to 27.

The DUREL is only one of a battery of scales that were included in the overall survey. Demographic characteristics of the respondents collected as part of the overall survey were also used in this analysis.

Statistical methods

The data were prepared using double entry verification in EpiData 3.1. SPSS 20.0 was used for the analysis. Reliability analysis and test-retest analysis were done in all subjects and in subgroups of respondents based on residence (urban versus rural), ethnicity (Han versus Hui), and educational level (above and below the median level of education). Cronbach’s alpha was used to assess internal consistency of the Chinese version of the DUREL and intra-class correlation coefficients (ICC) were used to measure the test-retest reliability of the scale.

Exploratory factor analysis and confirmatory factor analysis were used to assess the factor structure of the Chinese version of the DUREL. The responses to each item were skewed to the right (see Table 1), so the five item scores were used as ordered categorical (i.e., ordinal) variables in these analyses. Subjects were first randomly allocated into two groups. Exploratory factor analysis was performed on the first group (n=1231) using geomin(oblique) rotation. Eigenvalues, a scree plot, and item factor loadings were used to determine the most appropriate number of factors; the number of factors with eigenvalues above 1 was used to determine the number of factors in the data set. AMOS 17.0 and Mplus 7 software [15] were used to conduct confirmatory factor analysis on the second group of respondents (n=1194). This process assesses the stability of the factor structure identified in the exploratory factor analysis by comparing the predicted covariance matrix to the observed covariance matrix. The robust weighted least square method was used to estimate the factor loadings, the variance of the latent variable was fixed at 1 (so the loadings of the observed variables can be freely estimated), and the indices used to access model fit were chi-squared, comparative fit index (CFI), Tucker-Lewis index (TLI) and root mean square error of approximation (RMSEA). Values of CFI and TLI>0.95 indicate good model fit; RMSEA values of <0.05 indicate good model fit and RMSEA values of >0.1 indicate that modifications of the model are needed.[16]

Results

Table 2 shows the demographic characteristics of the 2425 respondents who completed the survey and in the subgroup of 188 respondents who repeated the survey in the test-retest part of the study. All participants had a mean (sd) age of 45.7 (15.3) years and those who repeated the survey had a mean age of 47.8 (15.5) years. With the one exception that the proportion of women who repeated the survey was greater than the proportion that did not repeat the survey (64% vs. 55%, χ2 =5.90, p=0.015), the characteristics of the 188 individuals who repeated the survey were not statistically different from those of the 2237 respondents who did not repeat the survey.

Table 1 shows the basic response pattern in all 2425 respondents for the 5 items in the scale. As expected, the results in the total sample (the majority of whom are atheists) are shifted to the right (i.e., most respondents report no religious activities or beliefs). The mean (sd) scores of the five items in the total sample were 1.7 (1.3), 1.4 (1.2), 1.8 (1.4), 1.9 (1.5) and 1.8 (1.4), respectively. The mean total score (range 5-27) was 8.5 (5.8). Spearman’s correlation coefficient (rs) between the five item scores ranged from 0.50 to 0.82.

Internal consistency of the five items (at the first administration of the scale) was assessed using Cronbach’s alpha. The alpha value for the full sample was 0.90; it was 0.79 in respondents of Han ethnicity, 0.70 in respondents of Hui ethnicity, 0.86 in urban respondents, 0.90 in rural respondents, 0.87 in respondents with 9 years of formal education or more, and 0.90 in respondents with less than nine years of formal education.

Results of the exploratory factor analysis are shown in Table 3 and in the scree plot (Figure 2). Only one of five components’ eigenvalues was greater than 1. This component explained 83.8% of the total variance. The loadings of items 1 to 5 on this factor in the geomin (i.e.,oblique) rotated matrix were 0.837, 0.876, 0.914, 0.974 and 0.943, respectively. These results clearly suggest that the one-factor solution is optimal.

This unidimensional model of the Chinese version of DUREL was then examined in the second half of the sample using confirmatory factor analysis. As shown in the left panel of Figure 3, factor loadings of the five items ranged from 0.84 to 0.98. The CFI and TLI were both greater than 0.99, indicating excellent goodness of fit. However, the RMSEA was 0.105 (90% CI=0.084, 0.127), which indicated suboptimal fit (χ2 =70.49, df=5). Considering that the first two items are both about active participation in religious activities (unlike the last three items), we added a correlation between these two items to the model. As shown in the right panel of Figure 3, the correlation between the residuals of items 1 and 2 was 0.14 (p<0.001) and the factor loadings in this revised model were essentially the same as in the original model (change<5%). All fit indices of the revised model suggested excellent fit (RMSEA=0.046, 90% CI=0.022, 0.074; CFI and TLI>0.99; χ2 =14.32, df=4).

Since there was only one dimension for the Chinese version of DUREL, test-retest reliability assessed in the 188 participants who completed the scale twice was based on the total score of the 5 items (ranging from 5 to 27). As shown in Table 4, the ICC of the Chinese version of DUREL was 0.87 in all participants and ranged from 0.63 to 0.90 in the different subgroups of respondents. The test-retest reliability results (ICC) of each of the 5 items in the scale were 0.83, 0.77, 0.61, 0.72 and 0.73, respectively. Thus the test-retest reliability of each item is satisfactory and the test-retest reliability of the total score is satisfactory both in the total sample and in all of the subsamples considered.

Discussion

Main findings

This study confirms the internal consistency and test-retest reliability of the Chinese version of the DURELL. We used a large, community-based sample in Ningxia, a province in China that is more religiously diverse than other parts of China because a substantial minority of the population is of the Hui ethnic group who, unlike the majority Han ethnic group, are Moslems. The internal consistency and test-retest reliability were good in both ethnic groups, in both urban and rural residents, and in individuals with more or less education.

We did not confirm the three-factor structure reported for the original English-language version of the scale. The exploratory factor analysis identified a single factor and the confirmatory factor analysis confirmed the one-factor solution, though the model was improved when a term representing the correlation of item 1 and item 2 (i.e., organized and private religious activities) was added to the model. Other researchers who conducted factor analysis of the original English language scale[13] and of the Persian translation of the scale[14] also reported a single factor.

Limitations

There were some limitations in the current study. (a) Among the randomly selected households that were occupied, 23% refused to participate in the screening interview and in 21% of the households that participated in the screening the household member randomly selected for the survey did not complete the survey (primarily because they were not available). Thus those who completed the survey may not have been fully representative of the population. (b) Similarly, 37% of the individuals randomly selected to complete the test-retest assessment were not located or refused so those who completed the test-retest assessment may not have been fully representative of all those who completed the survey the first time; those who completed the test-retest assessment were more likely to be female than those who did not complete the test-retest assessment but there was no difference in any of the other demographic variables between these two subsets of participants. (c) The very short test-retest interval (mean of 2.5 days), which was necessitated by the practical consideration of travelling to the 20 primary sampling units around the province, may have inflated the test-retest reliability measures. (d) In the current survey respondents were interviewed by trained investigators; the internal consistency and test-retest reliability of the scale may be different when administered as a self-completion instrument. (d) Perhaps most importantly, in the absence of a ‘gold standard’ measure of religiosity it was not possible to cross-validate the results of the scale by comparing it to other measures of religiosity.

Implications

The Chinese version of the DUREL is a reliable and valid measure of religiosity. The overall score (ranging from 5 to 27), rather than the individual items scores, is the best measure derived from the scale. Further work assessing the scale as a self-completion instrument and relating the results of scale to other proposed measures of religiosity would be useful, but the current study confirms its utility as a measure that can be used in other studies in China. Given the close relationship between religiosity/spiritually and wellbeing,[1],[2],[3],[4],[5],[17], the inclusion of this brief measure in assessments of the physical and psychological health of communities will add an important dimension to our understanding of these complex phenomena.

Table 1.

Back-translation of Chinese version of the 5-item Duke University Religion Index (DUREL) and percent of each level of response in the representative sample of 2425 adult community residents from Ningxia who completed the survey

1. How often do you attend group religious meetings? [i.e, mosques, churches, temples, puja, pilgrimage visit, and so forth]	never(67.5%)	once ayear orless(16.4%)	less thanonce amonth(7.7%)	less thanonce aweek(1.5%)	less thanweek(2.4%)	more thanonce aweek(4.5%)
2. How often do you spend time in private religious activities? [i.e, prayer, meditation, reading scriptures or Bible, worship, and so forth]	few ornever(87.4%)	less thanonce aweek(3.3%)	once aweek(2.0%)	more thanonce aweek(1.2%)	once aday(1.4%)	more thanonce aday(4.7%)
3. In your life, do you experience the presence of the Divine? [i.e, Allah, Buddha or God and ther apparitions]	definitely nottrue(73.2%)		tends not tobe true(2.6%)	unsure(7.1%)	tends to betrue(6.1%)	definitelytrue(67.5%)
4. Are your religious beliefs what really lie behind your whole approach to life?	definitely nottrue(71.1%)		tends not tobe true(3.1%)	unsure(5.5%)	tends to betrue(7.1%)	definitelytrue(13.3%)
5. Do you try hard to promote your religious beliefs to people around you?	definitely nottrue(75.1%)		tends not tobe true(3.3%)	unsure(3.5%)	tends to betrue(6.0%)	definitelytrue(12.2%)

Appendix 1.

Revised Chinese version of Duke University Religion Index (DUREL)

Duke大学宗教指数中文版
1. 您通常参加集体宗教活动的频率如何？ [如去清真寺、教堂、寺庙、法会、进香园等。]	从来不去	每年一次	少于每月一次	少于每周一次	每周一次	每周多于一次
2. 您独自从事宗教活动的频率如何？ [如一个人祈祷、打坐、诵读经书、礼拜等。]	很少或从来没有	少于每周一次	每周一次	每周多于一次	每天一次	每天多于一次
3. 现实生活中，您能感受到神灵的出现吗？ [如：真主、佛祖或上帝等显灵。]	绝对不会	也许不会	不确定	也许会	绝对会
4. 您的宗教信仰决定了您对待生活的态度吗？	绝对不会	也许不会	不确定	也许会	绝对会
5. 您努力把您的宗教信仰传播给您身边的人吗？	绝对不会	也许不会	不确定	也许会	绝对会

Table 2.

Characteristics of residents who completed the survey and of subgroup who completed the repeat survey

Characteristics	Residents whocompleted thefirst survey(n=2425)		Subsample ofresidents whocompletedrepeat survey(n=188)
	n	%	n	%
Gender
Female	1356	55.9%	121	64.4%
Male	1069	44.1%	67	35.6%
Residence
Urban county	1207	49.8%	95	50.5%
Rural county	1218	50.2%	93	49.5%
Ethnicity
Han	1846	76.1%	150	79.8%
Hui	561	23.1%	36	19.1%
Other	18	0.8%	2	1.1%
Educational level
less than 9 years ofeducation	1373	56.6%	112	59.6%
9+ years ofeducation	1052	43.4%	76	40.4%
Marital status
never married	193	8.0%	10	5.3%
married	2014	86.8%	167	88.8%
divorced, separatedor widowed	128	5.3%	11	5.9%
Religion
No religion	1583	65.3%	124	66.0%
Buddhist	211	8.7%	23	12.2%
Islamic	551	22.7%	37	19.7%
Other religion	80	3.3%	4	2.1%
Work
Peasant	881	36.3%	72	38.3%
Worker/serviceersonnel	294	12.14%	26	13.8%
Professional/cadre	334	13.8%	19	10.1%
Self-employed/temp worker	234	9.6%	15	8.0%
Retired/sick leave	239	9.9%	17	9.0%
Unemployed/outof work	139	5.7%	11	5.9%
Child/student/housewife	291	12.0%	28	14.9%
Other 13 0.6% 0 0.0%	13	0.6%	0	0.0%

Table 3.

Eigen values and percent of total variance explained in principal component exploratory factor analysis of one half of the sample (n=1231)

Component	Initial eigenvalues forgeneral factor analysis
	total	% of variance	cumulative %
1	4.21	84.2	84.2
2	0.43	8.5	92.7
3	0.17	3.5	96.2
4	0.13	2.5	98.7
5	0.07	1.3	100.0

Table 4.

Mean (sd) total scores of Duke University Religion index (DUREL) and intraclass correlation coefficient (ICC) in different groups of participants who completed the instrument twice

	Mean (sd) score for thefirst assessment		Mean (sd) score for therepeat assessment		ICC
	mean	sd	mean	mean
All participants(N=188)	8.3	5.3	7.8	5.4	0.87
Male participants (N=67)	8.7	6.0	8.4	6.3	0.88
Female participants (N=121)	8.1	5.0	7.6	4.8	0.85
Han participants(N=150)	6.3	2.8	5.8	2.3	0.63
Hui participants(N=36)	16.9	4.7	16.6	5.8	0.65
Participants living in urban counties (N=95)	8.0	4.7	4.6	5.8	0.83
Participants living in rural counties (N=93)	8.7	5.9	8.3	6.1	0.88
Participants with 9+ years of education (N=76)	8.0	4.5	7.9	5.1	0.90
Participants with <9 years of education (N=112)	8.6	5.8	7.8	5.5	0.85