Religiosity in black and white older Americans: Measure adaptation, psychometric validation, and racial difference.

Racial difference of religiosity in a heterogeneous older population had long been a focal point of gerontological research. However, most religiosity measures were developed from homogenous sample, few underwent rigorous psychometric validation, and studies on racial difference of religiosity had been obstructed. This cross-sectional study adapted a religiosity measure originally designed for blacks only to a heterogeneous older population of blacks and whites, validated its psychometric properties, and examined racial difference of religiosity. Based on qualitative research of concepts, intensive literature review, and abundant experiences in this field, we adapted the original measure. Then, using the data collected from a survey of 196 black and white Americans 55 years and older in Charlotte, North Carolina, we investigated full-scale psychometric properties of the adapted measure at the item-, domain-, and measure- level. These psychometric validations included item analysis, item-scale correlations, correlation matrix among items, confirmatory factor analysis (CFA) to determine if the original factor structure held after adaptation, and reliability analysis using Cronbach's alpha. Finally, using Multiple Indicators and MultIple Causes (MIMIC) models, we examined racial difference of religiosity through regression with latent variables, while potential measurement bias by race through differential item functioning (DIF) was adjusted in the MIMIC models. In result, we successfully adapted the original 12-item religiosity measure for blacks into an 8-item version for blacks and whites. Although sacrificed few reliability for brevity, the adapted measure demonstrated sound psychometric properties, and retained the original factor structure. We also found racial differences of religiosity in all three domains of the measure, even after adjustment of the detected measurement biases in two domains. In conclusion, the original measure can be adapted to and validated for a heterogeneous older population of blacks and whites. Although the adapted measure can be used to measure the three domains of religiosity in blacks and whites, the observed racial differences of religiosity need to be adjusted for measurement biases before meaningful comparisons.

Introduction

Measurement is of vital importance for science,[ including study of religion. Since most measures were developed from homogeneous samples, their validity in diverse populations is a big concern.[ While measurement in diverse populations is still an emerging science with research on measure adaptation clearly at its cutting-edge, reporting of this kind of work is missing.[ On the other hand, the relationship among different aspects of religion to the health status of older adults has long been a focal point in gerontological research and other field such as oncology.[ And some measurement problems of religiosity have been identified.[

There has been a long history at the measure of religiosity.[ Using a subset (n = 947) of a sample from Kauffman and Harder's 1975 survey of 5 Mennonite and Brethren in Christ denominations, Ainlay and Smith[ established an 11-item, 3-dimensional measure for religiosity. The 3 domains they established were public participation, attitudes toward participation, and private participation. Realizing the lack of external validity of Ainlay and Smith's measure due to the use of an exclusively Mennonite and Brethren sample, Chatters et al[ proposed and tested another measure using a subset (n = 446) of a sample from the National Survey of Black Americans (NSBA). The Chatters et al's[ measure had 3 domains that were similar to Ainlay and Smith's: organizational religiosity (OR), nonorganizational religiosity (NOR), and subjective religiosity (SR), but it had 12 items. Using the new measure, Chatters and coworkers[ found that religiosity plays an important role in the protection and maintenance of health among blacks. Although there had been some other measures for religiosity,[ Chatters et al's[ measure had been very influential in this field. However, a limitation of the Chatters et al's measure is its use in an exclusively black sample. Therefore, its validity in a broader and diverse population is of concern, but this topic has not been studied.

Basically there are 3 options when applying a race-specific measure to diverse populations: use it “as is,” develop a new one, or adapt it. Because the first option has obvious risks and limitations and the second one is usually too challenging, the third option is most realistic.[ Also, as demonstrated by the whole history of the Chatters and coworkers[ measure, adaptations and modifications had been constantly implemented.

Based on a cross-sectional design, this study adapts the Chatters et al's[ black-specific religiosity measure for use in a sample of black and white older Americans, with full-scale psychometric assessments at the item-, domain-, and measure-level. Also, using the adapted measure, this study examines racial differences in religiosity while adjusting for measurement bias. We hypothesis that the original measure can be adapted, that the adapted one will have sound psychometric properties, and that racial difference among blacks and whites will be detected by the adapted measure.

Methods

Measure adaptation

Given this study has a diverse black and white sample while the original measure was developed from an exclusive black sample, adaptation of the measure is obviously necessary. However, the adaptations in this study were rigorous, based on qualitative research of concepts, intensive literature review, and authors’ abundant experiences in this field.[

In the Chatters et al's[ measure (Table 1), the organizational religiosity (OR) domain included 5 items. Three of the 5 items required respondents to list the number of church clubs they belonged to (Y3), the number of other church activities that they engaged in besides the regular church service (Y4), and whether they held offices in the church (Y5). These 3 items were excluded in the modified scale because several gerontological studies have shown that these items in organizational religious involvement in older adults may be more associated with declining functional health than in the strength of organizational affiliation.[ Those authors suggested that nonorganizational and SR may increase to offset the decline in disengagement in formal organizational activities. Religious activities could also take on different forms in different cultures, with blacks and whites having different belief systems and domains of religious expression.[ Therefore, the SR question intended to assess the importance of black parents to send or take their children to religious services (Y12) was excluded.

Table 1

Adaptations of Chatters et al's[ religiosity measure.

There had been other considerations to shorten the Chatters et al's[ measure, which support the elimination of the 4 items (Y3, Y4, Y5, Y12, see Table 1). First, religiosity was only 1 out of many of interest in our original study of older adults, as is often the case in medical research projects. In these contexts, the time that can be allocated to the collection of religiosity data is usually very limited. Second, shorter measures with fewer items reduce the burden on participants,[ which is especially important when the population includes older adults and those with poor health status. Third, we believe that some of the items in the Chatters et al's[ measure are either unnecessarily redundant or inappropriate for nonblack participants, and redundancy is not always preferred in measure development.[

We also revised the options for 7 of the 8 remaining questions, in order to make the response options more concise and easier to read. This kind of practice, that is, adapting items from existing measures and making appropriate changes in order to better fit the survey populations, is highly recommended and widely implemented.[

Field survey and data collection

The protocol of this study was reviewed and approved by the Institutional Review Board (IRB) at the corresponding author's institute before the initiation of field survey. The sample consisted of adults 55 years and older living in Charlotte, NC. It was drawn according to a multistage, area probability procedure designed to insure that every household in Charlotte had the same probability of selection. A primary area was selected for interviewing based on the census distribution of the population. In contrast to the original study by Chatters of only black households, these sites were stratified according to racial composition to include white and black households, and smaller geographical areas “clusters” were randomly chosen. Inhabitable households were listed. Finally, within each inhabitable household, trained interviewers interviewed all household members who were 55 and older. The participants completed an extensive 1-hour questionnaire based on a survey developed at the University of Michigan for the NSBA, which included the just adapted religiosity measure.

Psychometric validation

The adapted measure was administered to all of the participants, and some basic demographic characteristics were also collected and analyzed. According to recent literatures,[ the modifications to this measure were at a “substantial” level and were made to its contents, contexts, and formats. From the findings of a recent systematic review on spirituality measures,[ even changes in the measures’ content after cultural adaptation were poorly reported, not to mention the adequate assessment of psychometric properties of the adapted measures and the well-known culture bias of spirituality measures. And as Chatters et al[ pointed out over a decade ago, rigorous and systematic research on religion measures across diverse populations should be critical for this field. Therefore, a full-scale psychometric assessment of the adapted measure was implemented on the data at the item-, domain-, and measure-level. These included item analysis, item-scale correlations, confirmatory factor analysis (CFA), reliability analysis using Cronbach alpha, and investigation of measurement bias by race through differential item functioning (DIF) analysis using Multiple Indicators and MultIple Causes (MIMIC) models. Moreover, racial differences in religiosity were also examined while adjusting measurement bias through the MIMIC models.

Item analysis

The basic psychometric property for an item is its variability.[ For each of the 8 items, frequency table of responses is reported. Optimally, items should not have any of its response categories chosen by either less than 5% or greater than 95% participants.[

Item-scale correlations

A good item shall substantially correlate with the domain it belongs to, and item-scale correlation is a measure for this property.[ There are 2 types of item-scale correlations: uncorrected and corrected. While the uncorrected item-scale correlation represents how representative the item is to the whole scale, the corrected one represents how closely the item is correlated to other items in the same domain.

Correlation matrix among items

Correlations among items play a key role for a measure.[ A good item should be sufficiently correlated with others, especially with items in the same domain.

Confirmatory factor analysis (CFA)

CFA was used to determine if the same factor structure of the Chatters et al's[ original measure holds after adaptation. The following model fit indexes and the cut-off values for adequate fit were used[: comparative fit index (CFI) ≥0.95, Tucker–Lewis index (TLI) ≥0.95, weighted root mean square residual (WRMR) ≤1.00, root mean square error of approximation (RMSEA) ≤0.08, and normed Chi-square (NC, which equals to the Chi-square divided by the degree of freedom of the model) ≤5.0.

Reliability analysis

For each of the domains and the whole measure, Cronbach alpha was used to assess the internal reliability. A Cronbach alpha value around 0.7, 0.8, or 0.9 is considered as an indicator of “adequate,” “very good,” or “excellent” internal consistency reliability, respectively.[

Racial difference

MIMIC modeling

The MIMIC model can be interpreted as a CFA model with covariates, and it has 2 parts[: a measurement part and a structural part. The measurement part includes the CFA model and possible direct effects of covariates to the indicators (items) after mediated by the latent traits (i.e., DIF effects).[ The structural part includes regressions of the latent traits to the covariates and correlations among the latent traits (Fig. 1). While a significant effect of covariate on the latent trait that stands for the population heterogeneity (e.g., racial differences in religiosity) is usually preferred, a significant direct effect from covariate to the items is not, since it stands for measurement bias, that is, DIF. An item (question) shows DIF when the probability of choosing a specific response differed significantly across the groups of interest conditional on the latent trait that the item is measuring.[ In an iterative fashion,[ DIF analysis was done by checking the modification indices (MI) for each of the 8 possible DIF effects (1 covariate, i.e., race, on 8 items).[ The MI pertaining to 1 possible DIF effect was calculated by comparing the differences in Chi-squares of the model before and after adding the DIF with a Chi-square distribution at 1 degree of freedom.[ An MI greater than 3.84 indicates there is a DIF effect. To start, the DIF effect (i.e., a direct effect from race to 1 of the 8 items) with the biggest MI greater than 3.84 was added into the MIMIC model. Then, the remaining MIs were rechecked. This procedure was done iteratively, and was stopped when no more remaining MI was greater than 3.84 (Appendix). No MI greater than 3.84 suggests that no DIF was observed. If some DIF are identified, then the relative change of regression coefficients from race to the 3 religiosity domains before and after adding the DIF effects to the model will be used to assess the degree to which the population differences in the religiosity domains are contaminated by DIF.[

Figure 1

The MIMIC model of the adapted religiosity instrument. The 3-factor MIMIC model with parameter estimates for the final model. The MIMIC model has 2 parts. The structural part is η = α + Bη + ΓX + ζ, and the measurement part is Y∗ = ν + Λη + KX + ε. Here, η is the latent variables (religiosity domains), B is the regression coefficients among latent variables, Γ is the regression coefficients of the latent variable on covariates, Y∗ is the continuous latent response variables, which represent Y through thresholds, Λ is the factor loadings, K is the direct effect (DIF) from covariate to religiosity items.

SAS (version 9.3) was used for data preparation, descriptive analyses, and reliability analyses. Mplus (version 7.1) was used for CFA and MIMIC modeling. For ease of CFA and MIMIC modeling, all of the observations that had at least 1 of the 8 measure items unanswered or had missing values on race were eliminated. Given the relatively small sample size, in order to avoid susceptible influence of outliers, Mahalanobis distance[ based on the scores of items in the adapted measure was used to detect and exclude multivariate outliers. Basic demographic characteristics of the excluded subjects are reported in comparison with those of included subjects.

Results

In summary, the adaptations to Chatters et al's[ religiosity measure (Table 1) included elimination of 4 (Y3, Y4, Y5, Y12) of the 12 original items, and revision of the options for 7 of the 8 remaining items. These resulted in an 8-item measure of religiosity for its 3 domains. Details are listed in Table 1.

Demographic characteristics of the study sample

Eighty-three geographic areas in Charlotte were selected for interviewing, and a total of 263 older adults were interviewed. Of the 263, 49 participants had at least 1 of the 8 questions in the religiosity measure unanswered, with the majority did not answer any of the 8; another 8 participants had race unrecorded. Although Mplus can handle missing values, for quality control purposes, these 57 participants were excluded in psychometric analyses. In addition, 10 other participants were detected as multivariate outliers as indicated by the Mahalanobis distance[ based on the 8 item scores (D2 > 26.09, df = 8, P < 0.01). Therefore, 196 participants remained. The basic demographic characteristics of the 196 included subjects and 67 excluded subjects are reported in Table 2. Overall, there were no statistically significant differences in gender, education, or income between the included and excluded subjects. Although there were some statistically significant difference in race and marital status, the P values were close to 0.05.

Table 2

Demographic characteristics of the study sample (N = 263).

Psychometric properties

Item analysis, item-scale correlation, and correlation matrix

Some of the psychometric properties of the adapted measure are summarized in Table 3. These include a frequency table of responses for each of the 8 items, uncorrected and corrected item-scale correlations, and the correlation matrix among the 8 items and with the whole measure. Five of the 8 items have each of their response categories fall in the range of 5% to 95%, as does the vast majority (29 out of 35, 83%) of the overall response categories across these 8 items. The uncorrected item-scale correlations range from 0.42 to 0.96, and the corrected item-scale correlations ranged from 0.16 to 0.39. Within each of the 3 domains, items are significantly correlated, and each of the 8 items is significantly correlated with the whole measure (P < 0.05).

Table 3

Distribution of responses, item-scale correlations, correlation matrix, and Cronbach alpha of the adapted measure among the study sample (N = 196).

CFA

CFA based on the original measurement model of Chatters et al's[ were conducted on whites (n = 119) and overall sample (n = 196), and the model fit indexes and factor loadings are summarized in Table 4. For blacks, CFA failed due to nonconvergence in the parameter estimation. For whites, the model fit indices are good, with CFI = 0.946, TLI = 0.911, RMSEA = 0.070, WRMR = 0.566, and NC = 1.578. The factor loadings of all of the 8 items are salient and significant, ranged from 0.424 to 0.908. For the overall sample, the model fit indices are excellent, with CFI = 0.978, TLI = 0.963, RMSEA = 0.038, WRMR = 0.516, and NC = 1.283. The factor loadings of all of the 8 items are salient, ranged from 0.231 to 1.100. All of these model fit indices satisfy their corresponding cut-off values stated earlier. These findings indicate that after adaptation, the measure sustained the original 3-dimensional structure for both of the overall sample and the white subset, but the conclusion to black subset is unknown due to the limitation of sample size.

Table 4

CFA results and Cronbach alphas for the adapted religiosity measure (196 participants, 8 items, 3 domains).

Reliability

The Cronbach alphas of the 3 domains and the overall measure are also reported in Table 4 for white and overall sample. For the whole measure on white and overall sample, and NOR on white, the Cronbach alphas are close to 0.70, which implied close to “adequate” internal consistency of the corresponding domains and measure. However, other Cronbach alphas indicated less than “adequate” internal consistency for other domains.

MIMIC modeling and racial differences

Results for the iterative DIF analysis using MIMIC model in Mplus are reported in the Appendix. DIF effects for race were detected for SR1 (How important was religion in your home when you were growing up?) and NOR2 (How often do you watch or listen to religious programs on TV or radio?). At the same level of religiosity, blacks tend to rate religion more important when growing up than whites do (SR1), and blacks tend to watch or listen to religious programs on TV or radio (NOR2) more often than whites do (Fig. 1). The model fit indices and standardized parameter estimates using the variances of the latent trait (religiosity) for the baseline model (without DIF effects) and the final model (with all DIF effects) are reported in Table 5. Standardized parameter estimates are used since the 2 models have a different residual variance for the latent trait (religiosity). The parameter estimates are not standardized based on variances of both the latent trait and the covariates because the covariate (race) is binary.[ The baseline model does not fit the data well, with CFI = 0.886, TLI = 0.813, RMSEA = 0.079, WRMR = 0.786, and NC = 2.235. This can be interpreted as an indication of possible DIF effects, although the standardized factor loadings are still salient, ranging from 0.277 to 1.113. The addition of 2 DIF effects into the final model improves the model fit substantially, with CFI = 0.976, TLI = 0.956, RMSEA = 0.038, WRMR = 0.549, and NC = 1.289, implying the final model with DIF effects fits the data very well. The factor loadings remain salient with standardized factor loadings ranging from 0.286 to 1.080. The differences in religiosity between the 2 races are represented by the regression coefficients of race on the 3 religiosity domains in the MIMIC model, and the impact of DIF on these differences can be judged by comparing the relative change in these standardized regression coefficients before and after the addition of DIF effects into the MIMIC model. From Table 5, we can find that blacks are more religious than whites at each of the 3 domains (OR, NOR, and SR). (Note: Race is coded as “0—White, 1—Black,” and the items are coded in a reverse direction as listed in Table 1.) Moreover, the 2 DIF effects inflated the racial difference in nonorganizational religiosity (NOR) and in SR, and they almost had no impact on the racial difference in OR (0.312 vs 0.307). Over 33% of the observed racial difference in NOR can be attributed to the effects of DIF [(0.567–0.378)/0.567 = 0.333], and this number in SR is over 40% [(0.788–0.469)/0.788 = 0.405].

Table 5

Baseline and final fitted MIMIC model results (N = 196).

Discussion

The purposes of this study were to adapt a race-specific religiosity measure in elders for use in diverse Black and White population with full-scale psychometric assessment, and to appropriately examine racial differences in religiosity among older Americans while controlling for measurement bias. The one-third decrease in the number of items from 12 to 8 makes the adapted measure more efficient for administration in medical research, where measures can become lengthy due to the addition of other items reflecting other domains of interest. And the extensive full-scale psychometric assessment of the adapted measure demonstrated that it was valid for religiosity in the study sample. Further, the results show that racial differences are evident in all of the 3 religiosity domains although substantial measurement bias (DIF) is contaminating the differences in 2 domains. Therefore, cautions should be warranted when examining racial differences in scores from the adapted measure, because measurement biases (DIF) were identified in 2 of the 8 items (NOR2 and SR1), which substantially contaminated the racial difference in 2 of the 3 domains: NOR and SR.

Consistent with earlier studies, this study found that blacks are more religious than whites. For example, from a nationwide survey of 748 older whites and 752 older African Americans, Krause[ found that blacks were more involved in church participation than whites. And, based on data of 1439 elders from the National Survey of American Life (NSAL), Taylor et al[ also found that older African Americans reported higher levels of religious participation, coping, and spirituality than older whites. Moreover, also from the NSAL, Taylor and Chatters[ found that blacks watched religious television programs and listened to religious radio programs significantly more frequently than whites. At regional level, Arcury et al[ found in their sample of 701 community-dwelling elders with diabetes in 2 rural North Carolina counties, African Americans had higher levels of private religious practice than was reported by Native Americans or Caucasians, and Native Americans reported higher levels of private religious practice than Caucasians. Similarly, Hughes et al examined religious beliefs among African American and Caucasian men who were newly diagnosed with prostate cancer. They found that the older African American study participants reported higher levels of religiosity than the older Caucasian participants reported.[

However, none of these previous studies about the racial difference in religiosity had tried controlling for measurement bias (DIF) while making comparisons, although DIF is very common and a big concern when measures are used in health research.[ The primary reason for the high prevalence of DIF is, while most studies in health sciences are targeting heterogeneous populations, most measures were developed from homogenous samples.[ This is exactly the case of the religiosity measure adapted in this study. And under this situation, appropriate statistical methodology such as the MIMIC modeling plays a key role when comparing the population differences without producing misleading results. On the other hand, a big challenge for studies on racial differences in religiosity is the lack of culturally appropriate measurement models. As Lewis[ noted, the lack of a culturally appropriate measure of religiosity has been a major limitation of studies examining the association between religiosity and health.

The present study filled this gap, by offering a religiosity measure adapted to diverse populations of blacks and whites with full-scale psychometric validation, and offering an appropriate methodology (MIMIC modeling) for adjustment of measurement bias (DIF) when making comparisons among racial groups.

Future studies could validate the newly adapted measure to more diverse populations other than blacks and whites, and evaluate religiosity among them with a number of health conditions that are common to older adults, such as diabetes, cardiovascular disease, and cancer. The results of these studies would broaden the applicability of this newly adapted measure, and highlight intervention targets to improve disease-related outcomes, and therefore would add significantly to the existing research literature on this topic.

Study strengths

The study has several considerable strengths. First, it offered a demonstration of measure adaptation and reporting in diverse populations, which is a premature research field.[ Second, through careful adaptation and rigorous validation of a measure that already showed high utility, this study indicated that the adapted measure can be used for future studies examining racial differences in religiosity for diverse populations of blacks and whites, and possibly for other races if appropriate tests are passed. Given the moderate length of the adapted measure (8 items), it is neither too complicated like other religiosity measures especially in psychology, nor too simple like those used in the National Comorbidity Survey (NCS).[ Long measures are redundant and may upset respondents in household surveys. This modified brief measure of religiosity can be well-suited for use in large surveys where the main focus is not on religion but some other health, mental health, or political outcomes. The third strength of this study is about methodology: through MIMIC models, it appropriately examined racial differences in religiosity when measurement bias (DIF) is present. The MIMIC model has an advantage in measuring group differences in latent trait when DIF is present. Although MIMIC models have been developed a few decades ago,[ they have only recently been applied in medical research.[ This study used a single group MIMIC model to study DIF, and used an iterative procedure. This strategy has the advantage of assessing the effect of DIF on the group differences, and avoids the shortcoming of doing too many statistical tests simultaneously.[ While multiple group confirmatory factor analysis (MG-CFA) is another approach to study measurement invariance among different groups, we do not use it in this study, because MG-CFA requires many statistical tests simultaneously that needs a big sample size that our study cannot offer, and because MG-CFA does not offer the capacity of adjusting for DIF when comparing group differences that MIMIC has.

Study limitations

The inadequate reliability as represented by the Cronbach alphas of some of the domains may be due to the following limitations. First of all, human behavior is never static, which directly impacts the reliability of a measure in social science.[ Second, due to simplification of the measure, the numbers of items in the domains and measure are very few (2 items in OR, 4 in NOR, 2 in SR, and 8 in overall measure). Cronbach alpha is a function of the number of items and the average correlation among the items,[ and at a given level of average inter-item correlation, Cronbach alpha decreases with the decrease in the number of items.[ Third, the limited sample size (n = 196) may also contribute to the low Cronbach alphas. Some authors argue that small sample size may result in unstable estimates for Cronbach alpha,[ and many authors think a sample size of 300 is “small” for Cronbach alpha estimation.[ Fourth, Cronbach alpha is the lower bound of reliability.[ Fifth, given that all of the 8 items are ordinal, the recently proposed ordinal version of Cronbach alpha may be calculated, which usually is larger than the traditional Cronbach alpha.[ However, this new version of Cronbach alpha was not used in this study, mainly due to its unpopularity and calculation difficulties. On the other hand, the low reliability can also be interpreted as a trade-off for the brevity of the adapted measure. Also, sample size is not optimal in study. However, according to literatures on this issue,[ it is reasonable to run CFA for whites (n = 119) and overall (n = 196), and to run MIMIC on the overall sample (n = 196).

It would be ideal to have another independent sample to cross-validate the findings of this study. Even though due to constrain of resources to this study we could not recruit such an independent sample, it is indeed an important direction for future efforts.

Conclusions

In summary, this study reported the adaptation and validation of a black-specific religiosity measure for diverse populations of blacks and whites, and examined racial differences of religiosity in an older population of blacks and whites. Although measure adaptation is appealing in the emerging scientific field of measurement in diverse populations, caution of measurement bias is warranted when applying the adapted measure to diverse populations. This is shown by the fact that in present study the observed racial difference in religiosity has been substantially contaminated by measurement bias (DIF), and therefore adjustment to DIF is a must when comparing racial difference of religiosity using this adapted measure. Moreover, this study illustrated the capability of the single-group MIMIC model to study population heterogeneity in a latent trait such as religiosity and to study the impact of DIF on this population difference.

Acknowledgments

The authors would like to thank 2 anonymous reviewers and Dr Peter J. McLaughlin for their insightful comments on an earlier version of this manuscript, which substantially improved the quality of this study.