Perceived stigma of caregivers: Psychometric evaluation for Devaluation of Consumer Families Scale.

Background/Objective: The Devaluation of Consumer Families Scale (DCFS) is commonly used to measure perceived stigma towards family members of people with mental illness. However, its factorial structure has never been confirmed using confirmatory factor analysis (CFA). This study aimed to test the psychometric properties of the DCFS Taiwan version (DCFS-TW). Method: Family caregivers (N=511) completed the DCFS-TW (97 completed the DCFS again after 2 to 4 weeks) and other instruments. CFA, test-retest reliability, internal consistency, concurrent validity, and known-group validity were analyzed.
Results: The three-factor structure of the DCFS-TW performed better than the one-factor structure. Test-retest reliability (r = .66) and internal consistency were satisfactory (α = .85); concurrent validity (absolute r = .20 to .58) was acceptable; known-group validity was supported by the significantly different DCFS-TW scores in clinical characteristics (had been vs. had not been hospitalized; had been vs. had not been compulsorily admitted). Conclusions: The DCFS-TW has decent psychometric properties and is suitable for health professionals to measure perceived stigma towards family members of people with mental illness.

The process of stigmatization defined by Goffman (1963, p. 6) as “an individual with an attribute which is deeply discredited by his/her society is rejected as a result of the attribute” is always a challenging and difficult issue for stigmatized individuals, especially for people with mental illness (PWMI). For example, PWMI may fear negative attitudes toward their illness from the society or community (Timimi, 2014). How the individual think most people or the society view him/her personally as a member of the stigmatized group is defined as perceived stigma (Brohan, Slade, Clement, & Thornicroft, 2010). Evidence shows that the general public holds negative stereotypes and prejudice against people with mental illness in economically developed countries (Angermeyer & Dietrich, 2006). In other words, PWMI experience negative attitudes and behaviors from the general public, or so-called public stigma (Parcesepe & Cabassa, 2013). Because of the public stigma, PWMI face unequal access to resources (e.g., education, quality healthcare) and discrimination (Centers for Disease Control & Prevention, 2012). Evidence also shows that stigma is related to poor health outcomes, including psychological health, social function and quality of life for PWMI (Chan and Mak, 2014, Lin et al., 2016, Livingston, 2012). Therefore, the issue of stigma should be tackled by both government and healthcare providers.

Although family members of PWMI, especially those in a caregiving role of PWMI, do not receive the same public stigma that PWMI encounter, they suffer from other types of public stigma (Chang, Yen, Jang, Su, & Lin, 2017). Specifically, society may blame the family members for the development of mental illness and may expect them to bear the responsibility of caring for their ill relative. For example, parents may be accused of causing the illness; siblings and spouses are blamed for not caring well for the PWMI; children are viewed as having a higher risk for developing a mental illness themselves (Corrigan & Miller, 2004). After perceiving the public stereotypes, family members of PWMI may consequently develop poor health outcomes as studies have shown that perceived stigma is correlated to the poor social interaction and inappropriate coping strategies. Because family members are those who understand and care for the PWMI most (Pérez-Fuentes, Gázquez Linares, Ruiz Fernández, & Molero Jurado, 2017), they are usually the key people working together with healthcare professionals to treat the PWMI. Thus, monitoring the perceived stigma of the family members is a critical issue (Chang et al., 2017). Specifically, if a family member perceives high levels of public stigma, he or she may have poor health and subsequently cannot provide high quality of care for the care recipients and may be less able to cooperate with healthcare professionals. Hence, using a validated instrument to monitor the perceived stigma of a family carer of PWMI may help health professionals develop appropriate interventions for PWMI and their family members (e.g., stigma reduction programs and coping skills). This may enhance effectiveness of treatments.

Although many studies (e.g., Sher, McGinn, Sirey, & Meyers, 2005) have assessed perceived stigma in family caregivers of PWMI, to the best of our knowledge, no studies have examined the psychometric properties of such instruments using confirmatory factor analysis (CFA) or test-retest reliability. The Devaluation of Consumer Families Scale (DCFS) which is the instrument for measuring perceived stigma towards family members of PWMI examined in this study was first developed by Struening et al. (2001). Struening et al. (2001) used exploratory factor analysis and found three factors in the DCFS: Community rejection (4 items), Causal attribution (2 items), and Uncaring parents (1 item). Although a recent study tested the validity and reliability of the three-factor Italian DCFS, the factorial structure has never been confirmed using CFA (Sideli et al., 2016). Moreover, Struening et al. (2001) and other studies treated DCFS as a one-factor instrument with only one summated total score (Mak and Kwok, 2010, Sher et al., 2005, Yang and Singla, 2011). Therefore, evidence for treating DCFS as a one-factor instrument or a three-factor structure is needed. In addition, based on our literature review, only DCFS was designed to measure perceived stigma towards families of people with mental illness. Although some stigma measures such as affiliate stigma scale (ASS; Mak & Cheung, 2008) and the Consumer Experience of Stigma Questionnaire (CESQ; Yang & Singla, 2011) can also be used or modified to apply to family members of people with mental illness, they do not measure the same stigma concepts as perceived stigma (self-stigma and experienced stigma, respectively).

Based on the current literature, we proposed three competing models to assess the factorial structure of DCFS: (1) Model 1, the one-factor structure that is commonly used (Mak and Kwok, 2010, Sher et al., 2005, Yang and Singla, 2011); (2) Model 2, the three-factor structure found by Struening et al. (2001); (3) Model 3, a one-factor structure with adjustment for three negatively worded items. From a methodological perspective, wording effects could cause the bias in exploratory factor analysis results (Lin, Luh, Cheng, Yang, & Ma, 2014), and it is possible that the DCFS can be treated as a uni-dimensional instrument. Specifically, the three negatively worded items (e.g., “Most people do not blame parents for the mental illness of their children”) separated from the positively worded items (“Most people look down on families that have a member who is mentally ill living with them”) could be due to the wording patterns (negative vs. positive wordings) instead of their underlying trait (i.e., perceived stigma). In addition, all the items embedded in community rejection are positively worded, while all the items embedded in causal attribution or uncaring parents are negatively worded. Although it is recommended to consist of more than three items in each construct (Bollen, 1989), we considered that testing Model 2 is still necessary for us to communicate with previous findings. That is, additional psychometric evidence is required to justify the uni-dimensional use and the wording effects.

The aim of this study was to validate an instrument measuring perceived stigma towards family members of people with mental illness in Taiwan, namely, the Devaluation of Consumer Families Scale Taiwan version (DCFS-TW). Additional psychometric evaluation on the DCFS-TW was also conducted to determine whether the DCFS-TW is suitable for clinical use. We hypothesized that Model 3 (i.e., the uni-dimensional model with wording effects controlled) would outperform the other two models (i.e., the three-factor model and the uni-dimensional model without controlling wording effects). We also hypothesized that the DCFS-TW had satisfactory concurrent validity with other related measures and that the DCFS-TW had the ability to differentiate the caregivers taking care of different types of PWMI regarding the clinical characteristics (viz., hospitalized, compulsory admission, and suicidal attempt).

Method

Participants and Procedure

Before collecting the data, the study was approved by the institutional review boards (IRBs) of Chi Mei Medical Center (IRB serial No.: 10402-002), Jianan Psychiatric Center, MOHW (IRB No.: 15-016), and Chang Gung Memorial Hospital (IRB No.: 104-1716B). The target participants were the family caregivers of the patients in three general hospitals and one psychiatric center, including the outpatients, inpatients, and day care and home care patients. Several psychiatrists identified potential participants (i.e., the caregivers of a patient with schizophrenia, bipolar affective disorder, or major depressive disorder) for this study. We defined the family caregiver as a family member who spent the maximum time and effort in caring for the patient, and we let the patient to identify such family caregiver.

The psychiatrists first ensured that the patients could fully understand the research purposes and provide valid consent. Afterward, three research assistants approached the patients to explain the study in detail, and asked for the written consent. The patients provided the consents to allow our study to collect their data (including those from their medical records) and to contact their family caregivers. In addition to demographics collected from medical records and clinical characteristics (hospitalization, suicidal behaviors, and compulsory admission) from their caregivers, we collected no other patient data.

After the patient completed the informed consent, the same research assistants contacted and provided detailed study information to the family caregiver who was identified by the patient (N=524). Thirteen family caregivers declined to participate, and we finally recruited 511 family members who had signed a written informed consent. The inclusion criteria of the caregivers were: (a) aged over 20 years; (b) had clear consciousness with the ability to understand Mandarin or Taiwanese. The exclusion criterion was obvious cognitive impairment identified by the research assistants (e.g., repetitious verbal expressions). All the research assistants were trained by a psychiatrist, who has extensive experience in stigma research, to ensure the quality of the standardized recruiting procedure, including how to determine obvious cognitive impairment. Table 1.

Table 1

Characteristics of caregivers and persons with severe mental illness (N=511).

	n (%)	M (SD)
Caregiver
Age in year		53.82 (13.66)
Years of education		10.59 (3.70)
Years of caregiving		10.64 (8.49)
Gender (Male)	242 (47.4)
Marital status (Married)	362 (70.8)
Employment status
Full-time employment	290 (56.7)
Part-time employment	49 (9.6)
No employment or retired	172 (33.7)
Religion (Yes)	417 (81.6)
Relationship with patient
Parents	199 (38.9)
Spouse	175 (34.2)
Siblings	70 (13.7)
Children	52 (10.2)
Others	15 (2.9)
Living with the ill relatives (Yes)	453 (88.6)
Primary caregiver (Yes)	501 (98.0)
Persons with severe mental illness
Age in year		46.11 (12.50)
Years of education		10.59 (3.70)
Age first contact to mental health services		34.52 (13.36)
Gender (Male)	242 (47.4)
Marital status (Married)	219 (42.9)
Employment status
Full-time or part-time employment	129 (25.2)
Supported or sheltered employment	5 (1.0)
No employment or retired	377 (73.8)
Diagnosis
Schizophrenia	240 (46.9)
Bipolar disorder	102 (20.0)
Major depressive disorder	169 (33.1)
Hospitalized (Yes)	312 (61.1)
Suicidal behaviors (Yes)	209 (40.9)
Compulsory admission (Yes)	147 (28.8)

Note: M indicates mean; SD indicates standard deviation.

All PWMI (including people with a clinical diagnosis of schizophrenia, bipolar affective disorder, or major depressive disorder) and their family members participated voluntarily in this study. In addition to the informed consent, all participants (i.e., family caregivers) completed several structured questionnaires (including on demographics, clinical characteristics, and instruments described in the Instrumentation section below) with the assistance of trained research assistants in the outpatient visit. The research assistants read out the questions for the participants who had low levels of literacy, and the time on filling out the questionnaire was about 30 minutes on average. Two to four weeks later, 97 participants who had come to the second outpatient visit completed the DCFS-TW again.

Instruments

The Devaluation of Consumer Families Scale Taiwan version (DCFS-TW) we investigated was based on the version designed by Mak and Kwok (2010), who added two items (“Most people would not be friends with families who have relatives with mental illness” and “Most people stigmatize families of relatives with mental illness”) on the original 7-item DCFS. Therefore, the DCFS-TW consists of nine items. The two items were added because Mak and Kwok intend to tap onto the general feelings of being discriminated against by the families (Personal communication with Mak).

Both original DCFS and the revised DCFS shared acceptable internal consistency (α = .71 to .86) (Mak and Kwok, 2010, Struening et al., 2001). Nine DCFS-TW items are rated on a four-point Likert scale (1: strongly disagree; 4: strongly agree), and three (Items DCFS2, DCFS5, and DCFS6) are negatively worded items. The three negatively worded items are reverse coded to make the higher DCFS-TW score represents higher levels of the perceived stigma.

Because Taiwan and Hong Kong use the same written language (i.e., the traditional Chinese characters), we directly adopted the DCFS revised by Mak and Kwok (2010), and did not translate it. However, in order to ensure the readability of the revised version in the Taiwan population, we conducted a focus group. The focus group consisted of five female and three male caregivers of people with mental illness (age ranged 27 to 63), and all of them ensure the readability of the DCFS-TW.

The Affiliate Stigma Scale (ASS) contains 22 items all rated on a four-point Likert scale, and a higher score represents a higher level of self-stigma. The psychometric properties of the ASS are satisfactory in both Hong Kong version (α = .94) (Mak & Cheung, 2008), and Taiwan version (α = .82 to .93) (Chang et al., 2016a, Chang et al., 2015). The internal consistency of the ASS in the current study was satisfactory (α = .83 to .95).

The Multidimensional Scale of Perceived Social Support (MSPSS) contains 12 items all rated on a seven-point Likert scale, and a higher score indicates that higher levels of perceived support. The MSPSS showed high internal consistency (α = .85 to .95) across different ethnicity samples (Canty-Mitchell & Zimet, 2000) including Chinese samples (α =.80 to .89; Chou, 2000, Zhang and Norvilitis, 2002). The internal consistency of the MSPSS in the current study was satisfactory (α = .84 to .91).

The Center for Epidemiologic Studies Depression Scale (CES-D) contains 20 items that rate the score between 0 and 3 in a seven-day basis. The higher score of the CES-D indicates more depressed. Moreover, the screen ability of the CES-D Taiwan version is satisfactory (area under receiver operative characteristic curve = .88-.90; Yang, Soong, Kuo, Chang, & Chen, 2004). The internal consistency of the CES-D in the current study was satisfactory (α = .92).

The Rosenberg Self-Esteem Scale (RSES) contains 10 items all rated on a four-point Likert scale. The higher scores of the RSES represent higher levels of self-esteem. The RSES Taiwan version showed acceptable internal consistency (α = .74 to .82; Lin & Tsai, 2016). The internal consistency of the RSES in the current study was satisfactory (α = .88).

Data analysis

Item properties of the DCFS-TW were analyzed using ceiling/floor effects (i.e., how many people responded in the extreme scores of 4 and 1), corrected item-total correlations, and the test-retest reliability for each item score. We proposed the acceptable item properties as follows: ceiling/floor effects < 20%; corrected item-total correlations > .3; test-retest reliability illustrated by Pearson correlation > .3. In addition, we assessed the ceiling/floor effects, Cronbach's α, and test-retest reliability for the entire DCFS-TW score to examine the reliability of the instrument in scale-level.

We examined the construct validity of the DCFS-TW score using the three competing CFA models. Model 1 is the one-factor model without controlling wording effects. Model 2 is the three-factor model found by Struening et al. (2001). Model 3 is based on Model 1 but also controls wording effects; that is, we correlated the item uniqueness of the three negatively worded items. Diagonally weighted least squares estimator was used in all CFAs to account for the categorical data (i.e., four-point Likert scale). In addition, we adopted the following fit indices to help us determine whether the structure of the DCFS-TW score is supported: comparative fit index (CFI) and Tucker-Lewis index (TLI) > .95; root mean square error of approximation (RMSEA) < .06; and standardized root mean square residual (SRMR) < .08 (Hu and Bentler, 1999, Lin et al., 2018). Moreover, we used χ2 difference test to determine the factorial structure of the DCFS-TW. If Model 3 does not significantly outperform Models 1 and 2, we would not use Model 3 for the following analyses because a factor (or subscale) is recommended to consist of more than three items (Bollen, 1989).

After confirming the structure of DCFS-TW, concurrent validity was determined using Pearson correlation and the comparisons between the correlation coefficients. We first computed the correlations between the DCFS-TW score with other instruments’ scores, including the ASS, MSPSS, CES-D, and RSES. Then, the correlation coefficients derived from the DCFS-TW and the ASS scores were compared with all the other coefficients using a Z test (Steiger, 1980). Because all the instruments measured the psychological aspects of the participants, we anticipated their scores being correlated. However, the correlation between the DCFS-TW and the ASS scores would be the highest because they share more similar psychological concept (i.e., the stigma) than do other instruments.

In order to understand the extent of the DCFS-TW score linking to the clinical characteristics of PWMI, an important concern for mental health professionals, we investigated the known-group validity. Three clinical characteristics (had been hospitalized, had suicidal behaviors, and had been compulsorily admitted) were adopted in the known-group validity because they are related to the level of self-stigma (Chang et al., 2016b, Rüsch et al., 2014). Before conducting the known-group validity, we tested the measurement invariance of the confirmed DCFS-TW structure using χ2 difference test across each clinical characteristic. Specifically, we tested the differences between three nested models (configural model; model with factor loadings constrained equal across group; model with factor loadings and item intercepts constrained equal across group). Non-significant differences indicate measurement invariance. Several independent t tests with effect size were used to care for the known-group validity; an effect size = 0.2, 0.5, and 0.8 means small, medium, and large, respectively.

The CFAs were analyzed using lavaan package in the R software (Rosseel, 2012), the comparisons of the correlations coefficients analyzed using online tool (Lee & Preacher, 2013), and other data were analyzed using IBM SPSS 23.0 (IBM Corp., Armonk, NY.).

Results

The characteristics of the participants (i.e., the family caregivers) and those of the PWMI are demonstrated on Table 1. The item properties of the DCFS-TW scores were acceptable in the ceiling/floor effects, corrected item-total correlation, and test-retest reliability. However, two items had relatively low test-retest reliability (r = .27 for Item DCFS2 and .23 for DCFS8). The negatively worded items (i.e., items DCFS2, DCFS5, and DCFS6) had the lowest corrected item-total correlations. The entire DCFS-TW also demonstrated satisfactory psychometric properties (Table 2).

Table 2

Item properties and reliability of the Devaluation of Consumer Families Scale Taiwan version (DCFS-TW).

Item#	Item description	Mean (SD)	Ceiling/floor effects	Item-total correlation	Test-retest reliabilitya
DCFS-TW	Most people…	2.23 (0.48)	1.0/1.0	.85b	.68
DCFS1	in my community would rather not be friends with families that have a relative who is mentally ill living with them	2.16 (0.57)	7.2/2.0	.60	.33
DCFS2c	believe that parents of children with a mental illness are just as responsible and caring as other parents	2.04 (0.47)	7.6/1.6	.32	.27
DCFS3	look down on families that have a member who is mentally ill living with them	2.28 (0.63)	6.1/3.7	.68	.49
DCFS4	believe their friends would not visit them as often if a member of their family were hospitalized for a serious mental illness	2.30 (0.63)	5.1/4.1	.67	.59
DCFS5c	would treat families with a member who is mentally ill in the same way they treat other families	2.41 (0.59)	3.1/2.2	.46	.38
DCFS6c	do not blame parents for the mental illness of their children	2.30 (0.59)	4.3/2.5	.32	.43
DCFS7	would rather not visit families that have a member who is mentally ill	2.27 (0.59)	4.1/3.1	.73	.51
DCFS8	would not be friends with families who have relatives with mental illness	2.16 (0.55)	6.1/2.3	.70	.23
DCFS9	stigmatize families of relatives with mental illness	2.24 (0.62)	6.3/3.7	.69	.51

Test-retest reliability examined using Pearson correlations (n=97), and all p-values < .01.

Reported as Cronbach's α.

Reverse coded items.

The CFA results showed that all the models had all fit indices satisfactory, and all the factor loadings were significant (p < .001). The χ2 difference tests showed that Model 1 was inferior than were Models 2 (Δχ2 = 29.88, Δdf = 2; p < .001) and 3 (Δχ2 = 32.38, Δdf = 3; p < .001). As comparing Models 2 to 3, the χ2 difference test was not significant between the two models (Δχ2 = 2.50, Δdf = 1; p = .11). The correlations among the uniqueness of the three negatively worded items were .13, .21, and .31 (Table 3).

Table 3

Three confirmatory factor analysis models for the Devaluation of Consumer Families Scale Taiwan version (DCFS-TW).

	Model 1	Model 2	Model 3
DCFS-TW Item
DCFS1	.657	.665a	.665
DCFS2	-.326	1.000c	-.292
DCFS3	.742	.748a	.748
DCFS4	.754	.763a	.764
DCFS5	-.467	.763b	-.425
DCFS6	-.314	.502b	-.258
DCFS7	.807	.815a	.815
DCFS8	.770	.777a	.776
DCFS9	763	.769a	.769
Fit indices
χ2 (df)	43.47 (27)	13.59 (25)	11.09 (24)
Comparative fit index	.988	1.000	1.000
Tucker-Lewis index	.984	1.012	1.014
Root mean square error of approximation	.035	.000	.000
Standardized root mean square residual	.060	.036	.033

Note. Model 1 is the one-factor DCFS model without controlling wording effects. Model 2 is the three-factor DCFS model found by Struening et al. (2001). aItems embedded in community rejection; bItems embedded in causal attribution; cItems embedded in uncaring parents. Model 3 is the one-factor DCFS model with three negatively worded items (Items DCFS2, DCFS5, and DCFS6) additionally correlated their uniqueness. The correlations among the three negatively worded items were .13 (DCFS2 and DCFS5), .20 (DCFS2 and DCFS6), and .31 (DCFS5 and DCFS6).

Concurrent validity of the DCFS-TW score was illustrated by the correlations in the anticipated directions between DCFS-TW total score and the scores of other instruments: r=.58 with ASS score; = -.34 with MSPSS score; = .36 with CES-D score; and r = .20 with RSES score. In addition, the correlation between the DCFS-TW score and the ASS score was the highest as compared with other correlations (Z=6.01 to 9.45; p < .001).

The nested models for measurement invariance in Model 3 were not significantly different from each other (Table 4). Hence, we examined the known-group validity as we described in data analysis section. The known-group validity of the DCFS-TW score was supported by two clinical characteristics: hospitalization experience and compulsory admission experience. The caregivers reported higher total scores in the DCFS-TW if the PWMI they cared had been hospitalized (2.30±0.41 vs. 2.14±0.36; effect size = 0.41; p < .001) or had been compulsorily admitted (2.39±0.44 vs. 2.18±0.36; effect size = 0.51; p < .001). However, the caregivers of the PWMI who had suicidal attempt had similar DCFS-TW score to these of the PWMI who had not suicidal attempt (2.26±0.41 vs. 2.22±0.38; effect size = 0.10; p = .23).

Table 4

Measurement invariance testing for Devaluation of Consumer Families Scale Taiwan version (DCFS-TW).

	Configural model	Loadings constrained	Loadings and intercepts constrained
Hospitalization
χ2 (df)/ p-value	19.02 (50)/ 1.00	24.84 (56)/ 1.00	27.46 (62)/ 1.00
Δχ2 (Δdf)/ p-valuea	--	5.83 (6)/ 0.44	8.44 (12)/ 0.75
Compulsory admission
χ2 (df)/ p-value	18.42 (50)/ 1.00	29.46 (56)/ 1.00	31.62 (62)/ 1.00
Δχ2 (Δdf)/ p-valuea	--	11.04 (6)/ 0.08	13.20 (12)/ 0.36
Suicidal attempt
χ2 (df)/ p-value	18.56 (50)/ 1.00	20.02 (56)/ 1.00	24.76 (62)/ 1.00
Δχ2 (Δdf)/ p-valuea	--	1.46 (6)/ 0.96	6.20 (12)/ 0.91

Comparing with configural model.

Discussion

This is, to the best of our knowledge, the first study applying series of psychological tests to examine the psychometric properties of the DCFS-TW score in a relatively large sample of Taiwanese caregivers (N=511). Our results showed that the DCFS-TW structure fit the one-factor solution when considering the wording effects. Moreover, the DCFS-TW score is related to the instrument score on stigma more than the instrument scores on psychological health; the DCFS-TW score has adequate reproducibility. In addition, the DCFS-TW score is able to differentiate family caregivers of PWMI who had been hospitalized or who had compulsory admission from those who had not.

Although no studies have used CFA to test the DCFS score, some clinical research did report the internal consistency of the DCFS score. Therefore, we could compare our internal consistency results with others. The satisfactory internal consistency in our study (α = .85) echoes the internal consistency reported previously: α = .86 in parents of children with autism spectrum disorder (Mak & Kwok, 2010); α = .71 and .77 in two samples of caregivers of PWMI (Struening et al., 2001); α = .73 in primary caregivers of people with major depressive disorder (Sher et al., 2005); α = .81 in caregivers of people with psychosis (Sideli et al., 2016). Based on the promising internal consistency, we tentatively concluded that the DCFS items, with or without the two additional items generated by Mak and Kwok (2010), are highly correlated.

In addition to the internal consistency, our findings of CFA support both three-factor and on-factor structures for the DCFS-TW. Unfortunately, our factorial structure results are hard to compare with previous studies because of the dearth in such information. Nevertheless, we believed that our findings echo what Struening et al. (2001) have found in their exploratory factor analysis: a three-factor solution. However, it is commonly viewed as suboptimal for a factor (or subscale) to consist of fewer than three items (Bollen, 1989). Therefore, we believe that our one-factor model, which also shows satisfactory fit indices in the CFA, is adequate and support the commonly practical use of DCFS: Struening et al. (2001) suggested using the DCFS as a uni-dimensional instrument, and the uni-dimensional structure has been applied in all other studies (Mak and Kwok, 2010, Sher et al., 2005, Yang and Singla, 2011).

As we justified earlier that the three factors generated from the exploratory factor analysis in the study of Struening et al. (2001) is due to the wording effects, our CFA results somewhat corroborate our postulation. Specifically, the one-factor CFA model significantly improved in the fit indices after we controlled for the negatively worded items. However, the factor loadings of the three negatively worded items were relatively low, and the correlations between the uniqueness of the three items were weak. Also, improvement was found in the three-factor structure. Therefore, we were unsure whether the three negatively worded items are embedded together with all the positively worded items or the three items belong to other factor(s). Because models with very few items tend to yield better fit indices than longer scales and the rules of thumb generally recommend a minimum of 3 (or even 5) items per factor for CFA (Kenny & McCoach, 2003), we tentatively concluded that the one-factor solution accounting for wording effects is suitable for the DCFS-TW.

Known-group validity supported the DCFS-TW score in two clinical characteristics: hospitalization and compulsory admission. However, the DCFS-TW score was unable to differentiate the family caregivers of PWMI who had suicidal attempt and those of PWMI who had no suicidal attempt. The significant differences of the DCFS-TW score between yes vs. no in the clinical characteristics of hospitalization and compulsory admission can be explained by the elevated public stigma on the two characteristics (Link, Cullen, Frank, & Wozniak, 1987): Most of community residents think those who have been previously hospitalized and involuntarily admitted are dangerous and tend to avoid them. Therefore, family members of PWMI with one of the two clinical characteristics might consequently perceive higher levels of stigma than their counterparts did.

In terms of the non-significant results in relation to suicidal behaviors, we postulated that suicidal behaviors only link to self-stigma and not to perceived stigma (Chang, Wu et al., 2016; Reynders, Kerkhof, Molenberghs, & Van Audenhove, 2014). PWMI with higher levels of self-stigma may consider help from healthcare providers as unimportant (Pattyn, Verhaeghe, Sercu, & Bracke, 2014), which decreases their attempts to seek help when encountering psychological problems, and might subsequently lead to suicidal behaviors. However, such a link between suicidal behaviors and stigma seems only appear in PWMI who has self-stigma, and may not be generalized to the perceived stigma of family caregivers. Nevertheless, future studies are warranted to corroborate our findings.

Because our results demonstrated sound psychometric properties (e.g., the satisfactory test-retest reliability, acceptable internal consistency, and validity) for the DCFS-TW score, we are confident to recommend using the DCFS-TW to assess perceived stigma towards family caregivers of PWMI. Another benefit of the DCFS-TW is its briefness with short administrative time (with a range of 5 to 10 minutes), which means that it is efficient to use even in a busy clinical setting. Therefore, we highly recommend that healthcare providers monitor perceived stigma among families of PWMI regularly and design interventions to prevent family caregivers from negative consequence caused by perceived stigma, which may also have a positive influence on the health of PWMI.

Our study findings provided several implications. First, currently only ASS and CESQ can be used to assess self-stigma and experienced stigma, respectively, of a family member who takes care of PWMI. Our DCFS-TW findings filled in the lack of a validated instrument on measuring perceived stigma of a family member. Second, the two added items in the DCFS-TW cover more comprehensively than the original DCFS in the perceived stigma concept. Moreover, the international comparisons using DCFS-TW and DCFS are applicable because we can simply remove the two added item scores from the DCFS-TW for the comparison. Third, given that both Hong Kong and Taiwan use the traditional Chinese characters in the DCFS-TW with the same nine items, we can directly compare the perceived stigma in family caregivers between Hong Kong and Taiwan. Fourth, the DCFS-TW can be used as an important outcome to assess the effectiveness of a stigma-reduction program in the policy level. If the policy is effective, we may detect the reduction in the DCFS-TW score. Moreover, we may approach those family members who have high DCFS-TW score to prevent them from negative consequences due to high levels of perceived stigma.

There are some limitations to this study. First, because we conducted a convenience sampling in the Southern Taiwan in a very specific sample, our results were unable to generalize to all of Taiwan. Moreover, as we did not randomly select our participants, the sample could be biased because the access possibilities of our study were accidental and regional directed. Second, most of the participants were primary family caregivers. Given that primary caregivers are very likely to have different levels of perceived stigma than other caregivers; cautions should be made when applying our results to non-primary caregivers. Third, the DCFS-TW version is slightly different from the original DCFS designed by Struening et al. (2001). Therefore, researchers who want to use the original DCFS should acknowledge that the psychometric properties of the DCFS-TW score might be slightly different from the original DCFS score. Future studies are thus warranted to test the psychometric properties of the different versions of DCFS. For example, we need evidence to determine whether the DCFS-TW really works better than the original DCFS in Taiwanese populations. Suppose that DCFS-TW is more suitable than DCFS to be used in Taiwanese population, a further question is whether the DCFS-TW score in Taiwan is comparable to the original DCFS score in the other countries.

In conclusion, the DCFS-TW is a brief instrument for healthcare providers to efficiently measure the perceived stigma of family caregivers of PWMI. We suggest simply summing up the item scores to represent perceived stigma, which is easy and quick for healthcare providers to do in busy clinical setting. Moreover, the DCFS-TW is a useful instrument which provides valid findings and relevant information to health care providers.