Validity and Reliability Evaluation of the Persian Version of the Heart Failure-Specific Health Literacy Scale.

BACKGROUND: Health literacy (HL) has important implications for health outcomes in heart failure (HF) patients. Studying health literacy requires culturally appropriate and valid instruments. The aim of the study was validation of the Persian version of the heart failure-specific health literacy scale (HF-Specific HL Scale).
METHODS: One hundred patients with heart failure were selected in Qom, Iran in 2017. The 'forward-backward' procedure was applied to translate the questionnaire from English into Persian. Content validity, face validity, construct validity have been employed to validate the prepared scale. Cronbach's alpha coefficients and the test-retest were used to assess the scale reliability. Data were analyzed using SPSS, version 16, and Smart PLS 3.0 software.
RESULTS: Confirmatory factor analysis completely supported the three-factor model of the HL scales. Convergent validity was satisfied in that all factor loadings and the average variance extracted exceeded 0.5. The divergent validity was verified using Fornel and Larcker method. R-square and path coefficient were higher than 0.43 and 0.65 respectively, indicating the good structural model. Composite reliability and Cronbach's α coefficient for all of the constructs were over the recommended threshold of 0.70, ensuring adequate internal consistency of the scale. The test-retest reliability ranged from 0.78-0.90, which indicated a good level of stability.
CONCLUSION: The findings indicated that the Persian HF-specific HL scale is reliable and valid for measuring health literacy among heart failure patients. Further research is required to measure the sensitivity and specificity of the scale.

INTRODUCTION

Heart failure (HF) is a common and chronic disease with poor prognosis and a severe decrease in the quality of life.[1] .6 million Americans in the United States and 37 million people all over the world have HF.[2] In Iran, because it is a developing country, HF is rising as a result of life changes, and now it is estimated that it has affected 3.3 percent of the population.[3] This disease severely affects the psychologic and function, and daily and social activities of the patients; it imposes a lot of expenses on the patient and health systems. In 2012, the estimated cost to treat heart patients was 20.9 billion dollars, and by 2030 the cost of treatment for HF is expected to reach 69.8 billion dollars.[4] HF outcomes can be improved with self care and the development of disease management programs.[5] Self-care is a comprehensive concept, as well as a decision-making process, that selects appropriate activities to prevent the exacerbation of symptoms.[6] The care complexity of HF patients can lead to threat to adherence to self-care behaviors, especially among people with low health literacy (HL).[7] HL represents a range of skills and resources related to the ability of individuals to process health related information.[8] In the view of the world health organization (WHO), HL has been introduced as a social cognitive skill that determines the motivation and ability of individuals to achieve, understand and use information in a manner that will lead to the maintenance and promotion of their health. WHO presented HL as one of the greatest determinants of health.[9] Only 12 percent of adults in the United States have of HL skills needed to manage their health care. 27 to 54 percent of patients with HF have low HL.[10] Inadequate HL is associated with poorer postoperative recovery,[11] drug inactivity, reduction in the use of preventive services,[12] reduction in self-care behaviors,[13] and worse quality of life.[14] Therefore, HL is an important consideration in promoting HF knowledge and self-care behaviors. If health providers understand the patients’ HL level, they may educate them to use methods that promote understanding of the concepts.[15]

Assessment of patient HL is critical to tailor educational programs appropriately and provide special assistance to patients with limited ability to manage complex medical conditions such as HF.[16] According to the mentioned literature, different tools have already been developed and used to measure HL worldwide. The test of functional HL in adults (TOFHLA)[17] and rapid estimate of adult literacy (REALM)[18] in medicine are the screening tools designed for use in health centers. Health workers use these tools to identify the patients who have problem in detecting health related terms (REALM) or understanding the text (TOFHLA). These tools are public and have a limited view; they only measure reading and calculation skills and do not consider a wide range of HL structures such as the critical thinking of interaction and communication.[19] The specific scales focusing on certain diseases could correctly reflect the condition and requirement of patients.[20]

HF-specific HL scale is a specific questionnaire, which was developed by Matsuoka et al. (2016) to assess the HL of patients with HF. The questionnaire consists of 12 items that evaluate three dimensions of HL including functional, communicative and critical literacy.[21] Functional literacy is sufficient basic skills in reading and writing for one to be able to function effectively in everyday situations, a definition which is broadly compatible with the narrow definition of HL; communicative literacy is more advanced skills to participate actively in activities, to extract information and derive meaning from different forms of communication. Also, critical literacy is advanced skills to analyze information critically and to use this information to exert greater control over life events.[22] The scale is short and comprehensive, and easy to understand. Based on the results of the original[21] and Chinese version,[23] this scale is a validated and reliable tool for assessing HL in patients with HF. The original scale was developed based on interviews with health professionals engaged in HF treatment and a focus group with nursing researchers. Results of construct validity identified three interpretable factors by exploratory factor analysis. Some socio-demographic factors such as low education level and living alone were associated with low HL, which confirms the criterion validity of the scale. Also, there was a correlation between the total HL score and other related measures, which supports the concurrent validity of the scale. The Cronbach’s alpha of the total scale was 0.71, and test-rest reliability coefficient was 0.88 -0.90 for the subscales.

It is argued that a disease-specific or context-specific HL tool may be more useful and relevant when it is applied to patients in need of managing a particular chronic condition.[24] Since there is no special HL questionnaire for patients with HF in Iran, and HL is a topic of context, this study was conducted in order to translate and determine the validity and reliability of the Persian version of HF-specific HL scale.

MATERIALS AND METHODS

This is a methodological study. The target population of the present study consisted of patients with HF hospitalized in two selected educational centers of Qom, Iran in 2017. The sampling method was purposive and inclusion criteria were age older than 18 years, ability to speak Persian, at least six month passed from a definitive diagnosis of the disease, Ejection Fraction (EF) less than 40 percent (referring to patient records based on echocardiography), no history of previous psychiatric and cognitive diseases, and willingness to participate in the research. Exclusion criteria were as follows: difficulty participating in the study owing to physical or mental problems and failure to complete the questionnaire. HF-specific HL scale and demographic and medical information questionnaire were used to collect the data.

HF-specific HL scale includes 12 items, which measures three dimensions of functional (1-4 items), communicative (5-8 items) and critical (9-12 items) HL in patients with HF. Questions were scored using 4-point Likert scale: 1=not relevant; 2=somewhat relevant; 3=quite relevant; 4=very relevant. The score of each item ranges from 1 to 4. The total score of the scale is 36. A higher score represents a higher level of HL.[21]

Socio-demographic and clinical characteristics include age, gender, material status, housing, education level, the economic situation, employment status, number of admissions, duration of diagnosis, and functional class, NewYork Heart Association functional classification (NYHA) which was completed on the basis of patient statements and records. The NYHA is often used to describe the functional capacity of adults with HF. It places patients in one of four categories based on how much they are limited during physical activity: I: No limitations of physical activity, II: Slight limitation of physical activity, but no symptoms at rest, III: Marked limitation of physical activity, but no symptoms at rest and IV: Inability to perform any physical activity without discomfort; symptoms may be present at rest.

Translation

We translated the scale according to International Quality of Life Assessment Project Protocol (LQOLA).[25] For this purpose, at first two independent professional translators translated two separate English Translations of the Persian version of the questionnaire. The Persian version of the two above-mentioned translations was obtained with the best translation available. Subsequently in the next step, two English language experts translated the final version into English again. After this step, the original English version was compared with the English version derived from the translation of language specialists by the research team, and at last, the final version of the Persian version was approved.

Content Validity

Content validity index (CVI) and content validity ratio (CVR) were measured to evaluate the content validity of the HF-specific HL Scale. CVR examines the essentiality of each item for the Iranian culture by using 3-points rating scale (essential, useful but not essential, and not essential). The CVR for every item was calculated using formula CVR=[Ne-(N/2)]÷(N/2) (Ne is the number of panelists indicating “essential” for each particular item and N is the total number of panelists). To obtain CVI for relevancy, we used simplicity and clarity of each item, ordinal scale with four possible responses. The responses include a rating from 1=not relevant, not simple and not clear to 4=very relevant, very simple and very clear. The number of those judging the item as relevant or clear (rating 3 or 4) was divided by the number of content experts. The acceptance of items based on CVI score was above 0.79 and CVR was higher than 0.8.[26]

Face Validity

Face validity was determined to assess the understanding of patients from the questions. For this purpose, the researcher asked 10 HF patients with the criteria for inclusion to fill out the questionnaire.

Construct Validity

Construct validity means that a test designed to measure a particular construct is actually measuring that construct. To assess the construct validity of the scale’s measurement model, we used convergent validity and divergent validity. It was performed using Confirmatory Factor Analysis (CFA).

Reliability

Internal consistency for the HF-specific HL Scale was determined using the Cronbach’s alpha and composite reliability. The Cronbach’s alpha coefficient and composite reliability of 0.7 or above was considered satisfactory. Test-retest reliability was assessed by computing the intra-class correlation coefficient (ICC) of each domain. The time interval for this assessment was two weeks. An ICC>0.80 indicated good test-retest reliability and stability.[27]

Ethical Considerations

Upon ethical approval from the Ethics Committee of the Qom University of Medical Sciences (IR.MUQ.REC.1395.155) and also coordination with educational centers, 100 patients entered the study according to the inclusion criteria. Since no intervention was performed in this study, implied informed consent was taken from the participants. The purpose of the plan and how it was done for the participants was explained. After ensuring the patients about the confidentiality of their information, the questionnaire were completed within six months.

Data Analysis

Participants’ characteristics were analyzed using descriptive analysis. In the present study, CFA was used to evaluate the HF-specific HL scale construct using Partial Least Square (PLS) version 3 software. Convergent validity was established by examining the outer loadings of the measurement indicators (higher than 0.70) and average variance extracted (AVE) (higher than 0.5) of the model’s constructs. Divergent validity is demonstrated using the Fornell-Lacker criterion; it was confirmed when the AVE of a composite construct was higher than the construct’s highest squared correlation with any other composite construct.[28] For the structural model, coefficients of p-value and R square criterion were used. R square is an index to measure each endogenous latent variable’s R-Square. The greater amount of the R square value for endogenous structural model is indicative of a better fitting model. Henseler (2009) proposed a rule of thumb for acceptable R square with 0.75, 0.50, and 0.25 described as substantial, moderate and weak, respectively. The direction and significance of path coefficient will determine whether the structural model is or is not fit. Therefore, weak, medium and strong fitness of the structural model is determined by R square standard and path coefficient.[29] A goodness of fit (GoF) index was calculated to display the model fit to the data.[30]

RESULTS

The mean and standard deviation of the patient’s age was 55.13±13.76 years and 59% of them were men. 80% of the participants were residents of the city and 68% of the patients were married. 60% of them were illiterate and low literate. The mean and standard deviation of the duration of the disease was 7.38±6.30. Further information about the personal characteristics of the participants in this study ispresented in Table 1.

Table 1

Clinical and socio-demographic information of the sampled respondents in Qom in 2017 (N=100)

Variables	Mean±SD
Age (years)	55.13±13.76
Duration of disease (years)	7.38 ±6.30
	N (%)
Gender
Male	59 (59)
Female	41 (41)
Marital status
Single	10 (10)
Married	68 (68)
Divorced/ widowed	22 (22)
Educational status
Illiterate	30 (30)
Primary school	30 (30)
High school	31 (31)
Secondary school	9 (9)
Employment status
Employed	43 (43)
Unemployed	24 (24)
Housewife	33 (30)
Economic status
Poor	31 (31)
Intermediate	39 (39)
Good	30 (30)
Smoking status
Smoker	26 (26)
Non-smoker	74 (74)
NYHAa
I	2 (2)
II	35 (35)
III	34 (34)
IV	29 (29)

NewYork Heart Association functional classification

Content and Face Validity

To assess the content validity, we evaluated the items’ CVI and CVR. In the expert panel review, all tasks received a CVI above 0.80 and CVR above 0.78, representing good content validity. Regarding face validity, all patients could understand every item; their judgment towards the meaning of the scale were consistent with the established purpose.

Convergent and divergent validity, a measurement model was tested using CFA. Convergent validity was satisfactory in that all confirmatory factor loadings exceeded 0.5. Factor loading of all questions was significant with a range of 0.56 to 0.88 (Figure 1). In addition, AVE from of all constructs exceeded the minimum criterion of 0.5,[28] indicating that a large portion of the variance was explained by the constructs. The final instrument included 12 items related to three dimensions which can explain HL including Functional (n=4), Communicative (n=4) and Critical (n=4). The loading factors for the items on each construct were greater than loadings with all the remaining constructs, and the AVE squared of any construct was greater than its correlation values with other constructs (Fornell and Larcker test), suggesting divergent validity (Tables 2 and 3).

Figure1

Confirmatory Factor Analysis of the HF-specific HL-Persian version m1-m12: Items of scale; HL: Health literacy; HF: Heart failure

Table 2

Convergent validity and reliability of the HF-specific HL Persian version

Variables	Cronbach’s Alpha	Composite Reliability	Average Variance Extracted (AVE)
Functional health literacy	0.801	0.855	0.520
Communicative health literacy	0.774	0.810	0.598
Critical health literacy	0.882	0.919	0.740

Table 3

Divergent Validity of the HF-specific HL scale

Variables	Functional health literacy	Communicative health literacy	Critical health
Functional health literacy	0.721
Communicative health literacy	0.587	0.773
Critical health literacy	0.215	0.292	0.860

The Structural Model of the HF-Specific HL

In the present study, the subscales’ R square and path coefficient were greater than the recommended threshold, which was the representative of the suitability of this criterion. The t-value derived from bootstrapping (100 resamples) in Smart-PLS shows the significance of the effects of variables on each other. All variables were significant at the confidence level of 0.1%. Table 4 shows the summary of the results of testing the structural model of the HF-specific HL scale. Finally, GoF=0.36, indicating the model good fit.

Table 4

Summary of results of structural model of the HF-specific HL, Persian version

Variables	R square	Path coefficient	t value	Significant level
Functional health literacy	0.432	0.657	5.400	0.001
Communicative health literacy	0.601	0.775	13.784	0.001
Critical health literacy	0.711	0.843	21.881	0.001

Composite reliability and Cronbach’s alpha coefficient for all of the constructs were over the recommended threshold of 0.70, indicating the adequate internal consistency of multiple items for each construct.[31] The stability of the scale was examined through a two-week interval test on a sample of 10 patients. It was satisfactory to good, with ICC value ranging from 0.78 to 0.90. The ICC had the lowest value in the critical dimension, and in the functional dimension, it had the highest value.

DISCUSSION

The aim of this study was to assess the psychometric features of HF-specific HL scale. This tool can be used to measure HL in the research and treatment of HF patients. The application of the questionnaire was simply possible and could be completed by patients or health care providers in a variety of settings such as a hospital or health center, in less than five minutes. The face and content validity of the questionnaire were approved with minor modifications. CVI and CVR for the Persian version of the HF-specific HL scale was greater than recommended amount, which was consistent with the Chinese version of scale.[23] However, the CVI and CVR of the original scale were not reported.

HF has different dimensions. Accordingly, Nutbeam has proposed a specific framework for this concept, in which unlike other scales, HL is measured in three domains: functional, communicative, and critical. Most previous instruments of HL in patients with HF used scales such as the TOFHLA and the REALM, which is applied only to the functional domain of HL. However, for patients with HF to change their everyday lives and assess signs and symptoms of their disease, it is essential to acquire suitable information and make good use of it in self-care behavior. As defined by Nutbeam, functional, communicative, and critical HL includes the individuals’ abilities that develop gradually. Functional HL refers to the basic skills in reading and writing. Communicative HL reflects ‘‘advanced cognitive and social skills’’ that can be used to extract meaning from different forms of communication and apply new information to change the circumstances. Critical HL includes ‘‘advanced cognitive skills and social skills’’ which can be applied to critically analyze information and use this information to exert greater control over life situations.

In this research, CFA was used to investigate the construct validity of the questionnaire. The results of the three factors confirmed functional, communicative and critical HL based on Nutbeam’s HL model. This finding is consistent with those of other studies,[19,21] which supported the construct validity of measuring the 3 HL domains. the results of the analysis of the main components of conceptualization and the construction of sub-scales were empirically supported in the original study.[21] Also, in Chinese version of the HF-specific HL scale, three factors were proposed in patients with HF, but there was not much difference between item 8 and item 9.[23] In our study, the factor loading of all questions was significant with the recommended range. Thus, the convergent validity of the scale was confirmed, which is consistent with the results of the original study.

The correlation between the constructs was lower than the value of the square root of the AVE for each of the constructs, showing that the indicators are more strongly related to their specific construct than to the other construct. Therefore, the divergent validity was verified. This is the first study to use PLS approach to validate an instrument that specifically evaluates HL in HF patients, which makes it difficult to compare these findings with the results of other studies.

In this study, the composite reliability and Cronbach’s alpha coefficient for all the sub-constructs were greater than 0.7, indicating a good result. The results of the original and Chinese version of the scale are consistent with our study. The reliability of the questionnaire was also evaluated by using a re-test; the results obtained from two times of testing with a two-week internal showed the high stability of the above-mentioned questionnaire. The original and Chinese version approved the stability of HF-specific HL scale.[21,23]

The study had some limitations that should be mentioned. The use of non-random sampling limits the generalizability of the results, and the sample was restricted to a group of patients referring to a teaching hospital in the Qom, Iran. In addition, further research is recommended to evaluate the sensitivity and specificity of the scale in feature.

CONCLUSION

In conclusion, HF-specific HL scale is a validated and reliable tool. Due to the features as simple scoring, appropriate reliability and validity, ability to be completed in short time, it seems that this questionnaire is a suitable instrument. HF-specific HL scale can be used by health providers for measuring HL and determine the effects of different interventions on the patients’ health outcomes.