Designing and Investigating the Validity and Reliability of the Health Literacy Questionnaire in Iran: Recognizing the Risk Factors for Cardiovascular Diseases, Diabetes, and Cancer.

BACKGROUND: Health literacy (HL) has been recognized as an important concept in patient education and disease prevention. The rising burden of noncommunicable diseases (NCDs) in Iran is significant. Hence, we designed and validated an HL questionnaire on the most important domains of NCDs, including cardiovascular diseases, diabetes, and cancer.
METHODS: Literature review was conducted to examine the definition and dimensions of HL. After reaching consensus about the HL dimensions and conceptual models in focus group discussions with experts, they designed questions in each domain. Then, face, content, and construct validity as well as reliability were determined by a pilot study on 72 participants. At the end, a cross-sectional study was implemented on 206 Hamedan university employees, to finalize the questionnaire.
RESULTS: After doing the pilot study and analyzing the collected data and according to the Bartlett's test of sphericity and Kaiser-Meyer-Olkin = 0.421 with P < 0.001, factor analysis was used. Considering the eigenvalue >1.4, a 27-item questionnaire in seven domains was obtained which included attitude toward health, understanding information, social support, socioeconomic conditions, access to health services, and application of health information. Cronbach's alpha was more than 0.70 in all domains except the last one (0.47). The second phase showed that overall 75.2% of the individuals had inadequate HL with lowest scores in the application of health information.
CONCLUSIONS: The designed tool seems appropriate for measuring the HL level among the Iranian population in the field of prevention of cardiovascular disease, cancer, and diabetes. The results can help policy makers to improve health promotion interventions. Copyright:

Introduction

“Health literacy is linked to literacy and entails people's knowledge, motivation and competences to access, understand, appraise, and apply health information in order to make judgments and take decisions in everyday life concerning healthcare, disease prevention and health promotion to maintain or improve quality of life during the life course.”[1]

The six important dimensions of health literacy (HL) include functional, interactive, autonomous, informational, background, and cultural competencies. HL has affected health behaviors and the use of health services. Furthermore, it affects health outcomes at personal and social levels such as quality of life, health costs, mental health, and equity in health.[1234]

Various studies have been conducted in different countries to determine the HL level, but an overall consensus has yet to be reached.[56789] The HL level depends on personal traits and the level of communication between people and the health system. Thus, different tools are required for different age groups and different stages of life, and these should be born in mind when measuring HL.[910]

In spite of the various tools and scales present for the assessment of HL, the design and standardization of tools such as the Iranian Urban Population Health Literacy Assessment or the Health Literacy Assessment of Adolescents Tools, and the undertaking of relevant investigations in this field, a scale that is generalizable to the country has not been developed yet.[1112] Based on a systematic review in Iran, a localized tool for the assessment of HL is still lacking.[13]

According to existing literature, Iran is exposed to the risk of noncommunicable diseases (NCDs) at an alarming rate,[14151617] and premature morbidity and mortality resulting from NCDs are on the rise.[181920] In Iran, 82.2% of NCD deaths are due to cardiovascular diseases (CVD), cancers, chronic pulmonary diseases, and diabetes; and diabetes mellitus and hypertension have a stronger relationship with sever coronary artery disease.[2122]

This study was an effort to design an HL questionnaire on the most important domains of NCD, including CVD, diabetes, and cancer in Iran and to examine its validity and reliability. By doing so, we may achieve basic data on the current status of HL and identify the interventions required for promoting it.

Methods

Literature review was conducted to present the definition of HL in a theoretic and functional form that takes into account the various dimensions of the structure under study, one that clearly describes the significance of each of the dimensions. To this end, the ISI, PubMed, Scopus, and Web of Science databases were searched. The search strategy was as follows: 17 keywords (definition, model, concept, dimension, framework, conceptual framework, theory, analysis, qualitative, quantitative, competence, skill, “public health,” communication, information, functional, AND critical) were combined using the preposition “and” with the terms “health literacy,” “health competence,” and health competence (without quotes), and health literacy (without quotes). The inclusion criteria of the studies were as follows: (1) being in English, (2) having examined HL in developing countries, and (3) being relevant to the definition, and/or concept of HL and/or a combination of the two. Using Google's search engine, we further searched the web using the same keywords. We used the references used by the articles and in particular the review studies to complete our documentations. The studies were selected for further reading after examining their titles and abstracts. Thus, the full texts of the selected articles were thoroughly studied. The existing definitions of HL and its conceptual models were extracted from the articles.

Thereafter, two focus group discussions (FGDs) were held with five experts (two community medicine specialists, two internal medicine specialists, and one cardiologist). During these sessions, the conceptual models and definitions obtained from the literature review were discussed and examined for the various dimensions of HL and eventually, consensus was reached. Each of the experts designed appropriate questions for each domain separately. Then, after sharing the questions through email, two more FGDs were held to reach consensus on the initial questions.

The content validity of the questionnaire was examined from the experts’ perspective by sending the initial questionnaire to 20 experts. We received 12 completed questionnaires in which the respondents had addressed and scored each of the question's appropriateness, transparency, and comprehensiveness.

Since the target groups of the questions were not experts, two FGDs were held with 15 nonexperts as potential participants of the study and two individuals from the main research team as facilitators. During these meetings, each person separately commented on the appropriateness, transparency, and comprehensiveness of each question and the entire questionnaire in relevant checklists.

Upon summarizing and analyzing the results of the earlier stages (correction or removal of questions that scored lower than 70% in their appropriateness and transparency in the consensus), the initial questionnaire was completed. The pilot study was conducted among 72 employees from Hamedan's Youth and Sports Department; nonrandomized simple sampling was done (23).

The data were analyzed using SPSS. First, the descriptive analysis was done, and then factor analysis was performed to classify the questions.

To finalize the questionnaire, a cross-sectional study was conducted to investigate the HL level of public employees on the risk factors of CVDs, diabetes, and cancer among 206 employees of Hamedan's Bu-Ali Sina University. Random sampling was done.

The data were analyzed using SPSS 16: Statistical Software (Chicago, IL, USA). After descriptive analysis, Cronbach's alpha was calculated for each domain to assess the reliability of each questionnairesection (CVD, diabetes, cancer). To calculate the internal validity of each questionnaire, the overall score for each domain was calculated. Moreover, Pearson's correlation coefficient between the questions of each domain and the overall score of each questionnaire was also computed. A correlation coefficient greater than 0.5 was considered appropriate.

To calculate the score of each domain, the sum of the questions was calculated and divided by the number of questions in that domain. The result was a number between 1 and 5; a score lower than 4 was considered an inadequate level of HL and equal to or greater than 4 was considered as adequate.

Mann–Whitney and chi–square tests were used to examine the association between HL and demographic factors. Level of significance was considered at 5% for all cases.

Results

Based on the agreed definition of HL[1] that covers different dimensions of HL [Table 1], our experts concluded that each individual requires four types of competencies: (1) access: meaning the ability to search, find, and acquire health information; 2) understanding, that is, the ability to understand the available health information; (3) appraisal: meaning the ability to interpret, filter, judge, and assess the health information available; and (4) application: meaning the ability to communicate and utilize information for the sake of adopting decisions that lead to maintenance and improvement of health.

Table 1

The different dimensions of health literacy definitions

WHO (1998)	“The cognitive and social skills which determine the motivation and ability of individuals to gain access to understand and use information in ways which promote and maintain good health”
American Medical Association (1999)	“The constellation of skills, including the ability to perform basic reading and numeral tasks required to function in the healthcare environment”
Nutbeam (2000)	“The personal, cognitive and social skills which determine the ability of individuals to gain access to, understand, and use information to promote and maintain good health”
Institute of Medicine (2004)	“The individuals’ capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions”
Kickbusch et al. (2005)	“The ability to make sound health decision(s) in the context of everyday life - at home, in the community, at the workplace, the healthcare system, the market place and the political arena. It is a critical empowerment strategy to increase people’s control over their health, their ability to seek out information and their ability to take responsibility”
Zarcadoolas et al. (2003, 2005, 2006)	“The wide range of skills, and competencies that people develop to seek out, comprehend, evaluate and use health information and concepts to make informed choices, reduce health risks ad increase quality of life”
Paasche-Orlow and Wolf (2006)	“An individual’s possession of requisite skills for making health-related decisions, which means that health literacy must always be examined in the context of the specific tasks that need to be accomplished. The importance of a contextual appreciation of health literacy must be underscored”
EU (2007)	“The ability to read, filter and understand health information in order to form sound judgments”
Pavlekovic (2008)	“The capacity to obtain, interpret and understand basic health information and services and the competence to use such information to enhance health”
Rootman and Gordon-Elbihbety (2008)	“The ability to access, understand, evaluate and communicate information as a way to promote, maintain and improve health in a variety of settings across the life course”
Ishikawa and Yano (2008)	“The knowledge, skills and abilities that pertain to interactions with the healthcare system”
Mancuso (2008)	“A process that evolves over one’s lifetime and encompasses the attributes of capacity, comprehension, and communication. The attributes of health literacy are integrated within and preceded by the skills, strategies, and abilities embedded within the competencies needed to attain health literacy”
Australian Bureau of Statistics (2008)	“The knowledge and skills required to understand and use information relating to health issues such as drugs and alcohol, disease prevention and treatment, safety and accident prevention, first aid, emergencies, and staying healthy”
Yost et al. (2009)	“Identify and interpret information presented in graphical format (charts, graphs and tables), and perform arithmetic operations in order to make appropriate health and care decisions”
Adams et al. (2009)	“The ability to understand and interpret the meaning of health information in written, spoken or digital form and how this motivates people to embrace or disregard actions relating to health”
Adkins et al. (2009)	“The ability to derive meaning from different forms of communication by using a variety of skills to accomplish health-related objectives”
Freedman et al. (2009)	“The degree to which individuals and groups can obtain process, understand, evaluate, and act upon information needed to make public health decisions that benefit the community”

This process creates knowledge and skills that enable the individual to have a higher HL level when s/he is in either of the following three health statuses. Each person is in either of these three health statuses: is unwell or ill in the healthcare system; is at risk of disease in the system of disease prevention; or is a citizen in community health promotion programs, the work environment, educational system, political environment, and the market.

The combination of four dimensions related to health data processing and three levels of individual health status yields 12 dimensions of HL [Table 2]. Understanding HL increases the individuals’ competencies over healthcare domains, disease prevention, and health promotion.

Table 2

The matrix of four dimensions of health literacy in three different health domains

	Access/acquisition of health-related information	Understanding health-related information	Processing and appraisal of health-related information	Application and utilization of health-related information
Healthcare	The ability to access information on medical and therapeutic matters	The ability to understand medical information and to extract their meanings	The ability to interpret and appraise medical information	The ability to make informed decisions on medical matters
Disease prevention	The ability to access information on health risk factors	The ability to understand information on risk factors and to extract their meanings	The ability to interpret and appraise information on health risk factors	The ability to make informed decisions on health risk factors
Health promotion	The ability to update oneself on social, physical, and environmental determinant factors of health	The ability to understand information on social, physical, and environmental determinant factors of health and to extract their meanings	The ability to interpret and appraise information on social, physical, and environmental determinant factors of health	The ability to make informed decisions on social, physical, and environmental determinant factors of health

Here, upon consultation with the project team and relevant experts, we concluded that at the level of disease prevention, different dimensions of HL (access, understanding, appraisal, and application) should be focused on.

The initial questionnaire on the risk factors of CVD – including 35 questions – was developed. According to the pilot study which was conducted among 72 employees from Hamedan's Youth and Sports Department, seven questions were removed for lack of correlation with the remaining questions. According to the Bartlett's test of sphericity and Kaiser–Meyer–Olkin = 0.421 with P < 0.001, the condition for performing factor analysis was established.

To extract the appropriate components (domain), considering the eigenvalue >1.4, seven components were eventually brought into the component matrix. Overall, 71% variance of the questions under study was describable with the seven selected components.

Correlation values higher than 0.3 were considered for allocating a question to a domain considering the component matrix. Eventually, to specifically load each question in the appropriate domain, the oblimin method was used. The results obtained included a 27-item questionnaire in seven domains, attitude toward health, understanding information, social support, socioeconomic conditions, access to health services, and application of health information.

After doing these steps and conducting the cross-sectional study on 206 employees of Hamedan's Bu-Ali Sina University, these results were obtained. The median overall HL level on recognizing the risk factors of the three diseases was 3.7. The median health literacy levels on recognizing the risk factors for CVD, diabetes, and cancer were 3.53, 3.44, and 3.58, respectively. Except for cancer, the HL level was significantly higher among women for the other two diseases (P = 0.02). A rise in educational status also raised the frequency of adequate HL (P = 0008). Overall, 75.2% of the individuals had inadequate health literacy with regard to the risk factors of all the three diseases. The lowest HL was seen in the application of health information.[23]

To assess the reliability of the questionnaire, Cronbach's alpha was calculated for each domain, the results of which are given below.

In the first domain (the patient's attitude toward health), Cronbach's alpha for the questions on risk factors of cancer, CVD, and diabetes was 0.87, 0.82, and 0.78, respectively. Upon taking into account the questions of all the three diseases, Cronbach's alpha was 0.93.

In the second to sixth domains (understanding information, social support, socioeconomic conditions, access to health services, and communication with a health specialist), considering the similarity of questions in the three questionnaires, Cronbach's alpha was 0.69, 0.71, 0.68, 0.74, and 0.74, respectively.

In the seventh domain (application of health information), Cronbach's alpha for the questions on risk factors of cancer, CVD, and diabetes was 0.43, 0.36, and 0.36, respectively. Upon considering the questions of all the three diseases, Cronbach's alpha was 0.47.

To assess the internal validity of the questionnaire, the overall score of each domain was calculated, and Pearson's correlation among the questions of each domain and their overall scores was obtained; P value was <0.001 for the risk factors of the three diseases in all the domains [Table 3].

Table 3

Pearson’s correlation among the questions of each domain and the overall score

First domain
Questions (cancer risk factors)*	12	13	17	18	19	20	25	26	27	28
Pearson’s correlation	0.45	0.67	0.66	0.66	0.73	0.68	0.64	0.69	0.7	0.71
Questions (CVD risk factors)*	12	13	15	22	23	24
Pearson’s correlation	0.59	0.76	0.73	0.76	0.68	0.63
Questions (diabetes risk factors)	12	13	16	22	23
Pearson’s correlation	0.61	0.76	0.67	0.77	0.68
Second domain*
Questions	9	10	11	37
Pearson’s correlation	0.8	0.71	0.76	0.47
Third domain*
Questions	29	30	31
Pearson’s correlation	0.83	0.8	0.75
Forth domain*
Questions	35	38	39	40
Pearson’s correlation	0.78	0.74	0.74	0.5
Fifth domain*
Questions	1	2	3	4	5	34
Pearson’s correlation	0.72	0.55	0.63	0.56	0.68	0.48
Sixth domain*
Questions	6	7	8
Pearson’s correlation	0.83	0.81	0.78
Seventh domain
Questions (cancer risk factors)*	14	21	32	33
Pearson’s correlation	0.63	0.45	0.68	0.56
(P<0.001)
Questions (diabetes and CVD risk factors)*	14	32	33	36
Pearson’s correlation	0.57	0.67	0.57	0.47

CVD: Cardiovascular disease, *P<0.001

To calculate the score based on Likert scale, in each domain, the total score of the questions was computed [Tables 4-6].

Table 4

Descriptive characteristics of the domains measuring health literacy on recognizing the risk factors for cardiovascular diseases, diabetes, and cancer based on Likert scale

Domains	Number	Mean	Standard deviation	Median
Participants’ attitude toward evaluated diseases	171	4.12	0.707	4.33
Cancer	183	4.04	0.709	4.2
Cardiovascular diseases	191	3.99	0.758	4.16
Diabetes	199	3.91	0.767	4
Understanding information	202	3.54	0.79	3.62
Social support	203	3.02	0.842	3
Socioeconomic status	201	2.77	0.857	2.75
Access to health services	203	3.92	0.794	4.16
Relationship with health professional	206	3.11	0.931	3
Applying health information in general	191	2.79	0.688	2.8
Cancer	193	2.96	0.729	3
Cardiovascular diseases	198	2.96	0.71	3
Diabetes	198	2.96	0.71	3

Table 6

Frequency distribution of respondents based on health literacy level in recognizing the risk factors for cardiovascular diseases, diabetes, and cancer

Health literacy level	>4	≥4	Total
Health literacy level	157 (100%)	0	157
Health literacy level about diabetes risk factors	150 (81.1%)	35 (18.9%)	185
Health literacy level about cancer risk factors	132 (78.6%)	36 (21.4%)	168
Health literacy level about CVD risk factors	143 (79.9%)	36 (20.1%)	179

CVD: Cardiovascular disease

Discussion

In this study, we designed and validated an HL assessment tool for Iran. First, the HL assessment questionnaire was designed based on the items defined in HL – with a focus on prevention of CVD, diabetes, and cancer. After doing a pilot study and analyzing the collected data through factor analysis, seven domains were extracted for the measurement of HL.

After conducting the second stage of data collection aimed at finalizing the questionnaire and performing the analysis, we found that Pearson's correlation coefficient between the questions of each domain and their overall scores was higher than 0.5, which is considered appropriate in internal validity assessment. However, the correlation coefficient was 0.45 for the domain of “application of health information” on cancer risk factors, which contained the following question:

If you are a candidate (age-wise or gender-wise) for performing screening tests for breast, colon, female reproductive organs, and prostate cancers, have you performed at least one test?

Cronbach's alpha for the domain of “application of health information” on cancer risk factors was 0.43, which increased to 0.47 after removing the aforementioned question.

The lack of correlation between the screening question and the other questions could be due to people's inadequate knowledge on cancer screening. The remaining measures related to this domain, such as sports activities and/or healthy nutrition, are being well taught to the public through mass media.

In the same domain, the lowest correlation was observed in the screening questions on CVD and diabetes. Approximately 47% of the individuals reported not performing screening procedures for either CVD or cancer, whereas frequency has been divided among different options in the remaining questions. The public receive no training in the field of screening measures or their timings. Thus, screening procedures had the least correlation with the remaining questions in our questionnaire.

In Japan, a study was conducted to measure functional, communicative, and critical HL among patients with diabetes. The participants of the study were 157 type 2 diabetic outpatients of Tokyo University's Hospital. The inclusion criteria were having type 2 diabetes and being constantly under care of the hospital's endocrinologists. The participants were randomly selected and the objectives and procedure of the research were explained to them. The patients were selected upon examining their electronic records, age, sex, hemoglobin A1C, and other problems. In this study, functional, communicative, and critical HL were assessed. The level of hemoglobin A1C was lower in individuals with higher HL levels. Similar to our study, a close association was observed between HL and reduced complications of chronic diseases.[24]

A semi-structured telephone survey was conducted in a research involving 145 adults in Sweden. A series of questions similar to ours were asked about three risk factors. All the respondents were asked about the measurement of and level of blood pressure, blood glucose, and cholesterol, and whether they had taken any medical measures for either one of them. They were also asked about cigarette smoking and whether they had made any changes in their lifestyles to reduce the risk of CVD.[25]

An Italian study has indicated that the level of awareness of CVD is somewhat vague. Family physicians were generally able to present a quantitative assessment of CVD risk among their patients. CVD prevention programs would be more successful if they emphasized on public knowledge and awareness and family physicians were encouraged to perform more accurate quantitative assessments of CVD risk factors.[26]

In Iran, Tehrani et al. initially underwent the complete stages of validity and reliability testing of the “Test of Functional Health Literacy in Adult” questionnaire. This questionnaire assesses HL at a general level. Thereafter, they used it to examine HL and its determinant factors in five provinces of the country. In the adjusted regression model, they too found that HL was higher among women than in men (P = 0.14). HL level was severely low among individuals of low economic status (P = 0.004).[27]

Khosravi et al. used the “Test of Functional Health Literacy in Adult” questionnaire in another study[28] and assessed the HL level among patients with diabetes in Shiraz. Using the test–retest method, the validity of the questionnaire was calculated at 0.99.

The “Test of Functional Health Literacy in Adult” questionnaire is composed of two sections, numerical understanding and reading. The reading section assesses the patient's ability to read and comprehend texts. These texts are related to the guideline of preparing for upper gastrointestinal imaging, the patient's rights and responsibilities in insurance forms, and the standard hospital consent form. The score considered for this section is between 0 and 50. In the numerical section, the individual's ability to understand and act upon the physician's order – which requires calculation – is evaluated. This section contains 10 health guidelines or descriptions on the utilization of prescribed drugs, the time of visiting the physician, the stages of using financial help, and an example of a medical laboratory test. The score considered for this section is also between 0 and 50. It has 17 questions. The sum of these two sections’ scores gives the overall HL score – which is a score between 0 and 100. A score of 0–59 is considered inadequate, 60– 74 is considered borderline, and 75–100 is considered as adequate. The mean HL level of patients with diabetes visiting Shiraz's medical centers was calculated at 66, that is, borderline level. HL was significantly associated with age, sex, educational status, membership of the diabetic association, and type of employment.[28]

Tuyen et al. conducted a study in six Asian countries, in which the “Health Literacy Survey Tool European Health Literacy Questionnaire” (HLS-EU-Q47) was validated. This tool contains 47 HL items in the main domain of healthcare, disease prevention, and health promotion. A Likert scale of 1–4 has been used, wherein the responses range from 1 (very difficult) to 4 (very easy). After being translated, the tool was evaluated in a cross-sectional study by being applied on 10,024 participants age 15 years and above. Thereafter, the final analysis was performed using confirmatory factor analysis, internal consistency analysis, and regression analysis. A high internal consistency, with Cronbach's alpha at 0.9, and a satisfactory item-scale convergent validity greater than 0.4 were observed. The HL level in the aforementioned population was strongly correlated with social (P = 0.001) and educational (P < 0.001) status. As an important determinant factor of HL, the latter finding is consistent with our results.[29]

Maindal et al. investigated the HL level in Denmark using a questionnaire that had been designed in Australia for the first time and assessed it for its validity and consistency. This questionnaire contained 44 questions in NINE HL domains (understanding healthcare, having adequate knowledge for health management, active management of health, social support for health, health information assessment, the ability to interact with health caregivers, healthcare navigation system, the ability to find useful health information, and adequate understanding of health information for taking action). The study also assessed the factor structure, homogeneity, reliability, and discriminant validity of the above-mentioned questionnaire. After translation and back-translation, consensus conference, and cognitive interviews with 15 cases, the questionnaire was assessed among 481 individuals. The questions were examined with tests such as level of difficulty, composite scale confidence interval, and confirmatory factor analysis. The factor analysis showed acceptable and high loading (range: 0.49–0.93). Cronbach's alpha and composite reliability were greater than 0.8. Among the responses given, the simplest scale was “social support for health” and the most difficult scale was the “healthcare navigation system.” A Likert scale of 1–4 was used, wherein 1 represented “completely disagree” and 4 represented “completely agree.”[30]

The majority of the above-mentioned studies have measured HL among patients, while our questionnaire measures HL in healthy people in the field of prevention of CVD, cancer, and diabetes to improve health promotion interventions in the population.

Conclusions

Based on the results of this study, the designed tool seems appropriate for measuring the HL level among the Iranian population in the field of prevention of CVD, cancer, and diabetes, and it can be used in different studies.

There is a low level of HL concerning the recognition of risk factors of CVD, diabetes, and cancer. Among the HL domains, the lowest score was obtained in “application of health information” (6%). There is a relatively high frequency of appropriate HL level in the attitude toward, access to, and understanding of information among the patients. Thus, what appears to be generally causing the low level of HL among individuals is the lack of social support and poor economic conditions that have eventually led to the lack of appropriate action toward the application of health knowledge.

Financial support and sponsorship

This study was funded by Vice Chancellor for Research and Technology of Tehran University of Medical Sciences and Health Services grant number 89-04-62-12017.

Conflicts of interest

There are no conflicts of interest.

Table 5

The overall health literacy level in recognizing the risk factors for cardiovascular diseases, diabetes, and cancer

Health literacy level	Minimum	Maximum	Mean	Standard deviation	Median
Health literacy level about CVD, cancer, and diabetes risk factors	1.4	3.85	2.62	0.51	2.7
Health literacy level about diabetes risk factors	1	4.79	3.42	0.56	3.44
Health literacy level about cancer risk factors	1	4.82	3.52	0.56	3.58
Health literacy level about CVD risk factors	1	4.8	3.47	0.56	3.53

CVD: Cardiovascular disease