Development of an Instrument to Measure Dietary Supplement Health Literacy.

BACKGROUND: Health literacy is one of the important social determinants of health.
OBJECTIVE: The aim of this study was to develop an instrument to measure Dietary Supplement Health Literacy (DSHL) of Iranian women.
METHODS: In this psychometrics study, an initial nine-factor instrument was developed. Face validity and content validity of the instrument were evaluated. The factor structure of the instrument was explored by the Exploratory Factor Analysis (EFA) among 400 women taking Dietary supplement. Confirmatory Factor Analysis (CFA) was done to determine the underlying factor structure of the instrument in this population. The internal and external reliability of the instrument was evaluated. KEY
RESULTS: According to expert panel opinions, 16 items were deleted. The results of the EFA showed that the Kaiser-Meyer-Olkin and Bartlett's test of sphericity were significant. EFA showed that 30 items could be grouped into nine factors that accounted for 60.84% of the variance. Since two items in the ability to actively engage with health care providers factor were loaded in factor 1, this factor was labeled Engaging in receiving informational supports from health care providers. In addition, one item of the dimension was loaded in factor 3. Given that the one item of factor 5 and 1 item of factor 6 were loaded in a new factor 9, this factor was labeled Applying information to decision-making. The CFA indicated that the nine-factor structure of the DSHL instrument had a poor fit. To modify indices, factor 9 with 2 items and 1 item of factor 6 were deleted. The Cronbach's alpha and intraclass correlation coefficient of the instrument were acceptable. Finally, a 27-item instrument with 8 dimensions was confirmed.
CONCLUSIONS: The results of the study showed that the instrument developed was a valid tool for identifying the DSHL of Iranian women. [HLRP: Health Literacy Research and Practice. 2022;6(2):e159-e166.] Plain Language Summary: This study sought to develop and validate a multidimensional instrument to measure the health literacy of Iranian women about dietary supplements that was performed from July 2019 to May 2020 in Iran. Findings showed that the 30-item instrument developed in this study is a valid instrument to be used for identifying the health literacy of Iranian women about dietary supplements.

Dietary supplement (DS) includes such ingredients as vitamins, minerals, herbs, amino acids, and enzymes. These products are marketed in the forms of tablets, capsules, soft gels, gel caps, powders, and liquids (U.S. Food and Drug Administration, n.d.).

The DS market in Iran is experiencing a significant growth (Khosroshahi et al., 2017). In this country, DS are distributed by the companies authorized by the Iranian Food and Drug Administration. The supplements are only provided by pharmacies to the customers with no need for a prescription (Food & Drug Organization of Iran, 2015). The use of DS is increasing among Iranian adults (Sotoudeh et al., 2015). They use DS for many reasons (Hoseini et al., 2021). Literature showed that gender has been identified as one of the determining factors in the consumption of DS (Hoseini et al., 2020). Women of reproductive age (15–49 years) are the main DS consumers (Raiten et al., 2003).

Despite the side effects of incorrect consumption of DS (Ebbing et al., 2009), the level of knowledge about DS has been reported to be low (Kołodziej et al., 2019). Most of the people believe that DS had no risk and consumed these products based on recommendations from friends or families rather than from health care professionals. There is a need to increase the consumers' knowledge as to the efficacy and safety of DS (El Khoury et al., 2016) and their health literacy (HL) (Pitug et al., 2020). HL should be considered when educating people about DS use (Leung et al., 2015). Considering that, HL is the knowledge and competence to access, understand, appraise, and apply health information for health judgment (Garcia-Codina et al., 2019). HL was introduced as one of the most important social determinants for health (Duong et al., 2017). In assessment of the HL of each target group in specific health topics, the existence of valid instruments is essential (Okan et al., 2018). To the best of our knowledge, some measurement tools for determining the knowledge of people about DS (Karbownik et al., 2019) or DS Choice (Kakutani et al., 2019) were developed. There is no instrument to assess the Dietary Supplement Health Literacy (DSHL). Valid measurement tools may provide knowledge to perform effective intervention in the field of HL (Okan et al., 2018). The aim of this study was to develop and validate a multidimensional instrument to measure the DSHL of Iranian women.

Materials and Methods

Study Design

This psychometrics study was performed from July 2019 to May 2020 in Quds City, Tehran province, Iran. This study protocol was approved by the ethics committee of Iran University of Medical Sciences (code: IR.IUMS.REC.94-0527-27359). All participants were informed about the objectives of the study and written consent was obtained.

Developing Instrument for Measuring DSHL

For developing the DSHL, the following stages were done.

Scale development process was started with item generation. Deductive and inductive approaches were used for item generation (Boateng et al., 2018). In the study, we selected Health Literacy Questionnaire (HLQ) dimensions as the theoretical framework to develop the DSHL instrument dimensions (Osborne et al., 2013). Measurement tools of HL vary greatly in the factors of HL that they measure (Dodson et al., 2015). Most developed instruments tend to measure general HL in limited dimensions (Liu et al., 2018). The HLQ covered nine conceptually distinct areas of HL and captured a wide range of the lived experiences of people attempting to engage in understanding, accessing, and using health information and health services. The tool also provides a reflection of the quality of health and social service provision (Osborne et al., 2013). Once the domains of DSHL instrument were delineated, an item pool to measure the dimensions was prepared. Forty-two semi-structured interviews were done with women taking DS and 10 interviews with experts in health education, nutrition, and pharmacology. During the interviews, the participants were asked about the items of each of the dimensions of the instrument. For example, in the dimension having social support for health, the participants were asked which people and sources of information they would prefer to provide the necessary support and information to take DS. Sampling continued until the data saturation. All interviews were recorded, listened to, and transcribed word for word. The data were explored using content analysis. An initial item pool of 85 items was generated. After a careful review of the items by the research team, the number of items was reduced to 46 (Ghanbari et al., 2016). Finally, a 46-item instrument with 9 dimensions including feeling understood and supported by healthcare providers (3 items), having sufficient information to manage my health (6 items), actively managing health (7 items), having social support for health (6 items), appraising health information (5 items), actively engaging with healthcare providers (4 items), navigating the health care system (3 items); being able to find good health information (6 items), and understanding health information well enough to know what to do (6 items) was developed.

In this stage, content validity of the instrument was assessed. An expert panel consisting of 15 specialists in health education and 7 in nutrition and pharmacology judged about the relevance or representativeness of the scale items for assessing the quantitative content validity. According to their reflections, Content Validity Index (CVI) and Content Validity Ratio (CVR) of the items were assessed. The CVR is defined as the direct linear transformation of a proportional level of agreement on the number of experts who assess an item as essential. The CVR formula is CVR = (ne - N/2)/(N/2), where CVR is the content validity ratio, “ne” is the number of panelists indicating an item “essential,” and N is the total number of panel members. The necessity of the items was assessed using a three-point rating scale as (3) essential, (2) useful but not essential, and (1) not necessary. The CVI is also computed by counting the number of experts who assessed the item as 3 or 4 and dividing that number by the overall number of experts. Given that the relevance of the items was assessed using a four-point rating scale: (1) not relevant, (2) slightly relevant, (3) relevant, and (4) very relevant; the items which had CVR less than 0.75 and CVI less than 0.79 (Lawshe, 1975; Polit & Beck, 2004) were deleted. Also, the experts were asked to review the items and reflect their comments in terms of grammar, wording errors, use of appropriate words, placement of items in the right place, and scaling of the items (qualitative content analysis).

Face validity was assessed quantitatively by asking for opinions of 30 women taking DS regarding the importance level of each item and its alternatives in a 5-point Likert scale from not important at all (score = 1) to very important (score = 5). Afterwards, the impact score of each item was specified by estimating the result of multiplying the importance coefficient by relative frequency; the items scored ≥1.5 remained in the instrument (Abbasi et al., 2020). Also, the participants told their judgments on the relevance, ambiguity, and difficulty of the items (qualitative face validity).

Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were done to approve the construct validity of the instrument. Literature showed that a minimum of 300 to 450 participants were needed to observe an acceptable comparability of patterns (Guadagnoli & Velicer, 1998). In the study, 800 (EFA = 400, CFA = 400) women consuming DS was selected from two Primary Health Care Facilities of Quds city affiliated to Iran University of Medical Sciences, Tehran, Iran, using simple random sampling method. They completed the instrument. The inclusion criteria were as follows: (1) female sex, (2) use of at least one DS in the past month, (3) residence in Quds city, (4) willingness to participate in the present study, (5) Iranian nationality, and (6) age 18 to 65 years.

First, to determine the adequacy of the sample and the appropriateness of factor analysis model, the Kaiser-Meyer-Olkin (KMO) test and the Bartlett's test of sphericity were done. Then, EFA with varimax rotation, a cutoff point of 0.4 for the factor load, and an Eigen value of >1 was performed to identify the main factors of the instrument (Hsu & Hsieh, 2013; Tran et al., 2013).

CFA was used to investigate how well the measured variables represent the number of latent constructs. Model fit was evaluated by the following indices: Chi-Square test, Chi-square to df ratio (χ2/df) <5, Comparative Fit Index (CFI) >0.90, Tucker-Lewis Index (TLI) >0.90, Root Mean Square of Approximation (RMSEA) <0.10 and Standardized Root Mean Square Residual (SRMR) <0.10 (Wang & Wang, 2012).

The external reliability and internal consistency of the instrument were calculated. As to the measurement of the internal consistency using Cronbach's α coefficient, 30 women taking DS completed the instrument. To assess the external reliability of the instrument, intra-class correlation coefficient (ICC) was measured (with a 2-week interval between the tests) in the 30 women taking DS. Cronbach's alpha values ≥0.70 (Shahsavari et al., 2020) and ICC ≥0.70 (Clark et al., 2015) were considered satisfactory.

Statistical Analysis

Statistical analyses and EFA were performed using SPSS (version 20.0, SPSS, Inc) software package. CFA was conducted using JASP 0.13.1 software (JASP Team, version 0.13.1). The participants' general characteristics were analyzed using frequency, percentage, mean, and standard deviation.

Results

Content Validity

At this stage, 16 items having CVR <0.75 and CVI <0.79 were removed from the initial instrument.

Face Validity

The results of revealed that all 30 items had impact score index ≥1.5. Some minor wording errors were edited.

Construct Validity

Demographic characteristics of the participants in EFA and CFA are shown in Table . The KMO test value was 0.752, which indicates that the study sample was adequate. Given that the result of Bartlett's test of sphericity was significant (3179.468, df = 435, p < .001), the factor analysis model was appropriate (Wang & Wang, 2012). EFA yielded 9 factors with an Eigenvalue of >1. The factor loads ranged between 0.48 and 0.69, and the total variance of the nine-factor model was 60.84% (Table (Wang & Wang, 2012). After performing content validity, three items remained in the ability to actively engage with healthcare providers domain. The results of EFA showed that two items of this dimension were loaded in factor 1 (feeling understood and supported by health care providers dimension). According to the content and conception of five items loaded in factor 1 and the opinion of experts, factor 1 was labeled Engaging to receive informational supports from health care providers. In addition, 1 item of ability to actively engage with healthcare providers dimension was loaded in factor 3 (navigating the health care system). Therefore, the number of items of factor 3 increased to 4. Findings of this stage showed that 1 item (i.e., If I have any questions about dietary supplements and how to take them, I know who to consult) of factor 5 (appraising health information) and 1 item of factor 6 (being able to find good health information) were loaded in the new factor 9. According to content and conception of two items loaded in this factor and the opinion of experts, this factor was labeled Applying information to decision-making (Table .

The goodness of fit indices of CFA of the instrument developed with 30 items were inadequate. Therefore, to improve instrument, two items of factor 9 (Applying information to decision making) have been removed. Also, the third item (I can easily understand the explanations given by a doctor or other health professionals about DS) of factor 6 (Ability to find good health information) having factor load of <0.3 was deleted from the instrument. Finally, the CFA model provided a good fit to the data (χ2 = 412.32, df = 322, p < .001, χ2/df = 1.28, TLI = 0.96, SRMR = 0.05, CFI = 0.97, and RMSEA = 0.027) (Wang & Wang, 2012). The final instrument included eight dimensions with 27 items and was a reasonable fit to the data. The results of CFA are presented in Figure .

Confirmatory factor analysis diagram.

Reliability

Cronbach's alpha and the ICC of the developed instrument dimensions were, respectively, in the ranges of 0.72 to 0.91 and 0.76 to 0.90. Given the internal consistency and external reliability of all dimensions were acceptable and thus we did not delete any items in this stage (Table ).

Final Instrument

The final version of the DSHL instrument had 27 items with 8 dimensions including engaging to receive informational supports from healthcare providers (5 items), having sufficient information to manage health (3 items), actively managing health (4 items), having social support for health (3 items), appraising health information (3 items), navigating the health care system (4 items), being able to find good health information (2 items), and understanding health information well enough to know what to do (3 items). The items of all dimensions were measured on a Likert scale ranging from 5 = always to 1 = never.

Discussion

In the study, an instrument was developed to assess the DSHL of Iranian women. The measurement tool consisted of 8 factors with goodness-of-fit indices, which were verified based on the results of CFA. It is important to mention that there is no consensus about the conceptual dimensions of HL. Literature showed that there were 12 conceptual models containing 12 dimensions on HL (Sørensen et al., 2012). In the study, the conceptual framework of HLQ was selected to develop the DSHL instrument dimensions (Osborne et al., 2013). In the study, according to EFA, two factors were integrated, and one new factor labeled Applying information to decision making generated. Because CFA indicated that the nine-factor structure of the DSHL instrument had no appropriate fit, newly generated factor was deleted. Finally, eight factors were confirmed CFA.

Some factors of the DSHL instrument including appraising health information, being able to find good health information, and understanding health information well enough to know what to do were measured in most of the HL measurement tools (Brørs et al., 2020; Chau et al., 2015; McCormack et al., 2010). These three factors consisted of items confirming the basic ability of individual to access, understand, and interpret and evaluate health information. For some time, most emphasis on these factors and the first HL questionnaires designed mainly measured these dimensions (Davis et al., 1993). Since that definition and concept of HL have become too broad in recent years, the assessment of only these factors is not enough (Peerson et al., 2009). Pleasant et al. (2011) showed that the existing measures of HL often ignore important dimensions such as how health professionals and systems communicate with clients. They suggested that developing new comprehensive approaches to assessing HL is essential (Pleasant et al., 2011).

One of factors of the DSHL instrument was engaging to receive informational supports from health care providers. On EFA, this factor was generated from the integration of the initial two factors including Ability to actively engage with health care provider and Feel understood and supported by health care providers. In the HLQ and the instrument validated by Maindal et al. (2016), these two factors were confirmed separately by EFA (Osborne et al., 2013). Some instruments of HL have considered the people's ability to communicate with health providers and ask them to re-explain their (Haghdoost et al., 2015). This factor plays an important role in HL (Berkman et al., 2020) and needs attention to the HL measurement tools.

Navigating the health care system was one of the factors of the DSHL instrument. The factor is important. Because people having low ability in the factor are not able to advocate on their own behalf and are unable to find someone who can help them use the healthcare system services. Assessing the factor may provide a guidance about the needs and outcomes of individual and health care organizations (Osborne et al., 2013). Recent definitions of HL have highlighted the importance of interactions between people, health care providers and health care systems (Liu et al., 2020).

Having social support for health was another factor of the DSHL instrument. In the factor, information support from social networks (family and friends) was investigated. Despite the importance of the social context in HL (Sentell et al., 2017), few tools have considered social support as one of the domains of HL. In a study by Chinn and McCarthy (2013), access to support networks was recognized as one of the dimensions of the developed scale to measure HL.

In the DSHL instrument, there were items to measure two factors having sufficient information to manage health and actively managing health. These factors highlight the fact that getting sufficient health information is necessary, but not enough. Information does not necessarily predict the outcomes and decision making (Souza et al., 2020). People must be able to use the information to manage their health. The purpose of these two factors is to foster the individuals' self-management and empowerment. In the inventory developed by Jordan et al. (2013), the use of information to make decisions about one's health was confirmed as a factor. In the instrument developed by Sørensen et al. (2013), applying health information in tasks concerning decision-making factors were confirmed. Developing new instruments to measure HL using a skills-based approach is essential.

Study Limitations

The data were collected using a sample of women who had referred to the primary health care facilities in Quds city, Tehran, Iran. Since Quds city is in the low- to median-income areas of Tehran province, the homogeneity of the samples may limit the extent to which the findings can be generalized to other women residing in other areas of Tehran province and Iran.

Conclusion

In conclusion, the findings of the study showed that the DSHL instrument presented within the HLQ had appropriate validity and reliability. The results of the study may be used by health providers to develop tailored education interventions to increase DSHL in Iranian women.

Table A

Results of Exploratory Factor Analysis of Health Literacy Instrument in Terms of Dietary Supplements (N = 400)

Items	Factors									Communalities (Extraction)
	1	2	3	4	5	6	7	8	9
	Engaging to received informational supports from health care providers	Actively managing health	Ability to navigate the health care system	Have sufficient information to manage health	Appraise health information	Ability to find good health information	Have social support for health	Ability to understand health information well enough to know what to do	Applying information to decision-making
F1I1. I talk to my doctor, pharmacist, or health care professional about possible drug interactions with dietary supplements	0.803									0.669
F1I2. Before taking any dietary supplement, I make sure to consult my doctor, relevant specialist or health care provider	0.787									0.657
F1I3. I ask my doctor, pharmacist, or other health professionals for questions about the side effects of taking dietary supplements	0.755									0.597
F1I4. I have a doctor and a health expert whom I will refer to if I have symptoms of vitamin and mineral deficiency	0.676									0.556
F1I5. The information provided by my doctor or health experts will help me make the right decision	0.608									0.578
F2I1. I read drug supplement brochures before taking them and ask the pharmacist or other experts for vague tips		0.794								0.666
F2I2. I manage and take dietary supplements according to my doctor's instructions		0.766								0.653
F2I3. To take dietary supplements, I ask my doctor to determine how much my body really needs these supplements through testing		0.761								0.625
F2I4. I present the list of medications I take to my doctor or pharmacist to determine any possible drug interactions with dietary supplements		0.753								0.629
F3I1. When I feel the need to take supplements, I know who to turn to for advice			0.800							0.693
F3I2. When there are symptoms of lack of supplement in my body, I can ask my doctor or health care professional to check my condition			0.765							0.616
F3I3. When I feel the need to take supplements, I can find someone who can give me the advice I need to use the relevant health services			0.731							0.635
F3I4. If I have any questions about dietary supplements and how to take them, I ask the pharmacist or other experts			0.558							0.533
F4I1. I know that dietary supplements should be based on the body's needs identified by medical tests				0.807						0.678
F4I2. I know that taking too many supplements is bad for my body				0.727						0.662
F4I3. I have enough information about how to take dietary supplements				0.656						0.636
F5I1. I can assess and analyze the information and advertisements provided on the Internet or other media about dietary supplements					0.793					0.693
F5I2.When confronted with new information about dietary supplements, I can check and analyze their accuracy					0.724					0.603
F5I3. I can review and analyze the ingredients when buying supplements					0.600					0.525
F6I1. I easily understand that taking supplements should be according to the needs of the body						0.771				0.627
F6I2. It is easy for me to read what is written in supplements brochures (such as uses, dosage and side effects)						0.736				0.587
F6I3. I can easily understand the explanations given by a doctor or other health professionals (for example, nutrition) about dietary supplements (each person's body needs and drug interactions)						0.663				0.608
F7I1. I talk to my friends and relatives about taking supplements							0.730			0.599
F7I2. My family reminds me of the side effects of taking supplements arbitrarily							0.692			0.562
F7I3. My family supports me in preparing and consuming dietary supplements prescribed by my doctor							0.691			0.577
F8I1.I can get information about the side effects and drug interactions about dietary supplements from various sources								0.689		0.546
F8I2. I can get information from various sources about the side effects of taking dietary supplements arbitrarily								0.630		0.547
F8I3. I can get information about the uses and benefits of dietary supplements from authorized scientific sources								0.549		0.489
F9I1. I decide to consume a dietary supplement based on information received of various resources in terms of its side effects									0.777	0.674
F9I2. I decide to consume the dietary supplements based on information that I receive from people around me									0.638	0.533
Eigenvalue	4.527	2.752	2.351	2.195	1.477	1.372	1.269	1.196	1.114
Explained variance (%)	15.091	9.175	7.836	7.316	4.923	4.574	4.230	3.988	3.712
Cumulative variance (%)	15.91	24.266	32.102	39.418	44.341	48.915	53.145	57.133	60.845

Table 1

Demographic Characteristics of the Study Participants for Assessing the Exploratory Factor Analysis and Confirmatory Factor Analysis (N = 400)

Variable[a]	EFA	CFA
	n (%)
Marital status
Single	150 (37.5)	146 (32.5)
Married	236 (59)	239 (59.8)
Divorced or widowed	14 (3.6)	15 (3.8)
Education level
Illiterate	1 (0.3)	1 (0.3)
≤12th grade	323 (80.7)	323 (80.7)
>12th grade	76 (19)	76 (19)
Occupation status
Self-employed	13 (3.3)	13 (3.3)
Employee	30 (7.5)	32 (8)
Unemployed	101 (25.3)	97 (24.3)
Retired	2 (0.5)	2 (0.5)
Household duties	224 (56)	226 (56.5)
Causal labor	30 (7.5)	30 (7.5)

Age, Mean ± SD (EFA): 38.152 ± 11.441; Mean ± SD (CFA): 38.375 ± 11.425.

Table 2

Cronbach's Alpha and ICC of the Dimensions of Health Literacy Instrument (N = 30)

Dimension	Number of Items	M (± SD)	Cronbach's Alpha	ICC
Engaging to receive informational supports from health care providers	5	16.25 ± 2.36	0.78	0.76
Actively manage health	4	16.0 ± 2.67	0.76	0.90
Ability to navigate the health care system	4	16.12 ± 7.2	0.91	0.78
Have sufficient information to manage health	3	12.48 ± 3.23	0.72	0.77
Appraise health information	3	12.22 ± 2.39	0.77	0.78
Ability to find good health information	2	9.4 ± 2.31	0.76	0.76
Have social support for health	3	12.0 ± 2.16	0.72	0.76
Ability to understand health information well enough to know what to do	3	12.25 ± 2.36	0.86	0.76

Note. ICC = intraclass correlation coefficient; M = mean; SD = standard deviation.