Adaptation and validation of a measure of health literacy in Taiwan: The Newest Vital Sign.

BACKGROUND: Health literacy (HL) refers to the ability to obtain, read, understand, and use basic health care information required to make appropriate health decisions and follow instructions for treatment. The Newest Vital Sign (NVS) is an instrument developed for assessing aspects of HL relevant to reading and numeracy skills. This study aimed to develop a traditional Chinese version of the NVS (NVS-TC) and assess its feasibility, reliability, and validity in Taiwanese patients with type 2 diabetes.
METHODS: The original NVS was translated into traditional Chinese in accordance with established guidelines. A cognitive testing procedure was subsequently performed to evaluate the ease of understanding and acceptability of the test in 30 patients with diabetes. Thereafter, a quantitative survey (N = 232) was administered for validating the NVS-TC against the accepted standard tests of HL and participant education level.
RESULTS: The internal consistency (Cronbach's α) was 0.76. In accordance with a priori hypotheses, we found strong associations between the NVS-TC and objective HL and weaker associations between the NVS-TC and subjective HL. The known group validity of the NVS-TC was demonstrated through multivariate regression analyses, which showed that educational differences in the NVS-TC scores remained significant after adjustment for age, gender, and working in healthcare.
CONCLUSIONS: The results suggest that the NVS-TC is a reliable and valid tool that facilitates international comparable HL research in Taiwan. The NVS-TC can be used to investigate the role of HL in health care and can be easily incorporated into daily clinical practice for diabetes management.

Diabetes is a prototypical multifactorial chronic condition characterized by a high level of complexity and requires extensive self-care education and major lifestyle changes. The demands on individuals with diabetes are complicated by the frequent dependency of the performance of diabetes self-management tasks on printed educational material and verbal instructions; therefore, patients with diabetes must possess adequate health literacy (HL) skills [1], [2]. HL is a set of skills that people require to function effectively in the health care environment. These skills include the ability to obtain, read, understand, and use basic health care information required to make appropriate health decisions and follow instructions for treatment [3], [4]. HL is considered crucial not only for understanding health information but also for health care outcomes. Studies have consistently associated low HL with poor diabetes knowledge [1], fewer self-management behaviors [5], [6], poor glycemic control [7], [8], [9], [10], and higher health care costs [11]. With emerging evidence of the adverse effects of inadequate HL on health care and outcomes in people with diabetes, the assessment of HL skills has become crucial in this population [3].

At a glance commentary

Scientific background on the subject

Adequate measures of health literacy facilitate the development of effective ways to improve self-care and health outcomes, particularly among those with low health literacy. However, limited health literacy measures suitable for use in clinical settings are currently available in Taiwan.

What this study adds to the field

These psychometric findings are in line with the results of other studies describing the development of the NVS for other languages. This research adds to the growing literature on health literacy measures and provides a valid health literacy screening tool in clinical settings in Taiwan.

An appropriate and valid instrument of HL is essential to determine the effect of HL on an individual's health. Several HL assessment tools have been developed to assess skills or screen for inadequate HL [3], [12]. These tools vary in their development, structure, and measurement scope and consequently in their psychometric properties. Most instruments measure selective domains of HL and mainly focus on reading and writing skills; they do not address other crucial skills such as numeracy [3]. Numeracy skills are particularly crucial for diabetes self-management, as these skills facilitate adherence to medication and are required for tasks such as interpreting blood glucose levels and food labels [13], [14]. Newest Vital Sign (NVS) is a brief and easy-to-use assessment instrument to evaluate numeracy skills; it is a six-question tool to assess an individual's ability to find and interpret information (both text and numerical information) on an ice cream nutrition label [15]. Assessment of HL numeracy skills is of considerable importance for diabetes management in clinical practice [9], [13].

The NVS was originally developed in English and Spanish [15] and has recently been validated for use in Japan [16], the Netherlands [17], Turkey [18], and the UK [19]. Cross-cultural adaptation of HL measures is crucial for the development of valid instruments that enable international comparison.

This study assessed the feasibility of the traditional Chinese version of the NVS (NVS-TC). We describe the process of adapting the NVS for use in Taiwan by reporting the translation process and the results of psychometric tests in patients with type 2 diabetes. The main outcome of this study was a translated measurement tool suitable for routine use by researchers and clinicians in a variety of healthcare situations rooted in Chinese culture.

Material and methods

This study was approved by the Institutional Review Board of En Chu Kong Hospital in New Taipei City (ECKIRB110301). A cross-sectional questionnaire survey was administered at the endocrinology division of the hospital. In total, 273 people with type 2 diabetes were enrolled during their routine clinic visits, of which 232 returned completed questionnaires (response rate = 85.9%). Of the 41 questionnaires excluded from analysis, 29 were non-responders (23 cited time constraints and 6 refused), whereas 12 contained missing data.

Development of the traditional Chinese version of the NVS

The NVS was developed for assessing the HL skills of patients in clinical settings [15]. Participants read the information on an ice cream nutrition label and answered six questions; the responses were used to assess participant ability to read and interpret text and numerical information in the label. A psychometric study of the original NVS demonstrated satisfactory reliability (Cranach's α = 0.76) and construct validity (r = 0.59), indicating a significant correlation with more complex literacy measures [15]. Scores vary from 0 to 6, with higher scores indicating higher degrees of literacy. A score of more than 4 indicates adequate literacy skills, whereas a score of less than 4 indicates the possibility of limited literacy.

Translation and cross-cultural adaptation of the NVS-TC were performed in accordance with established guidelines. First, an expert psychologist and a clinical nutritionist independently translated the original NVS into Chinese. Subsequently, an expert committee comprising specialists in psychology, public health, and nutrition evaluated and synthesized the two translations. Use of idiomatic language and readability were particularly considered during the synthesis. As an example of conceptual equivalence, the instruction provided to the subject “this information is on the back of a container of a pint of ice cream” was slightly modified; “a pint” (473.18 ml) was substituted with “half a liter” (500 ml) because the system of imperial units is not commonly used by the general population of Taiwan. The translated version was back translated by two native English translators. The back translated version was compared with the English version. Thirty volunteer participants were recruited for cognitive debriefing to ensure that there were no difficulties in understanding the translated version. The average time to complete the NVS-TC was approximately 5 min. The major differences noted were corrected to ensure direct syntactic and semantic matches. Finally, the NVS-TC was produced and readied for participant administration.

Measures

In addition to the NVS-TC, all interested participants completed several questionnaires, including (1) a questionnaire that captured the demographic characteristics, (2) a nutrition knowledge test specific to diabetes (Diabetes Nutrition Knowledge Test, DNKT), (3) a validated measure of numeracy and reading skills (Nutrition Label Survey, NLS), and (4) a validated subjective HL measure (3 Brief Screening Questions, 3BSQ).

Demographic characteristics

Data on demographic characteristics, such as age, gender, and education level, were collected from participants.

Diabetes nutrition knowledge test

Knowledge of nutrition specific to diabetes was assessed using the 10-item DNKT, an instrument whose reliability and validity has been demonstrated test [20]. Scores were summarized as a continuous variable ranging from 0 (indicating little knowledge) to 10 (indicating in-depth knowledge). The scale demonstrated acceptable internal consistency (Cranach's α = 0.75). Criterion validity was established by comparing scores among different groups of subjects, in which nutritionists had the highest mean scores (9.478 ± 0.745), followed by nurses (7.571 ± 1.414), and patients, who had the lowest mean scores (6.147 ± 2.509) [20].

Nutrition label survey

Numeracy skills were assessed using the NLS, which is based on the national HL survey of Tang et al. [21]. A nutrition label of grape juice with numerical values in units commonly used in Taiwan was shown to the participants. The participants then answered five questions associated with the label; the responses were used to evaluate the participants’ comprehension of nutrition information and calculation of carbohydrate intake. The psychometric properties of this food label test were established using a population-based sample (N = 2086). Validity was established by conducting known group comparison, in which the NLS could significantly differentiate between less and highly educated patients.

3 Brief Screening Questions for health literacy

The 3BSQ is an indirect HL measure in which participants self-report their confidence in HL skills, with scores of 0–4 assigned to each response option [22]. HL levels are determined using the following three screening questions with five response categories: 1) “How often do you have someone help you read hospital materials?” 2) “How confident are you filling out medical forms by yourself?” and 3) “How often do you have problems learning about your medical condition because of difficulty understanding written information?” The 3BSQ has been found to be effective in detecting inadequate HL (area under the receiver operating characteristic curve of 0.87, 0.80, and 0.76, respectively).

Analyses

Psychometric analyses of the NVS-TC included analyzing score distributions (including floor and ceiling effects), estimating internal consistency reliability by using Cronbach's α coefficient, and testing scale validity by using construct and known group comparison approaches. In terms of item description, the percentages of patients with diabetes who achieved either the highest score (ceiling) or the lowest score (floor) were calculated, because a strong ceiling or floor effect limits the ability of the NVS-TC to detect changes over time.

Internal consistency and reliability were estimated using Cranach's α coefficient. Because higher levels of reliability increase the statistical power, a minimum reliability of 0.70 for measures used within-group comparisons is recommended.

Validity was tested using construct-based and known group comparison approaches. Construct validity was assessed using correlation analysis, where the Pearson correlation coefficient r > 0.3 was considered relevant. Construct validity was also tested by comparing the scores of the NVS-TC with those of the NLS, DNKT, and 3BSQ. We anticipated that the NVS-TC scores would have stronger correlations with the NLS and DNKT than with the 3BSQ scores, because NVS-TC, NLS, and DNKT are all objective measures of similar concepts. Furthermore, known group validity was assessed by calculating differences in mean scores between low, intermediate, and high education levels and diabetes knowledge. We hypothesize that if the NVS-TC was a valid instrument, participants with a higher education level would have higher NTS-TC scores than did their counterparts.

Results

Responses and characteristics of participants

Table 1 presents the characteristics of the study population. In this study, 232 valid questionnaires were completed by 128 men (55.2%) and 104 women (44.8%), with a mean age of 58.06 ± 9.49. More than half of the participants (51.3%) had completed secondary education, 90 (38.8%) completed primary or lower education, 51 (22.1%) completed middle school education, and 91 (39.2%) completed high school education or higher. In this study, the mean NVS-TC score was 2.11 (SD = 1.8). The score distribution of the NVS-TC is presented in Fig. 1. In total, 39 (16.8%) participants answered all questions incorrectly (floor effect), and 13 (5.6%) answered all questions correctly (ceiling effect). Among the three NVS categories for the HL level, approximately half of the participants (47.4%) were classified into the limited literacy category (score of 0–1), 67 (28.9%) into the limited literacy possible category (score of 2–3), and 55 (23.7%) into the adequate literacy category (score of 4–6).

Table 1

Distribution of patient characteristics.

Patient Characteristics	N	%
Sex
Women	128	55.2
Men	104	44.8
Age (years)
30–39	6	2.6
40–49	36	15.5
50–59	83	35.8
60–69	78	33.6
≥70	29	12.5
Education level
Low – Primary school	90	38.8
Intermediate – Middle school	51	22.1
High – High school or higher	91	39.2
Health literacy
Limited literacy 0–1	110	47.4
Limited literacy possible 2–3	67	28.9
Adequate 4–6	55	23.7

Fig. 1

Distribution of the NVS-TC scores.

Reliability and validity

Cronbach's α coefficient for the total NVS-TC score was adequate (α = 0.70). The correlation patterns for construct validity confirmed our hypothesis. Table 2 shows that the NVS-TC had the strongest correlation with the NLS (r = 0.58, p < 0.001), followed by that with diabetes nutrition knowledge (r = 0.38, p < 0.001). The NVS-TC showed the least correlation with the 3BSQ (r = 0.18, p < 0.01).

Table 2

Pearson correlations among measures of health literacy and diabetes knowledge.

	Mean (SD)	1	2	3	4
1. NVS-TC	2.11 (1.74)	(0.70)
2. NLS	2.43 (1.92)	0.585***	(0.85)
3. DNKT	6.40 (1.85)	0.383***	0.388***	(0.75)
4. 3BSQ	11.02 (2.14)	0.175**	0.229***	0.237***	(0.87)

Note. NVS-TC = Newest Vital Sign – Traditional Chinese version; NLS = Nutrition Label Survey; DNKT = Diabetes Nutrition Knowledge Test; 3BSQ = 3-Brief Screening Questions. Scale reliability was represented on the diagonal.

*p < 0.05, **p < 0.01, ***p < 0.001.

Known group comparisons confirmed the hypothesis that the NVS-TC could significantly differentiate between patients with low and high education levels (F = 18.15, p < 0.0001). Participants with a high education level exhibited the highest mean NVS-TC score (2.77 ± 1.83), followed by those with intermediate (2.29 ± 1.72) and low education levels (1.33 ± 1.32). Multivariate regression analyses showed that educational differences in the NVS-TC scores remained significant after adjustment for age, gender, and nutrition knowledge [Table 3].

Table 3

Regression analysis with NVS-TC as dependent variable.

Variable	Beta	95%CI	t	p
Educational level (ref Low)
Intermediate	0.601	0.061 to 1.141	2.193	0.029
High	1.011	0.510 to 1.512	3.976	0.000
Gender (ref male)	0.252	−0.166 to 0.671	1.189	0.236
Age	−0.037	−0.060 to −0.014	−3.113	0.002
Nutrition knowledge	0.211	0.092 to 0.330	3.492	0.000

Adjusted proportion of the variance explained by the model: adjusted R2 = 0.259.

Beta denotes the variable's estimated regression coefficient.

Discussion

A crucial objective of the current study is to determine whether the NVS measurement instrument can be applied in Taiwan. In general, the findings of the current study provide evidence that the concepts embodied in the NVS can be conveyed to Taiwanese people and can be feasibly applied in Taiwan. This study demonstrated that the NVS-TC has high reliability and validity. The association patterns found mostly confirmed a priori hypotheses in direction and strength, thereby supporting the construct validity of the NVS-TC. Differences in the NVS-TC scores among participants with low, intermediate, and high education levels showed evidence of known group validity for the NVS-TC. These differences remained significant in the multivariable regression model after adjustment for age, gender, and diabetes nutrition knowledge.

Our results are comparable with international findings regarding the NVS [16], [17], [18], [19]. The mean distribution of the NVS scores varied with participant characteristics. The mean distribution of the NVS-TC (2.11 ± 1.74) was comparable with that of NVS-Netherland (1.8 ± 1.9), as participants had chronic conditions (eg, CAD and T2DM) in both studies, and the mean distribution was much lower than those obtained from general outpatients (eg, NVS-Turkey = 2.6 ± 0.8) or the general population (eg, NVS-UK = 3.47 ± 1.88). Cranach's α of the NVS-TC (α = 0.70) is similar to the coefficients of the NVS-Turkey (α = 0.70) and NVS-Spanish (α = 0.69) and lower than the NVS-Japan (α = 0.72), NVS-UK (α = 0.74), NVS-US (α = 0.76), and NVS-Netherland (α = 0.78).

Construct validity was established by comparing the NVS-TC scores with the objective and subjective HL measures. As expected, the NVS-TC had higher correlations with NLS and nutrition knowledge and lower correlations with SQ. The higher correlation between NVS-TC and NLS was because both questionnaires objectively assess numeracy and reading skills. The correlation between NVS-TC and 3BSQ is weaker because 3BSQ is a self-reported measure for a broader range of HL skills, whereas the NVS specifically assesses the ability to identify and interpret text and numerical information. In addition, the low correlation between the objective NVS-TC and subjective 3BSQ is consistent with that obtained by studies in the field of psychometrics; these studies showed the level of correspondence between objective and subjective measures of HL was usually not high. Regarding known group validity, the results confirmed our hypothesis that the NVC-TC could significantly differentiate among patients with high, intermediate, and low education levels.

HL research is growing in the field of chronic care because emerging evidence has shown the adverse effects of inadequate HL on health care and outcomes. Developing a validated objective measure of HL skills in individuals with chronic conditions is imperative [3]. This study is one of the few on cross-cultural applicability of HL measures. Cross-cultural adaptation of HL measures is not only essential for further development of HL research in Taiwan but also offers a starting point for feasibility evaluation in patients with diabetes. In addition, the translated NVS can be used in research on cross-cultural differences in HL and associated factors.

Although this study illustrates the feasibility of applying the NVS to Taiwan, recognizing its limitations, which are potential research areas, is critical. First, we could not compare the NVS-TC against a standardized HL measure (eg, REALM, TOFHLA, and s-TOFHLA), because no traditional Chinese version of any valid instrument was available when this study was conducted. We therefore used the numeracy skill test, which was validated in a population-based study and had partly overlapping constructs with the NVS. Although this facilitated the establishment of the construct validity of the NVS-TC, further research is required to demonstrate sensitivity and specificity of the NVS-TC for establishing cut-off points of health levels in Taiwan. Second, the research sample in the validation study was recruited from an outpatient department of a hospital. This sample may not be representative of the diabetes population in Taiwan. The effects of HL problems may differ according to context and setting. We therefore recommend that the applicability of the NVS-TC be tested in various studies in public health and health care settings to further investigate its reliability and validity in various contexts and populations.

Validating a scale for cross-cultural adaptability is not a task that can be completed in a short period. The test–retest reliability of the NVS-TC is not fully established in the current study; thus, future assessment of reproducibility and responsiveness to health education intervention is necessary. In addition, the NVS-TC has acceptable psychometric properties of reliability and known group validity, but the predictive validity requires further confirmation. Linking the NVS-TC results to actual outcomes such as a decision or history of action would provide a more reliable assessment of the association between the NVS-TC and healthcare behavior.

Conclusions

This study presents a cross-culturally adapted instrument for assessing objective HL in Taiwan. The results suggest that the NVS-TC is a reliable and valid tool that allows international comparable HL research in Taiwan. The main outcome of this study was a translated measurement tool suitable for routine use by researchers and clinicians in a variety of healthcare situations rooted in Chinese culture.

With accumulating evidence of the association of inadequate HL and health outcomes in people with diabetes [1], measuring HL skills has become imperative. Although the measurement scope of the NVS is limited to aspects of reading and numeracy skills, the NVS has several desirable characteristics as an HL screening tool in clinical settings. One of the most crucial features is that the NVS can rapidly provide the clinician with valuable insight into the underlying issues (eg, struggles to interpret blood glucose levels or food labels) often associated with poor glycemic control [1], [23]. Furthermore, the use of a nutrition label to assess HL is intuitively appealing because nutrition labels are familiar items used for health management in many chronic diseases to achieve healthy eating habits [15]. Because healthcare providers often overestimate the HL of patients [24], [25], [26], routine prescreening of the HL of patients is a must to ascertain the amount of information that patients can comprehend. The NVS is a viable tool for assessing the HL of patients and can be easily incorporated into intervention programs to facilitate patient comprehension of health information in daily clinical practice.

Conflicts of interest

The authors declare that they have no competing interests.