Quick screen of patients' numeracy and document literacy skills: the factor structure of the Newest Vital Sign.

INTRODUCTION: The Newest Vital Sign (NVS) is a survey designed to measure general health literacy whereby an interviewer asks six questions related to information printed on a nutritional label from an ice cream container. It enables researchers to evaluate several health literacy dimensions in a short period of time, including document literacy, comprehension, quantitative literacy (numeracy), application, and evaluation. No study has empirically examined which items belong to which latent dimensions of health literacy in the NVS using factor analysis. Identifying the factor structure of the NVS would enable health care providers to choose appropriate intervention strategies to address patients' health literacy as well as improve their health outcomes accordingly. This study aimed to explore the factor structure of the NVS that is used to assess multiple dimensions of health literacy.
METHODS: A cross-sectional study administering the NVS in a face-to-face manner was conducted at two family medicine clinics in the USA. One hundred and seventy four individuals who participated were at least 20 years old, diagnosed with type 2 diabetes, prescribed at least one oral diabetes medicine, and used English as their primary language. Exploratory factor analysis and confirmatory factor analysis were conducted to investigate the factor structure of the NVS.
RESULTS: Numeracy and document literacy are two dimensions of health literacy that were identified and accounted for 63.05% of the variance in the NVS. Internal consistency (Cronbach's alpha) of the NVS were 0.78 and 0.91 for numeracy and document literacy, respectively.
CONCLUSION: Numeracy and document literacy appropriately represent the factor structure of the NVS and may be used for assessing health literacy in greater detail for patients with type 2 diabetes.

Introduction

Health literacy is defined as the degree to which individuals are able to collect, process, and comprehend basic health information needed to make appropriate health-related decisions.1,2 Compared to individuals with adequate health literacy, patients with inadequate health literacy are more likely to have poorer health outcomes, lower medication adherence, increased hospitalization, higher mortality, and higher health care costs,3,4 probably associated with their difficulty in comprehending health information, following medical instructions, and performing self-care tasks.5 Inadequate health literacy may pose a barrier to diabetes care, as it is prevalent among patients with diabetes and is associated with worse glycemic control.4,6,7 Therefore, valid, reliable, and convenient health literacy measurement instruments could provide insight into the specific barriers that hinder patients’ understanding of health information, in order to help health care professionals tailor interventions to improve health outcomes.

Recently, Sørensen et al reviewed past empirical research and concluded that there were eleven unique dimensions of health literacy.8 To date, no single instrument has been developed to assess all dimensions of health literacy. Currently available health literacy instruments put emphasis on different dimensions of health literacy. Consequently, to obtain critical information on patients’ health literacy, it is important to administer appropriate instruments that evaluate specific dimensions of health literacy that may be amenable to change with a focused intervention. Health care providers are able to tailor information based on individuals’ levels of health literacy and improve individuals’ health literacy to facilitate their self-care skills.

One frequently used health literacy instrument is the Newest Vital Sign (NVS), a 6-item survey developed to measure general health literacy among a variety of patients.1,9 The NVS enables researchers to evaluate several health literacy dimensions in a short period of time, including document literacy, comprehension, quantitative literacy (numeracy),10 application, and evaluation.1 Document literacy is defined as the ability to search, comprehend, and use non-continuous texts in various formats; quantitative literacy is the ability to identify and perform computations, either alone or sequentially, using numbers embedded in printed materials.11 Numeracy is similar to quantitative literacy, but not exactly the same,10 but numeracy is used more often in other published literature and will be used interchangeably with the phrase “quantitative literacy” in this article.

The first four NVS questions assess all of the health literacy dimensions as previously mentioned, but have a strong quantitative literacy dimension. However, the last two questions only have document literacy, comprehension, application, and evaluation dimensions. Unfortunately, the National Assessment of Adult Literacy, the last time that quantitative and document literacy were assessed in USA adults, it was reported that 34% have significant trouble understanding text documents (basic and below basic document literacy), but 55% have significant trouble understanding text with numbers (basic and below basic quantitative literacy).12 This suggests that the first four and last two NVS questions may measure different dimensions and might be useful to help guide health literacy improvement interventions.

To our knowledge, no study has empirically examined which items belong to which dimensions of health literacy in the NVS by using factor analysis, a useful statistical tool that examines underlying data structure of psychometric instruments.13 This study aimed to explore the factor structure of the NVS and assess the validity of using separate components’ results.

Methods

Design

In this cross-sectional study, the researchers administered a face-to-face questionnaire at two family medicine clinics in the USA from March to August 2016. The Health Sciences Institutional Review Board at the University of Wisconsin-Madison approved the study procedures (2015-1284). All participants provided written informed consent for this study.

Sample and procedure

A list of potential patients was gathered from the electronic medical record database at two family medicine clinics. Study participants were eligible if they were 20 years of age or older, diagnosed with type 2 diabetes, presently prescribed to take at least one diabetes medicine by mouth daily, and able to understand English. All eligible individuals on the patient lists scheduled for an appointment at either family medicine clinic were invited. After the patients consented to participate, in a private area in the clinic, one researcher read out the items in the health literacy instrument and the researcher recorded the responses on the questionnaire sheet. Participants were compensated US$25 cash after completing the survey.

Measurement

A structured face-to-face survey included the complete 6-item NVS survey, in addition to participants’ sociodemographic background and clinical characteristics.

Health literacy

The 6-item NVS was used to measure the health literacy of study participants.1,9,14 This validated instrument has been extensively used across different studies on various diseases.15–18 Since the original NVS was only validated and tested using a face-to-face administration, the same face-to-face approach was adopted in the present study, which took an interviewee 3–5 minutes to complete.1,18 In the NVS, an interviewer asks six questions pertaining to the information printed on a nutritional label from an ice cream container.1,9,19 Respondents are required to do mental calculations for items 1 to 4 and answer items 5 to 6 based on their reading ability and reasoning. They have to answer the questions in sequence and they are unable to answer item 6 if they provide wrong answers to item 5.9 Each question is scored “0” for incorrect and “1” for correct answer, yielding a total score ranging from 0 to 6, with higher scores indicating better health literacy.1,9 A score of lower than 2 suggests a high likelihood of limited health literacy, 2 to 3 represents a possibility of marginal health literacy, and higher than 3 indicates adequate health literacy.1,9,20

Sociodemographic background and clinical characteristics

The sociodemographic background information collected were age, gender, race, highest education level, whether the participant has health insurance, and annual household income. Clinical characteristics included self-reported health status, number of medications used, number of chronic illnesses, frequency of daily diabetes medication use, duration of diagnosis of diabetes mellitus, and whether the participant used insulin for diabetes control or not.

Statistical analysis

Descriptive statistics were used to summarize the characteristics of the study participants. The mean and the SD were provided for all continuous variables, and the count and percentage for categorical variables. Since previous literature did not clearly identify which items belong to which latent dimensions of health literacy, both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to analyze the factor structure of the NVS. Prior researchers suggested that the adequate sample size for factor analysis is ten cases per variable, and a sample size between 50 and 100 is sufficient to evaluate psychometric properties of instruments of social constructs.21,22 The participants were randomly assigned to two groups with equal sample size. One group was used in EFA, and the other in CFA. In EFA, the principal axis analysis method with direct oblimin rotation was conducted to analyze the factor structure of the NVS.23,24 In the first step of EFA, we examined the correlation matrix of all six items and calculated the mean inter-item correlation to identify the strength of association between the items. The eigenvalue-greater-than-one criterion and the scree plot were investigated to determine the number of factors. The Kaiser-Meyer-Olkin (KMO) statistic and Bartlett’s test of sphericity were both examined to determine if the assumptions of EFA were met.25 The component matrix after direct oblimin rotation was used to determine whether the items were grouped and loaded onto latent factors in a conceptually sound manner. Items with a factor loading of 0.40 or greater were considered adequately measuring a factor and items with factor loadings of 0.40 or greater on two factors were considered cross-loaders and would be omitted in the CFA.26 The results of the EFA (eg, the number of latent factors, the grouping of items underneath each latent factor) informed the verification of the distinctiveness of each factor using CFA.27 Both EFA and CFA models were estimated by using maximum likelihood estimation, and a number of fit indices were adopted to examine if the models fit the data.26 The assessment of model fit in CFA relied on the comparative fit index (CFI), Tucker-Lewis index (TLI), root mean error of approximation (RMSEA), and standardized root mean square residual (SRMR). The CFI and TLI values above 0.95, and RMSEA and SRMR below 0.05 as well, indicated acceptable model fit.26 The internal consistency of each NVS dimension was evaluated using Cronbach’s alpha. Descriptive statistics, Cronbach’s alpha, and EFA were carried out in SPSS version 23, while CFA was performed in R version 3.3.1 with lavaan package with the statistical significance level at a two-sided p < 0.05.27

Results

A total of 199 individuals were approached; 174 (87.4%) participants were enrolled and completed the survey. The majority (n = 100, 57.5%) of the participants were female with ages ranging from 26 to 92 years old (mean age = 58.7, SD = 12.8). One hundred and eighteen participants (67.8%) were White and 43 (24.7%) were Black. A majority of the participants (n = 108, 62.1%) reported having received some college education or above (Table 1).

Table 1

Descriptive statistics of the participants, separated by samples analyzed in EFA and CFA

Variables	Total (n = 174)		EFA (n = 87)		CFA (n = 87)
	n (%)	Mean (SD)	n (%)	Mean (SD)	n (%)	Mean (SD)
Age		58.7 (12.8)		61.2 (11.0)		56.2 (14.0)
Gender
Female	100 (57.5)		51 (58.6)		49 (56.3)
Male	74 (42.5)		36 (41.4)		38 (43.7)
Race
White	118 (67.8)		62 (71.3)		56 (64.4)
Black	43 (24.7)		18 (20.7)		25 (28.7)
Other	13 (7.5)		7 (8.0)		6 (6.9)
Education
High school or less	66 (37.9)		31 (35.6)		35 (40.2)
Greater than high school	108 (62.1)		56 (64.4)		52 (59.8)
Health literacy		3.7 (2.0)		3.7 (2.0)		3.7 (2.0)
Limited (score = 0–1)	31 (17.8)		17 (19.5)		14 (16.1)
Marginal (score = 2–3)	39 (22.4)		18 (20.7)		21 (24.1)
Adequate (score = 4–6)	104 (59.8)		52 (59.8)		52 (59.8)
Numeracy (full score = 4)		2.3 (1.5)		2.2 (1.6)		2.3 (1.5)
Document literacy (full score = 2)		1.5 (0.8)		1.5 (0.8)		1.4 (0.9)

Abbreviations: EFA, exploratory factor analysis; CFA, confirmatory factor analysis.

The mean NVS score of all participants was 3.7±2.0. One hundred and four (59.8%) participants had adequate health literacy; however, there were 31 (17.8%) and 39 (22.4%) participants with limited and marginal health literacy, respectively (Table 1). The Cronbach’s alpha coefficients of numeracy and document literacy were 0.78 and 0.91, respectively, suggesting moderate to high reliability, and the Cronbach’s alpha would decrease if any item was omitted (Table 2). The inter-item correlation ranged from 0.26 to 0.84 (Table 3).

Table 2

Reliability test of the NVS (n = 174)

Item	Question	Mean (SD)	Corrected item-total correlation	Cronbach’s alpha if item deleted
Item 1	If you eat the entire container, how many calories will you eat?	0.70 (0.46)	0.59	0.77
Item 2	If you are allowed to eat 60 g of carbohydrates as a snack, how much ice cream could you have?	0.61 (0.49)	0.64	0.76
Item 3	Your doctor advises you to reduce the amount of saturated fat in your diet. You usually have 42 g of saturated fat each day, which includes one serving of ice cream. If you stop eating ice cream, how many grams of saturated fat would you be consuming each day?	0.48 (0.50)	0.54	0.78
Item 4	If you usually eat 2,500 calories in a day, what percentage of your daily value of calories will you be eating if you eat one serving?	0.47 (0.50)	0.51	0.79
Item 5	Is it safe for you to eat this ice cream?	0.77 (0.42)	0.49	0.79
Item 6	(Ask only if the patient responds “no” to question 5): Why not?	0.70 (0.46)	0.62	0.76

Note: Cronbach’s alpha was 0.80 for the total scale.

Abbreviation: NVS, Newest Vital Sign.

Table 3

Inter-item correlations of the NVS (n = 174)

Item	Item 1	Item 2	Item 3	Item 4	Item 5	Item 6
Item 1	1.00
Item 2	0.57	1.00
Item 3	0.43	0.48	1.00
Item 4	0.43	0.52	0.41	1.00
Item 5	0.26	0.26	0.28	0.18	1.00
Item 6	0.41	0.40	0.35	0.29	0.84	1.00

Notes: Cell entries are Pearson’s correlation coefficient. Refer to Table 2 for the contents of the questions from item 1 to item 6.

Abbreviation: NVS, Newest Vital Sign.

In the EFA, the KMO statistic was 0.749 and Bartlett’s test of sphericity was significant (χ2 = 228.413, df = 15, p < 0.001), indicating that assumptions of EFA were met. Following the eigenvalue-greater-than-one rule, EFA revealed that the two-factor solution should be chosen, with 63.05% of the total variance of the NVS collectively represented by the two factors (Table 4 shows the factor loadings after oblique rotation). Item 1–4 loaded onto factor 1, and item 5 and 6 onto factor 2. These two factors represented two clear, distinctive dimensions identified as the numeracy component (factor 1) and the document literacy component of health literacy (factor 2). The CFA was conducted to validate the factor structure of the NVS. The results indicated that a two-factor solution fit the data the best (χ2 = 9.479, df = 9, p = 0.393, CFI = 0.998, TLI = 0.996, SRMR = 0.041, RMSEA = 0.025), and the factor loadings ranged between 0.575 and 0.938. The correlation coefficient between the two factors was 0.46. The factor structures of the NVS remained the same in EFA or CFA regardless of participants’ gender.

Table 4

Component matrix after oblique rotation in EFA (n = 87)

Item	Question	Factor 1 rotated component loading	Factor 2 rotated component loading
Item 1	If you eat the entire container, how many calories will you eat?	0.666	−0.211
Item 2	If you are allowed to eat 60 g of carbohydrates as a snack, how much ice cream could you have?	0.769	−0.042
Item 3	Your doctor advises you to reduce the amount of saturated fat in your diet. You usually have 42 g of saturated fat each day, which includes one serving of ice cream. If you stop eating ice cream, how many grams of saturated fat would you be consuming each day?	0.605	−0.156
Item 4	If you usually eat 2,500 calories in a day, what percentage of your daily value of calories will you be eating if you eat one serving?	0.725	0.176
Item 5	Is it safe for you to eat this ice cream?	−0.065	−0.907
Item 6	(Ask only if the patient responds “no” to question 5): Why not?	0.191	−0.843

Notes: Factor 1 is identified as the numeracy component of health literacy. Factor 2 is identified as the document literacy component of health literacy.

Abbreviation: EFA, exploratory factor analysis.

Discussion

To our knowledge, this is the first study to explore the factor structure of the NVS, showing that the NVS assesses two distinct aspects of health literacy. Among the six questions in the NVS, the last two questions assess document literacy, comprehension, application, and evaluation; the first four items also add a strong quantitative literacy assessment. This is a particularly important distinction as the NVS instructions state “if a patient is still struggling with the 1st or 2nd question after 2–3 minutes, the likelihood is that the patient has limited literacy and you can stop the assessment.”9 All questions must be asked if the NVS is to be used to assess both distinct health literacy aspects. It also suggests that a person with low numeracy skills, but higher document literacy skills will inappropriately score lower and appear to have less document literacy, comprehension, application, and evaluation skills.

The NVS is arguably a more comprehensive assessment of health literacy, compared with The Rapid Estimate of Adult Literacy in Medicine (REALM) and the Short Test of Functional Health Literacy in Adults (S-TOFHLA), the two most commonly used assessment tools for general health literacy.14,20,28 On one hand, the REALM and the S-TOFHLA both measure one single dimension of health literacy, focusing mainly on reading and comprehension skills without addressing other relevant skills to diabetes management, such as numeracy.28 Numeracy (quantitative literacy) is one of the pivotal dimensions of health literacy for chronic diseases such as asthma, hypertension, and diabetes care (eg, adjusting diabetes medication dosage based on the interpretation of the number displayed on a glucose meter). Health literacy measures that assess numeracy, such as the NVS, should be consistently used.14,18 On the other hand, while some existing health literacy instruments do measure numeracy skills (eg, the Diabetes Numeracy Test and the Diabetes Numeracy Test-15 for patients with diabetes,5,29 the Asthma Numeracy Questionnaire for patients with asthma,30 and the High Blood Pressure-Health Literacy Scale for patients with hypertension),31 they may not be widely used across studies or in clinical care due to a longer administration time and a lack of validity.14 The NVS thus has the potential to provide a more comprehensive assessment of health literacy and to be used in assessing populations of various chronic disease patients, due to its ease of assessment and administration in clinical practice. As a result, researchers may better understand which dimensions of health literacy could influence patients’ health behaviors by dividing the NVS into different dimensions for analysis. The differentiation would also provide more insights for researchers to integrate the relevant dimensions of health literacy into intervention study designs to improve patients’ health outcomes. Health care professionals may also assess patients’ health literacy in clinical practice with the NVS in a short time.

Current recommended strategies to improve patients’ comprehension of health information range from introducing and revisiting certain key points of self-care at each patient visit through jargon-free communication, using pictures to clarify medication use, to confirming patients’ knowledge of health information via the show-me or teach-back method.32 Recent literature indicates that the strategies for improving patients’ numeracy skills include matching words and numbers to demonstrate a patient’s understanding of numbers, using pictorial images and shapes to visualize the numbers, and using teach-back techniques.33,34 Instead of teaching patients how to perform multi-step math, White et al used the iconographic rotating disk (eg, InsuCalc Wheel) to help patients estimate carbohydrate intakes and monitor blood glucose by simplifying multi-step calculation.35 There-fore, health care professionals could integrate these strategies in tailored interventions to address patients’ health literacy when limited numeracy skills are identified as a barrier to understanding and appropriate actions. Health care providers may aim to help patients understand the meaning of the numbers rather than increase patients’ ability to calculate the numerical operations alone.

This study was part of a larger research project that focused on the medication adherence of patients with type 2 diabetes.36 A limitation of the study is the unknown generalizability of the results across other diseases. We only included a sample of patients with type 2 diabetes in a Midwest location. Our sample may not be representative of patients with other diseases in other geographic locations. Accordingly, studies that are more extensive are needed to validate the findings by involving a diverse population across different races or with other diseases (eg, asthma and hypertension).

Conclusion

Numeracy and document literacy are two health literacy dimensions that are identified to appropriately represent the factor structure of the NVS, and can be used for assessment of health literacy skills in greater detail. The NVS provides an easy and quick screen to evaluate patients’ ability to comprehend and perform basic numerical operations based on printed health information. Health care providers may tailor interventions accordingly to address patients’ health literacy in terms of information comprehension and numerical operation.