Health Literacy Estimation of English and Spanish Language Caregivers.

BACKGROUND: Provider ability to estimate caregiver health literacy (HL) in English-speaking caregivers has been shown to be poor, but estimation of HL in Spanish-speaking caregivers by physicians and staff has yet to be studied. Linguistic differences can further hinder communication in medical care.
OBJECTIVE: This study evaluated how well pediatric providers and staff predict caregiver HL as measured by two HL tools in a bilingual (English/Spanish) population.
METHOD: For this study, we obtained a convenience sample of caregivers, evaluating one group with the Newest Vital Sign (NVS) and the second group with the Short Assessment of Health Literacy (SAHL). Physicians/nurse practitioners (NPs), and medical assistants (MAs) estimated caregiver scores for each tool. We dichotomized estimated and actual scores for each tool using published standards. We used McNemar's test and Cohen's Kappa to evaluate agreement between dichotomized predicted and actual scores. We used log binomial regression to examine how caregiver's language affected agreement between dichotomized caregiver scores and provider estimates. All physicians/NPs were native English speakers only and all MAs were native bilingual English/Spanish speakers. Physicians/NPs used interpretation services when appropriate. KEY
RESULTS: Fifty caregivers were evaluated using the NVS and 50 using the SAHL. There was no overall association between dichotomized physician/NP or MA estimation and caregiver score for either tool. However, providers' estimates were less likely to match caregiver scores when the caregiver's language was Spanish (NVS: relative risk [RR] = 0.57 [95% CI 0.37, 0.87], SAHL: RR = 0.37 [95% CI 0.23,0.6]).
CONCLUSION: Physician/NP and MA ability to estimate caregiver HL in English proficient and limited English proficiency caregivers is poor. The physician/NP group was less likely to estimate HL correctly if the caregivers spoke Spanish. Providers must use additional caution when providing cross-language care. [HLRP: Health Literacy Research and Practice. 2018;2(2):e107-e114.]. PLAIN LANGUAGE
SUMMARY: This study examined estimation of health literacy of English- and Spanish-speaking caregivers by pediatric providers and medical assistants. We found that both providers and staff estimate caregiver health literacy poorly, and that primary language discordance may be a factor. The results support the institution of universal health literacy precautions for all caregivers of pediatric patients.

In the United States, Andrulis and Brach (2007) calculated that approximately 40% of adults have limited health literacy, and that over one-half of them are nonwhite and/or Hispanic. Brice et al. (2008) found low health literacy levels in Spanish-speaking patients in an emergency department, and Alba, Britigan, Lyden, and Johansson (2016) demonstrated similar results in Spanish-speaking patients in federally qualified health centers across the Midwestern U.S. Functional health literacy has been described as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions” (Kindig, Panzer, & Nielsen-Bohlman, 2004). This definition is undergoing continual revision as research in the field advances and as researchers move away from a “deficit model” toward a realization that health literacy is a social construct that is variable and interdependent on social, economic, demographic, and other unknown factors (Pleasant, 2014). Acknowledging this ambiguity in measurement, definition, and etiology, the fact remains that based on current measures, low health literacy presents challenges in quality care delivery, which can be further exacerbated by language.

Low health literacy affects communication with the health care team and patient/family understanding of their medical instructions and illness. Studies have linked low patient health literacy with poor health outcomes across a variety of disease processes and populations (Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011). Unsurprisingly, parental health literacy has also been linked to health outcomes in the pediatric population (Brega et al., 2016; DeWalt, Dilling, Rosenthal, & Pignone, 2007; Miller, Lee, DeWalt, & Vann, 2010). Additionally, caregivers with low health literacy levels have been reported to perceive more difficulties in accessing care for their children and perceive a lower level of shared decision-making (Yin, Dreyer, et al., 2012). Ongoing research continues to evaluate caregiver health literacy for multiple diseases (Cooper, Chisolm, & McLeod, 2017; Dingemans et al., 2017).

Health literacy is a multifaceted concept that is difficult to define and measure, comprised of different domains and skills such as print literacy, oral literacy, and numeracy (Berkman, Sheridan, Donahue, Halpern, Viera, et al., 2011; Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association, 1999). There can be a significant discordance in an individual patient or caregiver's aptitude across domains. For example, an in-depth examination found that among Spanish-speaking parents in four centers, scores on the Short Test of Functional Health Literacy in Adults (a test focusing on language) were adequate, but mathematical skills measured by the Arithmetic Wide Range Achievement Test and functional skills measured by the Parental Health Literacy Activities Test were suboptimal, especially in medication dosing (Yin, Sanders, et al., 2012). A discordance in abilities could complicate the health care team's estimate of a patient's ability to follow instructions or understand disease processes.

Multiple studies have found that physicians overestimate their patients' literacy levels (Bass, Wilson, Griffith, & Barnett, 2002; Lindau et al., 2002). Kelly and Haidet (2007) showed that physicians consistently overestimate adult patients' health literacy skills. Dickens, Lambert, Cromwell, and Piano (2013) showed a similar finding in nurses, and Bass et al. (2002) showed that resident physicians' ability to estimate literacy skills was poor. Potential discordance between a provider's estimation of caregiver health literacy and that caregiver's measured health literacy has also been demonstrated in caregivers of the pediatric population. A 2013 study revealed that provider estimates of English-speaking caregivers' health literacy correlated poorly with the Test of Functional Health Literacy in Adults and the Rapid Estimate of Adult Literacy in Medicine scores, and a 2012 survey indicated that physician assessment of which reading materials were appropriate for caregivers did not match general population literacy levels (Chesser et al., 2012; Harrington, Haven, Bailey, & Gerald, 2013). Critically, these studies did not examine staff estimation of caregiver health literacy or health literacy estimation of Spanish-speaking caregivers.

Medical assistants (MAs) are an essential part of the medical team at most pediatric practices. They keep patients moving through the clinic and ensure that providers have the information and tools they need. As one of the first points of contact for patients and their families, they often have a deep understanding of patients' personal, social, and medical histories. In many practices, MAs are taking on a larger role with counseling families, discussing medication and chronic disease management, or serving as health coaches (Chapman & Blash, 2017; Djuric et al., 2017; Ferrer, Mody-Bailey, Jaen, Gott, & Araujo, 2009). Additionally, in a clinic with multiple physicians, MAs may serve as a key source of continuity for patients and families. MA estimation of caregiver's health literacy in their role as part of the health care team has not been studied.

The need for health care professionals to accurately estimate such a complex and multifaceted construct such as parental health literacy can be mitigated somewhat by the institution of universal health literacy precautions. However, the health care team must still assess caregiver ability to follow instructions and understand disease so that we can identify ways in which we can improve in our role in the health literacy system by minimizing our own participation in barriers to care and health disparities. In this study, we sought to determine how well members of the health care team, providers (physicians/nurse practitioners [NPs]), and MAs in our pediatric clinic estimated the health literacy levels of Spanish-speaking and English-speaking pediatric caregivers as measured by two tools determining different domains of health literacy—print literacy and numeracy.

Methods

Sample/Data Collection

This endeavor was part of a larger study examining the relationship between health literacy tools and obesity and immunization completion. The original research plan was to perform the study using only the Newest Vital Sign (NVS), which focuses on comprehension and numeracy (Weiss et al., 2005). As the study progressed, the investigators decided to add an additional group using the Short Assessment of Health Literacy—Spanish and English (SAHL S&E). This tool measures different skills in the caregivers by focusing on reading ability (Lee, Stucky, Lee, Rozier, & Bender, 2010).

The enrollment goal was 100 caregivers, 50 for each of the two health literacy tools. No formal sample size calculation was performed. Enrollment and assessment for the group weighed with the NVS was performed in the spring and summer of 2016. Enrollment and assessment of the group measured with the SAHL S&E was subsequently performed in the summer and fall of 2016. We obtained convenience samples of caregivers who accompanied pediatric patients to a satellite clinic of an academic medical center in the Midwest. The satellite clinic serves a majority Spanish-speaking patient population who are low income. During the time that caregivers were waiting with their pediatric patient for a scheduled appointment, bilingual research assistants offered them the opportunity to enroll in the study. They obtained verbal consent per the Institutional Review Board approved protocol (University of Oklahoma Health Sciences Center, IRB #5902).

Research assistants reviewed the patient's chart and extracted the number of recent clinic visits and number of recent clinic visits with the specific provider (physician/NP) whom they were seeing that day. Research assistants administered either the NVS or the SAHL S&E, then asked both the treating MA and the physician/NP to estimate the score on the health literacy tool that the caregiver had taken. MAs and physicians/NPs also gave informed consent to participate in the study. Investigators showed them the tools ahead of time and instructed them on each tool's administration and scoring. MAs and physicians/NPs were blinded with respect to caregiver scores on the tools. All MAs were native Spanish speakers and bilingual in English and Spanish. Physicians/NPs either used an interpreter or spoke Spanish, but none were native Spanish speakers. Spanish language proficiency was determined either by a passing score on the ALTA Spanish examination or by direct evaluation by bilingual clinic staff (ALTA Language Services, 2003).

Outcome Measures

Primary outcome measures were caregiver scores and provider and MA estimations on each tool. We dichotomized scores and estimations to adequate/less than adequate denoting adequate health literacy using published standards (adequate >3 for the NVS and adequate >14 for the SAHL S&E) (Lee et al., 2010; Weiss et al., 2005).

We recognized that the tool published on the Agency for Healthcare Research and Quality (2016) website contained spelling and translation errors (“harmonía” should be “armonía” and “abnormal” should be “anormal”). We corrected these errors and evaluated the caregivers using the corrected tool.

Covariates

Covariates examined for the study included language used by the caregiver, use of an interpreter by the provider, and provider familiarity with the patient/caregiver (both by provider self-report and number of recent visits). We did not collect any further specific demographic information on caregivers in our population as investigators felt this may have negatively affected participation due to fears of potential legal ramifications in relation to immigration status (Sheehan et al., 2016).

Statistical Analysis

All data analysis was performed with SAS 9.4 software. Fisher's exact test was used to compare the proportions of caregivers assessed with each tool whose scores exceeded published cutoffs. For the analysis of estimation accuracy, we stratified the data by estimator type (physician/NP or MA) and by tool (NVS or SAHL S&E) for analysis. We used McNemar's test and Cohen's Kappa to evaluate for agreement between the caregivers' actual dichotomized scores and the providers' dichotomized estimations. Using separate log binomial regression models, we assessed how the probability of an accurate estimation by the provider was affected by the language of the caregiver, whether or not the provider used an interpreter or whether or not the provider was familiar with the family.

Results

We asked 88 caregivers to take the NVS tool. Thirty-seven declined and one was excluded due to being outside of the age range for enrollment. Thus, we analyzed responses on the NVS for a total of 50 caregivers (57% participation). We asked 77 caregivers to take the SAHL S&E, of which 27 declined. Thus, we analyzed responses on the SAHL S&E for a total of 50 caregivers (65% participation). See Table for details regarding the language of participants and results of the NVS and SAHL S&E by language group. We did not collect specific demographic information on the caregivers, but the population in the clinic is approximately 80% Spanish speaking, with approximately 75% of those indicating Mexico as their country of origin.

The percentage of caregivers with relatively high health literacy scores differed significantly between the two tools. Only 20% [95% confidence interval (CI) 8.5, 31.5] of the caregivers who took the NVS measured as having adequate health literacy (score >3), but 84% [95% CI 73.5, 94.5] who took the SAHL S&E measured as having adequate health literacy (score >14), (p = .0001). In our sample, English speakers performed better than Spanish speakers on the NVS (p = .0007), and the performance of the two language groups on the SAHL S&E was roughly equal (p = .3785).

Primary Outcome: Literacy Estimation

Participating providers included nine attending pediatricians, two pediatric residents, one NP, and five MAs. None of the physicians/NP were native Spanish speakers, but all of the MAs were bilingual in English and Spanish from childhood.

Both physicians/NP and MAs underestimated scores of the SAHL S&E and overestimated scores of the NVS. Cohen's Kappa scores for physician/NP and MA estimation for both tools was < 0.2. Physicians/NP correctly estimated a caregiver's dichotomized score for the NVS 56% of the time [95% CI 41.7, 70.2] and MAs gave an accurate estimation 46% of the time [95% CI 31.7, 60.3]. Physicians/NP correctly estimated a caregiver's dichotomized score for the SAHL S&E 48% of the time [95% CI 33.7, 62.3] and MAs also accurately predicted dichotomized score 48% of the time [95% CI 33.7, 62.3]. To show the magnitude and direction of misestimation on each tool, patient scores and physician/NP estimated scores are plotted for both tools in Figure . Points in the lower left and upper right quadrants represent agreement of dichotomized (adequate/less than adequate) scores. Plots of patient scores and MA estimates were similar (not included).

Language of the patient made a substantial difference in accuracy of score predictions by physicians/NP (Table ), but not in predictions by MAs. Log binomial regression revealed that physicians/NP were more likely to incorrectly estimate dichotomized score on either tool if the caregiver was speaking Spanish rather than English. This effect was statistically significant for both tools when an interpreter was not used, as well as for the SAHL S&E when an interpreter was used. The negative association was present but did not reach statistical significance when an interpreter was used in the NVS group. Estimation success of physicians/NP using an interpreter for Spanish-speaking caregivers did not significantly differ from those that communicated in Spanish without an interpreter (relative risk [RR] with an interpreter versus without an interpreter: NVS RR = 1.74 [95% CI 0.87, 3.52], SAHL RR = 0.68 [95% CI 0.26, 1.82]). Log binomial regression revealed a slight estimation advantage on the SAHL for MAs when they reported knowing the patient somewhat well or very well versus not well (RR = 1.71 [95% CI 1, 2.9]). No other association between accuracy of estimation and either provider familiarity with the family, the number of visits to the clinic, or the number of visits with the specific provider was detected.

Discussion

The primary aim of this study was to determine the accuracy of physician/NP and MA estimations of caregiver health literacy with the goal of improving effective health care team participation in health literacy cocreation. Our results showed that overall, physicians/NP and MAs performed poorly in their estimations of caregiver performance, irrespective of tool used. This suboptimal performance of the estimators is consistent with other published studies examining physician estimation of adult health literacy in patients and caregivers (Bass et al., 2002; Harrington et al., 2013; Kelly & Haidet, 2007). However, our data confirm a similar pattern with MAs, and more importantly, demonstrate that primary language discordance may play a significant role, even when adequate language skills or interpretation are used. These potentially inaccurate estimates can lead the health care team to give instructions to caregivers that are difficult to understand and follow.

Physicians/NP were significantly worse at estimating the health literacy scores of Spanish speakers than that of English speakers. This is consistent with findings showing that patients who have limited English proficiency have more communication barriers in a medical encounter (Wilson, Chen, Grumbach, Wang, & Fernandez, 2005). This discrepancy in estimation was present in both tools, each measuring different domains of the construct. Furthermore, the performance of the caregivers on the two tools differed significantly, and the performance of each language group on each tool was also different (English speakers performed much better than Spanish speakers on the NVS, but this was not the case on the SAHL S&E). Additionally, the effect was present despite the universal use of adequate interpretation services or documented language concordant physicians (although none of the physicians/NP were native Spanish speakers). The maintenance of the consistent pattern in difference in estimation accuracy by language despite varying tools and caregiver performance seems to indicate that discordant primary language between the provider and caregiver is the primary reason for the difference. This conclusion is further supported by the fact that the estimation difference by language was not present for the MAs, who were all primary Spanish/English bilingual. Although their estimations were uniformly poor, no difference existed between the language groups.

The poor estimation skills of the MAs are a finding that has not been previously reported in the literature. This is especially important in pediatrics, where MAs often are in the position to provide a significant portion of face-to-face counselling, continuity of care, and anticipatory guidance depending on office setup.

Prevalence of adequate caregiver health literacy scores differed significantly between the two measurement tools. This finding is consistent with previously published work by Singh, Coyne, and Wallace (2015) and likely reflects that the NVS and SAHL S&E measure different domains of health literacy, with the NVS more focused on numeracy.

Study Limitations

Measurement ambiguity was the source of the most significant limitation of our study, as using the health literacy tool as a proxy for a complex multidomain concept leaves room for misinterpretation and overgeneralization. Also, future studies would benefit from simultaneous measurement of the same caregiver with multiple health literacy assessment tools as well as evaluation of understanding and completion of targeted tasks essential to the care of the child to cross check the proxy measures. Estimation of task completion would be a better measure of the health care team and caregiver's face-to-face interaction creating a shared understanding of the relevant information. Although our caregivers were all from the same population, we administered the tools on separate groups. In light of these multiple limitations, the consistency of our finding that estimation efficacy is strongly associated with primary language concordance between provider and caregiver was even more notable. Despite the measurement ambiguity inherent to health literacy and the separate testing groups, the finding of a negative association between discordant primary language and estimation ability was preserved.

Conclusions

The inaccuracy of health care team estimation of caregiver health literacy demonstrated in our study strongly supports the institution of universal health literacy precautions for caregivers of pediatric patients based on population estimates of health literacy. Efforts such as Healthy People 2020 should be supported, and their implementation studied further (Office of Disease Prevention and Health Promotion, 2016). Processes such as universal teaching and Teach-Back seem both reasonable and essential for responsible clinical care. Critically, the health care team should be especially cautious when their native language differs from that of the caregiver, even when they have proficiency in the caregiver's language or adequate interpretation services.

Table 1

Tool, Language, and Screen Results for Caregivers (N = 100)

	Newest Vital Sign (n = 50)
Result	Spanish	English
Adequate	3 (7.9%)	7 (58.3%)
Less than adequate	35 (92.1%)	5 (41.7%)
	Short Assessment of Health Literacy—Spanish & English (n = 50)
Adequate	33 (86.8%)	9 (75%)
Less than adequate	5 (13.2%)	3 (25%)

Table 2

Health Literacy Estimation Accuracy by Caregiver Language and Provider Interpreter Usage (Physician/Nurse Practitioner Group)

	Newest Vital Sign
Language	Inaccurate	Accurate	Relative Risk Ratios [95% CI]
Spanish (no interpreter)	13	7	0.42[a] [0.22, 0.8]	0.57[a] [0.37, 0.87]
Spanish (with interpreter)	7	11	0.73 [0.46, 1.14]
English	2	10	ref	ref
	Short Assessment of Health Literacy—Spanish & English
Spanish (no interpreter)	14	9	0.43[a] [0.25, 0.73]	0.37[a] [0.23, 0.6]
Spanish (with interpreter)	11	4	0.29[a] [0.12, 0.69]
English	1	11	ref	ref

Note. No significant difference between Spanish groups (with/without interpreter) for either tool. Accuracy was defined as having a matched adequate/less than adequate score between the estimator and caregiver (adequate for Newest Vital Sign corresponds to >3 responses correct, adequate for Short Assessment of Health Literacy corresponds to >14 responses correct).

CI = confidence interval; ref = reference.

Significantly different from the reference (English).