Recommendations for Measurement of Child Health Literacy: A Pragmatic Approach.

Health literacy (HL) has been well-studied in adults; however, the literature on HL in children and adolescents has only recently burgeoned. Many authors argue child and adolescent HL is unique from adult HL and should be independently measured (Bröder et al., 2017; Guo et al., 2018; Okan et al., 2018; Ormshaw, Paakkari, & Kannas, 2013; Rothman et al., 2009; Velardo & Drummond, 2017; Wharf Higgins, Begoray, & MacDonald, 2009); however, there is a lack of consensus on the best framework and practice for this measurement. Although many current HL measures are validated for use among a broad range of children and adolescents (age 7–18 years) (Guo et al., 2018; Manganello et al., 2017; Okan et al., 2018; Ormshaw et al., 2013), these measures are insufficient.

First, as many of the broadly used measures were adapted from adult instruments, they are designed to describe HL as a static correlate to health outcomes, as it is often defined for adults (Nutbeam, 2008). For children and adolescents, many authors argue that HL should be defined as a dynamic asset to be leveraged to improve health behaviors and outcomes (Bröder et al., 2017; Guo et al., 2018; Okan et al., 2018; Velardo & Drummond, 2017; Wharf Higgins et al., 2009). Additionally, current measures do not assess the unique and dynamic contexts and processes of children and adolescents, including the nature of their social experiences, individual attributes (e.g, lack of concrete reasoning), or ways in which they navigate a complex, changing health environment (Ormshaw et al., 2013; Manganello et al, 2017). Finally, current measures do not generate data on a broad scale where children typically are—homes, communities, schools—to inform policy and practice.

We argue that the field needs to turn its attention to measuring children's HL using more dynamic, pragmatic, and solution-centered frameworks via everyday contexts. We present recommendations to establish a more unified approach to measurement and ensure that we collect highly desired data with policy and practice implications using psychometrically sound, developmentally appropriate, and practically administered instruments. Our recommendations are based on Glasgow and Riley's criteria for “pragmatic measures,” or measures that are relevant to stakeholders, feasible to use in most real-world settings, and actionable to solve public health problems (Glasgow & Riley, 2013). We define stakeholders as both individuals interested in better understanding HL (e.g., pediatric and developmental clinicians, government agency representatives, researchers, health educators, teachers) as well as individuals affected by HL (e.g., children/adolescents and their parents).

Glasgow and Riley present nine criteria for pragmatism that should be considered as measures are developed (Glasgow & Riley, 2013). We apply these criteria to the current state of the science on child/adolescent HL measurements and provide actionable steps within each to unify the field and move forward. Table defines each criterion and describes action steps.

Characteristics of Pragmatic HL Measures for Children and Adolescents

Related to Theory/Model

As mentioned, many authors have called for redefining HL for children and adolescents and ensuring that measures match this new definition. Many current measures are based on a static, risk-factor philosophy often used for adults (Nutbeam, 2008). Moving forward, measures should be grounded in a lifespan, social ecological perspective. This perspective posits that a child's HL and everyday health behaviors evolve from and are shaped by interpersonal relationships as well as environmental, media, policy, and societal factors (Bröder et al., 2017; Guo et al., 2018; Manganello et al., 2017; Ormshaw et al., 2013; Rothman et al., 2009; Sørensen et al., 2012; Wharf Higgins et al., 2009). These individual attributes and contextual factors change across developmental stages. Adolescents acquiring independent decision-making capabilities have different HL-related competencies than younger children who are not yet exercising autonomy. Interpersonal influences are heightened during adolescence, when people are behaving according to the behavior of others, shaping their world view, and understanding their responsibilities for their own health and the health of their families, social networks, and society (Bröder et al., 2017; Fleary, Joseph, & Pappagianopoulos, 2018; Paakkari & Paakkari, 2012; Velardo & Drummond, 2017; Wharf Higgins et al., 2009). Further, children and adolescents are particularly vulnerable to social determinants of health (Bröder et al., 2017). Measures should capture HL as a dynamic construct and incorporate both the contextual and individualized factors that might influence HL and hinder or facilitate decision-making at various stages of childhood and adolescence.

Psychometrically Strong

Current HL measures for children and adolescents have not been consistently tested for reliability and validity in this population. Further, in general, measures assess only a few aspects of HL that are prioritized in adults (e.g., functional HL, reading comprehension, numeracy) (Bröder et al., 2017; Fleary et al., 2018; Guo et al., 2018; Manganello et al., 2017; Okan et al., 2018; Ormshaw et al., 2013), which is insufficient to capture the dynamic, multidimensional concept in children and adolescents. New measures should be psychometrically strong, use mixed methods, and related to the child-specific definition or framework previously described (to the extent possible without a “gold standard”) (Guo et al., 2018). Instruments should be developed to ensure that consistent benchmarks uniquely define “limited” or “adequate” HL, allowing for meta-analytic comparisons across studies to establish clinically significant thresholds (Jordan, Osborne, & Buchbinder, 2011; Park et al., 2017). To ensure that measures capture all constructs of the more complex framework/definition required, quantitative instruments should be tested for reliability and validity (Jordan et al., 2011), and trustworthiness of qualitative instruments should be assessed.

Important to Stakeholders

To ensure that new measures have broad application and meaningful use, they should be relevant and important to adult and child stakeholders. Adult stakeholders from various sectors (clinicians, researchers, educators) should reach a consensus on the general definition or framework for HL, so that measures based on this framework meet their needs. Importantly, child and adolescent stakeholders should be active participants in this process (Bröder et al., 2017). Despite knowledge that children and adolescents are interested in and capable of being involved in their own health decisions, and that their perspective is “irrefutably valuable,” (Velardo & Drummond, 2017), they are seldom involved in discussions about HL (Brown, Teufel, & Birch, 2007; Pais, Rodrigues, & Menezes, 2014). They can provide valuable insight on how, from whom, and under which context they receive health information and make everyday health decisions (Perry, 2014). Thus, authentically involving these stakeholders in measures' development increases the likelihood that those measures are relevant, acceptable, and understandable (Doustmohammadian et al., 2017; Ghanbari, Ramezankhani, Montazeri, & Mehrabi, 2016; Glasgow & Riley, 2013; Pais et al., 2014; Wharf Higgins et al., 2009).

Low Respondent/Staff Burden

Many of the current broadly used instruments that were adapted from adult measures are brief and orally administered by trained professionals. This is intended to ensure a universal precautions approach and reduce testing bias and respondent burden; however, it also prevents obtaining rich, comprehensive data, and limits the ability to collect data in everyday settings (schools, homes, clinical settings) where children and adolescents ranging from age 7 to 18 years navigate health decisions (Guo et al., 2018; Park et al., 2017; Velardo & Drummond, 2017). Current instruments also increase staff burden, making the collection of data in these settings expensive and complex. Harnessing technological advances, novel survey development theories, and rapid data collection techniques can facilitate improved administration, while still minimizing burden and bias and taking a universal precautions approach.

Actionable in “Real-World” Settings

Protocols are needed to ensure that clinicians, educators, policymakers, and other stakeholders are aware that the instruments exist, are trained on how they can be administered across settings (schools, homes, clinical settings), and have a clear understanding of how to interpret findings and take action (Glasgow & Riley, 2013; Perry, 2014). To meet this criterion, findings should directly translate to intervention mechanisms, including programs or policies that can affect change in schools, health care settings, and perhaps in homes (Velardo & Drummond, 2017). Measures should combine instruments designed to be administered more broadly with more tailored mixed methods tools that capture the data that are actionable in a particular setting, with consideration given to time constraints in these settings and the sample's developmental stage (Glasgow & Riley, 2013; Park et al., 2017; Rothman et al., 2009).

Sensitive to Change

Because most adult measures were constructed under the philosophy that HL is a risk factor for various adverse health outcomes, rather than as a potentially changeable construct, they are typically administered cross-sectionally (Jordan et al., 2011; Nutbeam, 2008). Under the more dynamic, asset-based philosophy we advocate for in children and adolescents, HL both evolves across developmental stages and can be improved through targeted programs and/or policies (Nutbeam, 2008; Velardo & Drummond, 2017). To assess the hypothesized interactions between developmental phases and HL dimensions or the response of participants to a program/policy, measures will likely needed be repeatedly, reliably administered over a long period (3–5 years) and/or cross-sectional responses will need to be compared across age groups (Paakkari, Torppa, Kannas, & Paakkari, 2016).

Broadly Applicable

Many previous validation studies have taken place in either clinical settings or as part of nongeneralizable intervention studies (Guo et al., 2018; Manganello et al., 2017; Ormshaw, Paakkari, & Kannas, 2013). New measures should be applicable not only across everyday settings, but also across various races and ethnicities, income levels, and geographic locations (Nguyen et al., 2015). Formative studies need to deploy multiple community stakeholders throughout all phases and recruit samples from diverse communities and across various settings frequented by children and adolescents within those communities to ensure that the measure is well-understood and equitably applicable (potentially with minor adaptations) across subpopulations (Bröder et al., 2017; Nguyen et al., 2015).

Serve as a Benchmark

New measures should serve as a benchmark and be accompanied by a nationwide dissemination and use plan. Although a foundational measure is a clear priority for the field, there is also simultaneously a broad body of literature on topic-specific literacy measures (e.g., eHealth, nutrition/food) (Doustmohammadian et al., 2017; Guo et al., 2018; Paakkari et al., 2016). People interested in topic-specific measures should be key stakeholders in the reframing of the general definition of HL, and subsequently use a similar lifespan, social ecological approach for their topic. New HL measures (and their scoring mechanisms) should be broadly disseminated through a publicly available repository to serve as a foundation from which topic-specific measures can be systematically adapted.

Unlikely to Cause Harm

Measures should avoid asking questions that may not be well-understood or cause emotional harm, stigma, or shame. The extent to which children lack critical thinking skills should be considered to prevent this (Fleary et al., 2018; Rothman et al., 2009). Additionally, as with assessments of children's readiness for school and for any learning abilities, the ethics and sensitivity of how to report findings and work with families should be considered.

Summary and Future Directions

Glasgow and Riley's framework allows us to consider what “first steps” are needed to develop and test pragmatic measures for children and adolescents that can inform practice and policy to improve HL and ultimately health outcomes (Glasgow & Riley, 2013). For assessing HL in both children and adolescents and adults, it is time to focus on pragmatism, meaningfully engage key stakeholders, and ensure that measures have broad application and can be used beyond research studies to address public health problems.

In Table , we map actionable recommendations on to each of Glasgow and Riley's criteria, both required and recommended, to offer a direction for future research (Glasgow & Riley, 2013). These recommendations do not attempt to describe an end product; rather, they outline specific techniques to consider to ensure that this product is pragmatic. Meeting these recommendations will require time, commitment, and a reframed understanding of health literacy as a dynamic construct from clinicians, researchers, educators, funders, and other relevant stakeholders. To achieve pragmatism, stakeholders from each sector should convene a national working group, including children and adolescents, to redefine HL as it makes sense based on stakeholders' perspectives, developmental theories, existing literature, and public health priorities. Funding agencies should fund robust, longitudinal validation studies that prioritize pragmatism. Once these studies are conducted, stakeholders need to publicly disseminate new measures (and action-oriented scoring protocols) for broad use. We should hold one another accountable for using measures, publicly reporting findings, and ensuring that findings reach practitioners and decision-makers who can alter practice and policy. Finally, researchers need to continually revisit measures over time to improve applicability in this rapidly evolving field.

Table 1

Recommendations for Development and Validation of Pragmatic Health Literacy Measures for Children and Adolescents

Glasgow and Riley's Criteria for Pragmatic Measures	First Steps for Health Literacy Measures Development
Related to theory/model  Advances understanding and interpretation of results to promote scientific understanding	Reach a consensus on the framework/definition of health literacy, then develop measures that match that definitionRecruit a validation study sample that is large and diverse enough to conduct advanced path analysis to ensure that desired theoretical constructs are addressed
Psychometrically strong  Have good reliability, validity, trustworthiness	Conduct factor analyses to test and confirm new framework/definitionEstablish consistent benchmarks that align with framework and uniquely categorize children and adolescents by health literacy statusDevelop and assess reliability, validity, and trustworthiness of both quantitative and qualitative measures
Important to stakeholders[a]  Involves stakeholders on ongoing basis to satisfy differing priorities	Recruit an interdisciplinary team of invested stakeholders in both redefining health literacy and as part of the team developing new measures and hold regular stakeholder meetingsInvolve children and adolescents as stakeholders in all phases of development, formative testing, and validation to ensure that tools are relevant, understood, and measure what is intended
Low respondent/staff burden[a]  Kept brief and inexpensive (in terms of both time and cost)	Test multiple administration methods that do not require oral administration (e.g., online/telephone surveys, semi-structured phone interviews, open-ended questions)Maintain use of a universal precautions approach by incorporating technological advances (read-aloud features, pictures, interactive games)Use item response theory and strategic skip patterns to reduce number of items
Actionable[a]  Appropriate for use and immediate interpretation in busy, “Real-World” settings	Collect both qualitative and quantitative data using rapid data collection method (e.g., brief interviews) to ensure that data are desired by and relevant to end userIdentify a “common core” of measures that can be administered more broadly if the whole battery will not be feasible to administerEstablish simple scoring metrics and use of rapid data analysis techniques to enable quick identification and interpretation key findings
Sensitive to change[a]  Reliable over time, valid so that progress can be tracked and intervention effects detected	Ensure that constructs are dynamic in the newly identified framework/definitionObserve a subsample over a long period (3–5 years) and administer instruments yearlyAssess hypothesized interactions between developmental phases and health literacy constructs
Broadly applicable  Feasible for anyone to complete, so that it can be administered equitably and used to compare subgroups and settings	Recruit a nationwide sample across multiple age groups from locations outside the clinical setting that are frequented by children/adolescents (e.g, schools, sports teams, church youth groups)Conduct extensive pilot testing to ensure acceptability and relevance
Serves as a benchmark  Useful across settings/subgroups and publicly available to address public health goals	Develop an online, publicly available repository to store measures, publish scoring instructions, and hold other members of the field accountable for contributing toolsPublish adaptation protocol for specific topics (i.e., nutrition literacy)/subgroups (i.e., children with diabetes)Ensure that scoring system aligns with pre-established thresholds/benchmarks
Unlikely to cause harm  Invokes minimal unintended consequences or liability	Establish explicit protocols to ensure codes of conduct are followedEmploy developmental experts to monitor study proceedings and ensure that the potential for harm is minimized

Note.

Glasgow and Riley (2013) consider these “required” criteria for pragmatism.