Definitions and measurement of health literacy in health and medicine research: a systematic review.

OBJECTIVES: The way health literacy is understood (conceptualised) should be closely linked to how it is measured (operationalised). This study aimed to gain insights into how health literacy is defined and measured in current health literacy research and to examine the relationship between health literacy definitions and instruments.
DESIGN: Systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. DATA SOURCES: The MEDLINE, PsycINFO, ERIC and CINAHL databases were searched for articles published during two randomly selected months (March and October) in 2019. ELIGIBILITY CRITERIA: We included articles with a quantitative design that measured health literacy, were peer-reviewed and original, were published in the English language and included a study population older than 16 years. DATA EXTRACTION AND SYNTHESIS: Six researchers screened the articles for eligibility and extracted the data independently. All health literacy definitions and instruments were considered in relation to category 1 (describing basic reading and writing skills, disease-specific knowledge and practical skills) and category 2 (social health literacy competence and the ability to interpret and critically assess health information). The categories were inspired by Nutbeam's descriptions of the different health literacy levels.
RESULTS: 120 articles were included in the review: 60 within public health and 60 within clinical health. The majority of the articles (n=77) used instruments from category 1. In total, 79 of the studies provided a health literacy definition; of these, 71 were in category 2 and 8 were in category 1. In almost half of the studies (n=38), health literacy was defined in a broad perspective (category 2) but measured with a more narrow focus (category 1).
CONCLUSION: Due to the high degree of inconsistency between health literacy definitions and instruments in current health literacy research, there is a risk of missing important information about health literacy considered be important to the initial understanding of the concept recognised in the studies. PROSPERO REGISTRATION NUMBER: CRD42020179699. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

To our knowledge, this is the first systematic review to investigate connections between health literacy definitions and instruments used in current health literacy research.

The health literacy definitions and instruments were categorised based on a health literacy model described by Nutbeam.

The initial plan to assess health literacy definitions and instruments for three categories was changed to two due to difficulties in distinguishing between two of the categories.

Searches were limited to two randomly selected months.

Introduction

Health literacy is usually understood as cognitive and social skills that determine the motivation and ability to understand and use health information,1 and adequate health literacy is seen as a prerequisite for healthy behaviours. Researchers have increasingly worked to identify challenges associated with health literacy and investigate the role they play in an individual’s ability to comprehend self-care information and its relationship to health outcomes.2 Empirical studies have reported that low health literacy is associated with poor health-related outcomes, such as high hospital admission rates,3 4 low participation in preventive activities,5–9 poor self-management of chronic conditions,10 poor disease outcomes and high mortality.11–13

The concept of health literacy emerged in the 1970s when health education was viewed as social policy.14 A topic-specific query in the PubMed tools reveals a recent exponential growth of articles about health literacy, with 129 references between 1986 and 1990 increasing to more than 8000 in the past 5 years. Today, health literacy is seen as a global goal for enhancing health promotion through improved education and communication strategies to improve health outcomes.15

Health literacy is defined in numerous ways.14 16–18 In a systematic review by Sorensen et al,14 17 different definitions and 12 conceptual models were reported on health literacy. Another systematic review found that 34 research articles between 2010 and 2015 had an explicit objective to define the concept of health literacy.17 Moreover, the literature has reported the use and development of more than 150 health literacy instruments over the last decade.19–21 Traditionally, health literacy approaches have focused on individual skill deficits and health education based on the communication of factual information regarding health risks as well as how to use the health system,22 with the majority of health literacy research having used instruments measuring reading and numerical skills.23 However, in recent years, more multidimensional perspectives and instruments measuring health literacy have been introduced, such as the Health Literacy Questionnaire24 and the Health Literacy Survey European Questionnaire 47.25

The current and sometimes confusing use of various and inconsistent interpretations of health literacy is a challenge for the development of valid and reliable measurements.16 In 2000, Nutbeam proposed a health literacy model that is now widely cited in the health literacy literature and is seen by many health literacy researchers as useful in analysing health literacy abilities required in various health situations.22 According to Nutbeam, health literacy contains three different levels, progressing from basic skills in reading and writing (functional health literacy), to the ability to derive meaning from different forms of communication and apply new information to changing situations (interactive health literacy) and to more advanced cognitive skills which, together with social skills, can be applied to critically analyse information and to achieve policy and organisational changes (critical health literacy).

Different understandings of health literacy and different measurement tools may be useful as they complement each other and provide different perspectives. However, the way health literacy is understood (conceptualised) should be closely linked to how it is measured (operationalised) in each study context.23 Nguyen et al has described this as a ‘conceptual stumbling block’ that needs to be resolved for the field to progress.23 A first useful step for addressing this might be to systematically explore how it appears in current health literacy research. Hence, by performing a systematic review, our aim was to gain insights into how health literacy is defined and measured in current health literacy research. In particular, we will examine the relationship between health literacy definitions and instruments. This review may increase our understanding of potential conceptual and methodological challenges or gaps that need to be addressed in future research.

Methods

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement26 and registered in PROSPERO (https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=179699). The review was designed with a time frame limited to two randomly selected months in 2019 (March and October). Due to a high number of health literacy articles published every year, random selection was chosen in order to reflect current health literacy research. To ensure that March and October were not unique in terms of the number of articles published, we performed the same search strategy using the same databases for one other randomly selected month in 2019. This search yielded nearly the same number of articles.

Search strategy

Systematic literature searches were conducted in collaboration with a trained librarian (4 March 2020). The MEDLINE, PsycINFO, ERIC and CINAHL databases were searched for the term ‘health literacy’. For all databases except for ERIC (where this was not possible), the search was automatically restricted to two randomly selected months in 2019: March and October. Citations in ERIC were manually assessed for articles published in March 2019 and October 2019.

The search yielded a total number of 1038 citations. Endnote V.X9 was used to manage the generated research articles. After removing duplicates, 951 citations remained. All records not published for the first time in 2019 were removed, leaving 414 articles for screening (see online supplemental appendix 1 for the search history).

Selection criteria

The inclusion and exclusion criteria were developed a priori. The 414 published articles were distributed among six researchers (KHU, AKW, MHA, CRhB, SH and MHL), who worked in pairs. The articles were included if they fulfilled the following inclusion criteria: (1) ‘health literacy’ was mentioned in the title or abstract; (2) the article was peer reviewed; (3) the research was original; (4) it used a quantitative design; (5) it was published in the English language; and (6) the study population included individuals older than 16 years of age.

To reduce selection bias, the researchers independently screened the articles for eligibility according to the criteria. They then met in pairs to compare their results, resolve any conflicting opinions and decide whether to include each article. Conflicting opinions in pairs were presented and solved by the whole team.

Data extraction (selection and coding)

To achieve consistency in the data extracted from the included articles, an Excel V. 2019 spreadsheet was created. The initial question in this spreadsheet was whether health literacy was measured. If the answer was no, the article was excluded. The data extracted from the articles included information about the study design and context, such as country of origin and whether the study was conducted in a public health or clinical health setting. For clinical studies that included participants with health problems, the different types of diseases were categorised according to the International Statistical Classification of Diseases and Related Health Problems.27 For the public health studies, we categorised type of study populations. Furthermore, the data included information about whether the instruments were generic or disease specific, whether it was used in combination with other health literacy instruments or if a reference was provided for the instrument.

The published articles were analysed with respect to which health literacy definitions and instruments were used. Before the data extraction, a pilot assessment of three studies was performed to determine the feasibility of the data extraction tool. The initial plan was to assess health literacy definitions and instruments for three categories: (1) functional, (2) interactive and (3) critical health literacy, inspired by Nutbeam’ s description of different health literacy levels.22 As a result of the pilot testing, the number of categories was changed due to difficulties in distinguishing between the two latter levels. All health literacy definitions and health literacy instruments were therefore considered in relation to two categories: category 1 included definitions and instruments describing basic reading and writing skills, disease-specific knowledge and practical skill competencies needed to function in everyday situations; and category 2 included definitions and instruments that also described health literacy as skills to communicate and interact with healthcare providers as well as the ability to interpret and critically analyse health information (online supplemental appendix 2 illustrates the coding for the data extractions).

The researchers first extracted data and considered independently the definitions and instruments related to the two categories. They then met in pairs to compare their results and resolve any conflicting opinions. The remaining conflicting opinions within pairs were presented and solved by the whole team in a group meeting.

Patient and public involvement

No patient was involved.

Results

The search yielded 1038 articles in total (see the online supplemental appendix 1 for the search history). After removing duplicates, 951 articles remained. Of the 163 articles that were read in full text, 1 was a duplicate, and 39 were excluded because they did not measure health literacy. Ultimately, 120 articles were included (see figure 1).

Figure 1

Flow diagram shows the study selection process. HL, health literacy.

Characteristics of included studies

Sixty articles described studies with a clinical health focus, and 60 were conducted in public health settings. The clinical studies included a great variety of diseases, but the most frequent diseases were those in the circulatory system (n=11) and endocrine diseases (n=9) followed by mental illness (n=6) and cancer diseases (n=6). In the public health studies, the most frequent study group was the general population (n=19), looking into aspects such as vaccination programmes, oral care and mental health. Furthermore, a major part of the public health studies included students (n=13), mainly represented by health professional students. Six studies focused on parents/caregivers, while five studies focused on the elderly population.(table 1)

Table 1

Overview of study populations in clinical and public health studies

Clinical health studies		Public health studies
Populations classified by diseases and related health problems	n	Type of study population	n
Diseases in the circulatory system	11	General population	19
Endocrine diseases	9	Students	13
Mental illness	6	Parents/caregivers	6
Cancer	6	Elderly people	5
Diseases in the respiratory system	4	Immigrants	3
Diseases in the urine and genital organs	4	Work-related populations	3
Infectious and parasite diseases	3	Young adults	2
Diseases in the ear	2	Veterans	2
Diseases in the musculoskeletal system and connective tissue	2	Health personnel	2
Diseases in the nervous system	1	Men	1
Diseases in the digestive system	1	Women	1
		Pet owners	1
Pregnancy, birth, postnatal period	1	Relatives of patients with cancer	1
Others: general chronic conditions (n=2), chronic pain (n=3), patients in primary care (n=2), next of kin (n=2), patients in specialist care (n=1)	10	Churchgoers	1
Total	60	Total	60

The majority of the studies used a cross-sectional design (n=90). Nine studies used a randomised controlled design, and 10 studies were quasi-experimental. Four of the studies used a longitudinal design, while one was a case–control study. Figure 2 presents an overview of all study designs.

Figure 2

Overview of study designs of included studies. RCT, randomised controlled trial.

As figure 3 indicates, the majority of studies were conducted in North America (n=56) and Asia (n=31). The fewest studies were performed in Africa (n=4) and South America (n=3).

Figure 3

Studies’ continent of origin.

Health literacy definitions and instruments

Out of the 120 included studies, 88 used generic health literacy instruments, while 32 were context-specific. Eleven studies used a combincation of health literacy instruments. A total of 77 studies used instruments from category 1 (describing basic reading and writing skills, disease-specific knowledge and practical skills), whereas 43 instruments were from category 2 (describing communication and interaction skills and/or the ability to interpret and critically analyse health information). The most frequently used health literacy instruments in category 1 were Newest Vital Sign28 (n=19) and Test of Functional Health Literacy29 (n=13). The most frequently used health literacy instruments in category 2 were The European Health Literacy Survey Questionnaire30 and The eHealth Literacy Scale31 (n=10).

Of the 120 articles, 79 provided a health literacy definition. Of these, 46 were public health studies and 33 were clinical health studies. Only eight studies provided a definition addressed to category 1. Five out of these eight studies focused on mental health challenges.

The most frequently used reference in category 2 was Nutbeam.22 32 33 34 The most frequently used reference in category 1 was a definition provided by Jorm and colleagues.35 36 Table 2 gives an overview of the text in the most cited health literacy definitions. Words in italics are examples of key words considered important in the decision to add the definition to category 1 or category 2. For instance, definitions that described health literacy with terms such as ‘knowledge’, ‘skills’ and ‘attitude’ were referred to category 1, while definitions with terms like ‘appraise’ and ‘social skills’ were referred to category 2.

Table 2

Most frequently used references for health literacy definitions in the included studies

Reference	Definition	Studies (n)
Category 1
Jorm (1997)Jorm36	‘Knowledge about appropriate treatment options; and attitudes that facilitate recognition and treatment-seeking’‘Knowledge and beliefs about mental disorders which aid their recognition, management or prevention’	5
American Medical Association (1999)39	‘The constellation of skills, including the ability to perform basic reading and numerical tasks required to function in the healthcare environment’	1
Category 2
Nutbeam22 32–34	‘The cognitive and social skills which determine the motivation and ability of individuals to gain access to understand and use information in ways which promote and maintain good health.’	19
Sørensen et al14	‘Health literacy entails people’s knowledge, motivation and competences to access, understand, appraise, and apply health information in order to make judgments and take decisions in everyday life concerning healthcare, disease prevention and health promotion to maintain or improve quality of life during the life course.’	14
The Institute of Medicine (2004)40	‘The individuals’ capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.’	11

Regarding the connection between health literacy definitions and instruments in the 79 relevant studies, 41 articles used health literacy definitions and instruments characterised at the same level. Thirty-three of these were in category 2, and 8 were in category 1 (see figure 4 and table 3).

Figure 4

Studies categorised with the same level of HL definitions and HL instruments. *Category 1 includes basic reading and writing skills, disease-specific knowledge and practical skill competences. **Category 2 includes skills to communicate and interact with healthcare providers and the ability to interpret and critically analyse health information. HL, health literacy.

Table 3

Studies categorised with mutual levels of health literacy definitions and instruments (n=41)

Study reference	Study context	Design	Health literacy definition reference	Category	Health literacy instrument	Category
Category 1
Cehuen Neto et al 201941	Clinical	Cross-sectional	American Medical Association Ad Hoc Committee on Health, 199939	1	Short Test of Functional Health Literacy in Adults (Parker et al29	1
Clough et al 201942	Public health	RCT	Jorm et al35	1	Mental Health Literacy Scale (O’Connor and Casey, 2015)43	1
Clough et al 201944	Public health	Cross-sectional	Jorm et al35	1	Mental Health Literacy Scale (O’Connor and Casey, 2015)43	1
Joncho et al 201945	Clinical	Cross-sectional	Jorm et al35	1	Developed within the study context: Mental Health Literacy Questionnaire	1
Lichtveld et al 2019	Public health	Cross-sectional	Finn and O’Fallon, 2019Gray, 201846	1	Developing and validating a questionnaire measuring environmental health literacy: measuring media-specific knowledge, attitudes, and behaviours	1
Nye et al 201947	Public health	Longitudinal	US Department of Health and Human Services, 201048	1	The Upper Peninsula Oral Health Assessment Survey	1
Waldmann et al 202049	Clinical	Cross-sectional	Jorm36	1	Mental Health Knowledge Schedule.Depression Literacy Scale (Wilson, 2015)50	1
Williston et al 202051	Public health	Longitudinal	Jorm et al35	1	Mental Health Literacy Scale (O’Connor and Casey, 2015)43	1
Category 2
An et al 201952	Public health	Cross-sectional	Nutbeam22	2	The Health Literacy Scale (Chew, 2004)53	2
Azizi et al 201954	Public health	Mixed method	Nutbeam22	2	Health Literacy Scale for Workers54	2
Brandstetter et al 201955	Public health	Cross-sectional	Sørensen et al14	2	The European Health Literacy Survey (HLS-EU) (Sørensen et al25)	2
Degan et al 201956	Clinical	Cross-sectional	WHO, 19981	2	The Health Literacy Questionnaire (Osborn et al24)	2
Eo et al 201957	Public health	Cross-sectional	Nutbeam22	2	The Health Literacy Assessment Scale for Asian Immigrant Women (And and Yang, 2015)58	2
Ernsting et al 201959	Clinical	Cross-sectional	Sørenesen et al14	2	European Health Literacy Survey Questionnaire—short form (Sørensen et al 2013) and The eHealth Literacy Scale (Norman and Skinner31)	2
Erunal et al 201960	Public health	RCT	Nutbeam22	2	Turkish Health Literacy Scale-32 (Okyay et al 2016)61	2
Evans et al 201962	Public health	Cross-sectional	Sørenesen et al14	2	European Health Literacy Survey Questionnaire (Sørensen et al25)	2
Fernandez Gutierrez et al 201963	Public health	Quasi-experimental	Nutbeam22	2	HLS EU-16 (Sørensen et al25)Health Literacy App Questionnaire+five practical tests	2
Goto et al 201964	Public health	Cross-sectional	Nutbeam, 199833	2	The Functional Communicative and Critical HL Tool (Ishikawa et al 2008)	2
Guclu et al 201965	Clinical	Cross-sectional	WHO, 1998	2	European Health Literacy Survey Questionnaire (Sørensen et al 2013)25	2
Güner et al 201966	Public health	Cross-sectional	Nutbeam,200022	2	Questionnaire developed within the study context (consisting of 21 questions about HL knowledge, practice and attitudes)	2
Hu et al 201967	Public health	Cross-sectional	Nutbeam, 200022	2	Health Literacy Management Scale (Jordan, 2013)68	2
Indino et al 201969	Public health	Cross-sectional	Nutbeam, 201532	2	The Functional Communicative and Critical Health Literacy Tool (Ishikawa et al 2008)	2
Kaper et. al, 201970	Public health	Quasi- experimental	Kwan, 200671	2	Questionnaire developed within the study context by combining subscales from other questionnaires	2
Kobayashi et al 201972	Public health	Mixed method	Sørensen et al14	2	European Health Literacy Survey Questionnaire (Sørensen et al25)	2
Lim et al 201973	Clinical	Cross-sectional	Osborne et al24	2	Health Literacy Questionnaire (Osborne et al24)	2
Logullo et al 201974	Public health	Cross-sectional	Sørensen et al14	2	Short Assessment of Health Literacy for Portuguese Speaking Adults (Apolonario et al 2012)75	2
Lu et al 202076	Clinical	Cross-sectional	Sørensen, 201214	2	HLS EU-16 (Sørensen et al 2013)25	2
Lu et al 201977	Clinical	Cross-sectional	Sørensen, 201214	2	HLS EU-16 (Sørensen et al 2013)25	2
Ma et al 201978	Public health	Cross-sectional	Norman and Skinner31	2	The eHealth Literacy Scale (Norman and Skinner31)	2
Nokes et al 201979	Clinical	Quasi- experimental	Norman and Skinner31 and 2011	2	The eHealth Literacy Scale (Norman and Skinner)31	2
Oh et al 201980	Public health	Cross-sectional	WHO,19981	2	The eHealth Literacy Scale (Norman and Skinner31)	2
Oliffe et al 208120	Public health	Cross-sectional	Peerson and Saunders, 2009,82 O’Brian et al 2005	2	The Health Literacy Questionnaire (Osborn et al 2013)24	2
Pobhirun and Pinitsoontorn, 201983	Public health	Cross-sectional	Nutbeam,199833	2	A questionnaire developed within the study context including six dimensions: cognitive skills, access, communication skills, self-management, media literacy and decision-making skills	2
Rababah84 et al 2019	Public health	Cross-sectional	WHO, 2013, Sørensen et al14	2	The Health Literacy Questionnaire (Osborn et al 2013)24	2
Ruëgg and Abel, 201985	Public health	Cross-sectional	Selden et al 200086	2	Short survey tool for public health and health promotion research (Abel et al, 2014)87	2
Solhjoo et al 201988	Public health	Mixed method	Sørensen, 201214	2	The eHealth Literacy Scale (Norman and Skinner31)	2
Stømer et al 201989	Clinical	Cross-sectional	WHO, 19981	2	The Health Literacy Questionnaire24 (Osborn et al 2013)	2
Uysal et al 209020	Public health	Cross-sectional	Sørensen, 201214	2	European Health Literacy Survey Questionnaire (Sørensen et al25)	2
Wang et al 201991	Clinical	Cross-sectional	Kutner et al 200392	2	Chinese Health Literacy Scale for Diabetes (Leung, 2013)93	2
Wang et al 201994	Clinical	Cross-sectional	Nutbeam, 200995	2	Diabetes Health Literacy Scale (Lee et al, 2018)96	2
Zhang et al 209720	Clinical	Cross-sectional	Sørensen, 201214	2	Heart Failure-specific Health Literacy Scale (Matsuka et al 2018)98	2

HLS EU, European Health Literacy Survey Questionnaire; RCT, randomised controlled trial.

In the remaining 38 studies, there was a disconnect between levels of health literacy definitions and instruments. In all of these, health literacy definitions were from category 2, and all instruments were in category 1 (see table 4).

Table 4

Studies categorised with conflicting levels of health literacy definitions and instruments (n=38)

Study reference	Study context	Design	Health literacy definition reference	Category	Health literacy instrument	Category
Anderson et al 209920	Clinical	Cross-sectional	Defined within the study context (no reference provided)	2	S-TOFHLA (Parker et al29)	1
Avci et al 2019100	Public health	Cross-sectional	US Department of Health and Human Services, 200048	2	NVS (Weiss et al28) and REALM (Davis, 1993)101	1
Bonaccorsi102 et al 2019	Public health	Cross-sectional	Sørensen et al14	2	NVS (Weiss et al28)	1
Bonaccorsi et al 2019 II103	Public health	Cross-sectional	Sørensen et al14	2	NVS (Weiss et al28)	1
Carducci et al 2019104	Public health	Cross-sectional	Ratzan and Parker, 2000Nutbeam, 200022	2	TOFHLA (Parker, 1995)29	1
Chen et al 2019105	Clinical	Cross-sectional	Nutbeam, 200022	2	Health Literacy Scale for Diabetes (Lee et al 2016)106	1
de Melo et al 2019107	Clinical	Cross-sectional	Cavanaugh, 2011108	2	Short Test of Functional Health Literacy in Adults (Parker et al29)	1
Flynn et al 2019109	Public health	Cross-sectional	American Dental Association Council, 2011009	2	Oral Health Literacy Adults Questionnaire (Sistani et al 2014)111	1
Gaikwad, 2019112	Public health	Cross-secttional	Ratzan and Parker, 2000113	2	Rapid Estimation of Adult Literacy in Dentistry30-word version (Lee et al 2007)114	1
Güner et al 2019115	Public health	Cross-sectional	Nutbeam22	2	Developed within the context of the study	1
Han et al 2019116	Public health	Cross-sectional	Sørensen et al14 Nutbeam22	2	Assessment of Health Literacy in Cancer Screening (Han et al 2014)117	1
Himes et al 2019118	Public health	Cross- sectional	Institute of Medicine, US, Committee HL, 200440	2	Chew et al’s (2004) set of brief questions53	1
Irvin et al 2019119	Public health	Cross-sectional	Peters et al 2012120	2	The Water Environmental Literacy Level Scale (Irvin et al 2019)119	1
Kaur et al 2019121	Public health	RCT	Healthy People, 2010 (Oral health)122	2	TS-REALD Two-Stage Rapid Estimate of Adult Literacy (Stucky et al 2011)123	1
Kim et al 2019124	Public health	Cross-sectional	Ratzan and Parker, 2000113	2	Three questions, each addressing oral, listening, and written literacies (no reference)	1
Kim et al 2019125	Public health	Quasi-experimental	Joint Committee on National Education Standards, 1995126	2	Knowledge questions developed in the study context	1
Kim et al 2019127	Clinical	RCT	Institute of Medicine, US, Committee HL, 200440	2	S-TOFHLA (Parker et al29)	1
Kino et al 2020128	Public health	Cross-sectional	Inst of Medicine, 200440	2	Three indicators of HL (Haun et al20)	1
Lin et al 2019129	Public health	Quasi-experimental	Nutbeam, 200022	2	Mandarin Health Literacy Scale (Lee et al 2011)130	1
Lindahl et al 2020131	Clinical	Cross-sectional	Nutbeam and Kickbusch, 2000 132	2	The three-item Brief Health Literacy Screen (Cavanaugh et al 2015)133	1
Mackert et al 2019134	Public health	Cross-sectional	Berkman et al 2010135	2	NVS (Weiss et al28)	1
Mayer et al 2019136	Clinical	Cross-sectional	Nutbeam33	2	Developed within the context of the study	1
Meyers et al 2019137	Clinical	Cross-sectional	Institute of Medicine, 200440	2	NVS (Weiss et al28)	1
Miranda, 2019138	Clinical	Cross-sectional	Sørensen et al14	2	REALM-D (Davis, 1993)	1
Mock et al 2019139	Clinical	Cross-sectional	Institution of Medicine, 200440	2	Single Item Literacy Screening (Morris, 2006), S-TOFHLA (Parker et al29)	1
Mora-Pinzon et al 2019140	Clinical	Cross-sectional	Institute of Medicine, 200440	2	S-TOFHLA (Parker, 1995)29REALM-D (Davis, 1993)101	1
Noback et al 2019141	Clinical	Cross-sectional	Paasche-Orlow et al 2007,142 Berkman et al 2010135,	2	NVS (Weiss et al 2005)28Musculoskeletal specific literacy survey (LiMP questionnaire) developed within the study context	1
O’Conor et al 2019143	Clincial	Cross-sectional	Institute of Medicine, 200440	2	S-TOFHLA (Parker et al29)	1
Penaloza et al 2019144	Clinical	Cross-sectional	Nutbeam33 and Institute of Medicine, 200440	2	Short Assessment of Health Literacy Spanish (Lee et al 2006)145	1
Rafferty et al 2019146	Clinical	Cross-sectional	Institute of Medicine, 2004 40 and Sørensen et al14	2	Questions focusing on health advice and information-seeking skills, oral literacy, and print literacy	1
Tavakoly Sany et al 2019147	Clinical	Quasi-experimental	US Health Resources and Services Administration	2	TOFHLA (Parker et al29)	1
Scrivner et al 2019148	Clinical	Cross-sectional	Nutbeam, 200834	2	Three questions assessing health literacy (no reference)	1
Tucker et al 2019149	Public health	Quasi-experimental	Ratzan and Parker, 2000113	2	NVS (Weiss et al 2005)28	1
Van Wormer et al 2019150	Public health	Cross-sectional	Institute of Medicine, 200440	2	Oral Health Literacy in Adults Questionnaire (Sistani et al 2013)151	1
Weaver et al 2019152	Public health	Cross- sectional	Ratzan and Parker, 2000Nutbeam22	2	REALM (Davis, 1993)101
Zhong et al 2020153	Clinical	Cross-sectional	Kindig, 2004154	2	NVS (Weiss et al 2005)28	1
Williams et al 2019155	Clinical	Cross-sectional	Institute of Medicine, 200440	2	NVS (Weiss et al 2005)28	1
Winokur et al 2019156	Clincial	Cross-sectional	Morrison et al 2013157	2	NVS (Weiss et al 2005)28	1

NVS, Newest Vital Sign; RCT, randomised controlled trial; REALM, Rapid Estimate of Adult Literacy in Medicine; REALM-D, Rapid Estimate of Adult Literacy, Dutch Version; S-TOFHLA, Short Test of Functional Health Literacy in Adults; TOFHLA, Test of Functional Health Literacy in Adults.

Discussion

This systematic review aimed to gain insights into how current research defines and measures health literacy and, in particular, whether studies consistently used definitions and instruments. The high number of articles published in the defined time frame shows that health literacy is of high research interest in both public health and the clinical field in large parts of the world.

We found a large variety of instruments used, and the majority of the included studies (79 out of 120) presented a health literacy definition as part of the study’s theoretical background. However, there seems to be an inconsistency between the definitions and the instruments in a significant number of the studies. In nearly half of the studies, health literacy was defined in a broad perspective (including aspects such as social health literacy competence and the ability to process and appraise health information) while using instruments with a more narrow focus (measuring basic skills and knowledge). As a result, almost half of the articles in our review lacked data on the participants’ ability to critically appraise health information and their social health literacy competence despite the fact that the authors had stated such aspects to be health literacy. This concern has previously been addressed. Numerous systematic reviews have reported on the diversity of understandings of health literacy and the various use of instruments not aligned to the definitions in current research.16–21 However, the current study is, to the best of our knowledge, the first review to systematically investigate the relationship between health literacy definitions and instruments from the perspective of a health literacy model.

The use of instruments that focused on functional health literacy (addressed to category 1), such as Newest Vital Sign and Test of Functional Health Literacy, were predominant in our review.28 29 This finding corresponds with previous findings.19 20 The widespread use of Newest Vital Sign and Test of Functional Health Literacy is justified by the fact that they are screening tools—that is, they are quick, available in an ever-increasing number of languages and can be adapted to different settings.37 However, a broad range of skills and tasks covering functional, interactive and critical domains needs to be included in measures of health literacy in order to capture the health literacy definitions used.32 In recent years, an increasing number of multidimensional instruments have been developed. In our review, The European Health Literacy Survey Questionnaire and The eHealth Literacy Scale, both addressed to category 2, were the multidimensional instruments most often used.30 31 In light of the increased focus on a more multidimensional perspective of health literacy, and the fact that the current review only includes studies from 2019, one could expect a higher use of multidimensional instruments. Instead, they represented only 43 out of 120 instruments.

Clearly, our study highlights the discrepancy between definitions and measurements as well as the narrow focus of health literacy in a large amount of empirical research. An important question to raise is therefore how the findings from this specific study can guide future research strategies to overcome the identified inconsistency. In other words, as most health literacy researchers seem to base their research on a broad understanding of health literacy, what can be done to facilitate an increased use of broad measurements? A first step should be to make researchers aware about the existing mismatch in current research. Furthermore, it seems necessary to develop more instruments that can answer the research questions posed. Despite the high number of instruments, there still seems to be a need for questionnaires in the field of health literacy that capture more multidimensional dimensions besides the functional aspects.

Nutbeam’ s description of three levels of health literacy inspired the categorisation of definitions and instruments.22 Initially, we planned to distinguish among the three levels. However, the line between levels 2 and 3 was challenging to distinguish. Therefore, it was decided to merge the two latter categories. In a recent publication from 2020, Nutbeam has provided a more thorough description of the three levels with a more detailed explanation of how the levels should be understood.38 Perhaps, these descriptions would have contributed to a clearer guidance in our work with the categorisation. However, this material was not available at the time of our work and, in general, this situation illustrates the challenge of adapting a theoretical model into practise.

The interpretation of terms used in the definitions and instruments that guided the choice of category also represented some challenges. These were resolved through discussions both in pairs and as a research team. For instance, definitions describing health literacy using terms like knowledge and ‘beliefs’ were included in category 1, while definitions using terms such as appraise’ and ‘understand and process’ were included in category 2. An interesting finding is that, among the eight studies providing a health literacy definition from category 1, the majority (n=5) were in the context of mental health. The predominant reference in these studies was the definition provided by Jorm and colleagues, who, unlike others, defined health literacy as being linked to beliefs and ‘attitudes’.35 36 Whether these terms are more closely linked to mental health literacy challenges, compared with other more ‘physical’ health literacy issues, is not clear but would be interesting for further investigation.

Another aspect worth noting is that we discovered that many publications did not cite the primary source of the health literacy definitions but instead referred to secondary sources (other researchers presenting definitions of health literacy). Referring to the original sources should be the first choice and would perhaps make it easier for readers to recognise the definition’s affiliation.

The present study has some limitations. First, this study was designed to analyse and describe health literacy research in two randomly selected months. This period may not be representative of health literacy research in general. However, a large number of health literacy studies are published every year. A random selection can therefore give a good picture of health literacy research. Second, we did not conduct a quality assessment screening of the included studies. This was considered less relevant for the current study as the aim of the current study was to explore connections between health literacy definitions and instruments rather than to assess methodology. Furthermore, searches were limited to the English language only. It is possible that similar studies may have been published in languages other than English.

The current review included only quantitative measurements. However, qualitative approaches might provide valuable and more in-depth insights into the field. For future research, it would be interesting to also explore how qualitative research links health literacy definitions to the research questions posed.

Conclusion

There seems to be an inconsistency between the definitions and the instruments used in a significant part of current health literacy research. This situation raises the risk of missing information about health literacy that was considered to be important in the initial understanding of the concept recognised in the studies. This gap should be taken into consideration in future health literacy research. We hope our work contributes to making explicit where the problem might be rooted and that it can be useful in the discussion about strategies for moving forward to better align health literacy measurement with definitions of health literacy.