Health literacy characteristics of over-the-counter rapid antigen COVID-19 test materials.

BACKGROUND: The United States Food & Drug Administration's emergency authorized use, in December 2020, of over-the-counter (OTC) rapid antigen COVID-19 tests was a pandemic control milestone.
OBJECTIVE: To assess health literacy-related characteristics of OTC rapid antigen COVID-19 test materials.
METHODS: Between September-December 2021, we identified eleven (n = 11) OTC rapid antigen COVID-19 tests available for purchase in the US. We assessed readability (Flesch Reading Ease and Fernández-Huerta), formatting and layout features of English- and Spanish-language step-by-step OTC rapid antigen COVID-19 test package insert instructions. Video-based step-by-step OTC rapid antigen COVID-19 test instructions were evaluated for understandability and actionability (Patient Education Materials Assessment Tool for Audiovisual Materials [PEMAT-A/V]), overall quality (Global Quality Scale [GQS]) and cultural diversity and inclusiveness. Descriptive analyses were performed using IBM® Statistical Package for the Social Sciences.
RESULTS: Nine (81.8%) OTC rapid antigen COVID-19 tests included English-language (≈8th-9th reading grade level) step-by-step instructions, while 4 included Spanish-language (≈10th-12th reading grade level) instructions. On average, instructions were printed on a tabloid sized piece of paper, with text size ranging from 4 to 12 point and including nearly 20 illustrations. English-language step-by-step OTC rapid antigen COVID-19 test video-based instructions (n = 6) ranged from 1:04 to 5:41 min with PEMAT-A/V scores ranging from 80% to 100%. As indicated by GQS scores, English-language videos were of high quality (5 videos scored 5/5; 1 video scored 4/5). One COVID-19 test product manufacturing website included Spanish-language video-based instructions (time = 4:59 min; PEMAT-A/V = 100%; GQS = 5).
CONCLUSIONS: OTC COVID-19 test step-by-step instructions-both package inserts and video-based-included features shown to foster patient understanding and facilitate proper use. Moving forward, greater attention needs to be placed on expanding both Spanish-language and video-based OTC COVID-19 test material availability to improve accessibility across diverse populations.

Introduction

The ongoing COVID-19 pandemic has resulted in significant deleterious societal, economic and medical-related impacts across the globe. To control the spread of the highly contagious COVID-19, countries employed widespread non-pharmaceutical control measures (e.g., lockdowns, curfews, social distancing) in immediate response to the World Health Organization's declaration of the COVID-19 global pandemic on March 11, 2020. At the onset of the pandemic, the Centers for Disease Control and Prevention limited formal COVID-19 testing, in the US, to those with a confirmed COVID-19 close contact, recent travel to certain countries and/or COVID-19 hospitalization. The United States (US) Food & Drug Administration's (FDA) emergency authorized use (EAU) of over-the-counter (OTC) rapid antigen COVID-19 tests, in December 2020, was a significant pandemic milestone. An OTC COVID-19 testing option offered the public increased convenience accompanied by rapid (≈15 min) results.

Over the course of the pandemic, observed weekly COVID-19 cases and deaths in the US have fluctuated significantly. However, in late 2021, emergence of the omicron variant resulted in steep increases in COVID-19 cases throughout the US; thereby, propelling demand for OTC COVID-19 testing. To further mitigate COVID-19 community spread, on January 14, 2022, the Biden Administration announced widespread national distribution of free OTC COVID-19 tests to American households.

Studies conducted over the past two decades clearly document health literacy-related deficits (e.g., exceedingly high reading demands, suboptimal layout design features) of a broad range of consumer health information.8, 9, 10, 11, 12 To date, a handful of studies have critically assessed content and health literacy-related features of consumer COVID-19 information. In their assessment of COVID-19 consumer educational materials generated from Google, Yahoo!, and Bing internet searches (search terms included: coronavirus, COVID-19, COVID19, and COVID 19), Caballero et al. found most to be difficult to read and lacked recommended features such inclusion of visual images and clearly defined manageable instructions. Similarly, Kruse and colleagues found COVID-19 patient education materials, from US academic medical centers, to have high reading demands and low actionability. In their review of US state health department COVID-19 resources (e.g., webpages, infographics, and videos), Mani et al. found information to have high understandability and actionability, while needing improvement in terms of overall clarity and readability.

To our knowledge, as no studies have examined OTC rapid antigen COVID-19 test instructional materials, the central purposes of this study were to assess health literacy-related characteristics (e.g., actionability, formatting complexity and readability) of OTC rapid antigen COVID-19 test instructional materials. This study addressed two specific research questions, including: (1) What are the readability demands, formatting and layout characteristics of English- and Spanish-language step-by-step OTC rapid antigen COVID-19 test package insert instructions? and (2) How understandable, actionable, culturally diverse, and inclusive are video-based step-by-step OTC rapid antigen COVID-19 test instructions?

Methods

Selection of OTC rapid antigen COVID-19 tests

Between September–December 2021, we conducted comprehensive Amazon® searches to identify all OTC rapid antigen COVID-19 tests currently available for purchase in the US. We purchased eleven (n = 11) OTC rapid antigen COVID-19 tests, including: (1) BD Veritor™ At-Home COVID-19 Test (Becton, Dickinson and Company, Sparks, MD), (2) BinaxNOW™ COVID-19 Antigen Self-Test (Abbott, Abbott Park, IL), (3) Celltrion DiaTrust™ COVID-19 Ag Home Test (Celltrion USA, Inc., Jersey City, NJ), (4) ellume COVID-19 Home Test (Ellume Health, Frederick, MD), (5) Flowflex™ COVID-19 Antigen Home Test (ACON® Laboratories, Inc., San Diego, CA), (6) iHealth® COVID-19 Antigen Rapid Test (iHealth Labs, Inc., Sunnyvale, CA), (7) on/go™ COVID-19 Antigen Self-Test (Intrivo,™ Santa Monica, CA), (8) QuickVue® At-Home OTC COVID-19 Test (Quidel,® San Diego, CA), (9) COVID-19 (SARS-CoV-2) Antigen Test Kit—Colloidal Gold (Anhui DeepBlue Medical Technology Co., Inc.), (10) SARS-CoV-2 Antigen Rapid Test Kit (Ningbo Home Test Bio-technology Co., Ltd.) and (11) SARS-CoV-2 Antigen Test Set—Colloidal Gold Method (Jiangsu iiLO Biotechnology Co., Ltd.).

Readability of OTC rapid antigen COVID-19 test package insert instructions

The Flesch Reading Ease (FRE) was used to calculate reading grade level of English language step-by-step OTC rapid antigen COVID-19 test package insert instructions. The reading grade level of Spanish language step-by-step OTC rapid antigen COVID-19 test package insert instructions was estimated using the Fernández-Huerta (F–H) formula. FRE and F–H rate text on a 0 (very difficult to read) to 100 (easy to read) scale with lower scores indicating more difficult text. For example, an 81–90 FRE or F–H score is equivalent to approximately a 6th grade reading level.

Formatting and layout features of OTC rapid antigen COVID-19 test package insert instructions

OTC step-by-step rapid antigen COVID-19 test package insert instructions, in both English and Spanish, were measured to the nearest millimeter (mm) with a standard ruler. Predominantly used text font point size was determined by measuring distance from ascent line (top of the capital letters) to descent line (lowermost portion of lower-case letter g or y) with a Westcott Graphic Arts Ruler (Seneca Falls, NY). Total number of illustrations, per step-by-step instruction, were tallied. Illustration dimensions (width and length) were measured to the nearest mm.

We assessed OTC step-by-step rapid antigen COVID-19 test package insert instructions using User-Friendliness Tool (UFT) and Suitability of Materials Assessment (SAM) criteria. Specific features included: (1) layout (avoidance of all-capital letters, italics, and specialty fonts, use of ample white space, short paragraph length, extent of well-organized information), (2) illustration use (clear and realistic, easy to understand), (3) messaging (short headings, gets to point quickly, action-based, clarity, active voice), and (4) manageable information (short sentences, familiar/defined words, Need to Know focus, steps in chronological order).

The first two authors independently reviewed instructions and noted how much effort (little or no, some, or much) would be necessary to bring each criterion to an acceptable level of user friendliness. Overall, the first two authors had high agreement (85.4%) on UFT and SAM assessments. In cases where there was disagreement between the first two authors, the senior author served as a tie-breaker and assigned all final UFT and SAM scores.

Understandability and actionability of OTC rapid antigen COVID-19 test video-based instructions

In January 2022, we recorded all video-based step-by-step OTC rapid antigen COVID-19 test instructions (English and/or Spanish) available on the manufacturer's website for each test. The Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT-A/V) was used to assess understandability (content, word choice and style, organization, layout and design and use of visual aids) and actionability (e.g., material clearly identifies at least one action the user can take) of each video-based step-by-step OTC rapid antigen COVID-19 test instruction. Using the PEMAT-A/V user guide, the senior author scored each understandability (n = 13) and actionability (n = 4) item employing a 3-point scale (0 = disagree, 1 = agree, and NA = not applicable). Understandability and actionability PEMAT-A/V subscales range from 0% to 100%.

Utilizing methodology described by Higashi and colleagues, we assessed cultural diversity and inclusiveness by recording personal characteristics of individuals presented in each video. Using the established Global Quality Scale (GQS), we assessed overall video quality. GQS scores range from 1 (video is of poor quality, poor flow, lacking most information, and therefore not useful for patients) to 5 (video is of excellent quality, excellent flow, very useful for patients, includes completely accurate information).

Data analysis

Descriptive analyses (e.g., frequencies, means, standard deviations) were calculated to depict readability demands, formatting and layout characteristics of English- and Spanish-language step-by-step OTC rapid antigen COVID-19 test package insert instructions and understandability, actionability, cultural diversity, inclusiveness and quality of video-based step-by-step OTC rapid antigen COVID-19 test instructions. All data were entered and analyzed using the IBM® Statistical Package for the Social Sciences,® Version 28.0 (SPSS+, Chicago, Illinois).

Results

Readability, formatting, and layout features of OTC rapid antigen COVID-19 test package insert instructions

Nine of the eleven (81.8%) OTC rapid antigen COVID-19 tests included English-language step-by-step package insert instructions, while the remaining two (n = 2) products included English-language step-by-step instructions via a downloadable Smartphone application exclusively. Four (n = 2) OTC rapid antigen COVID-19 tests included Spanish-language step-by-step package insert instructions as well. As shown in Table 1 , average English-language step-by-step instruction readability was equivalent to 8th-9th grade level (range = 6th grade to college reading level). Reading demands of Spanish-language step-by-step instructions ranged from 7th grade to graduate school level with the majority in the 10th-12th grade range.

Table 1

Readability and formatting characteristics of English- (n = 9) and Spanish-language (n = 4) over-the-counter (OTC) step-by-step rapid antigen COVID-19 package insert instructions.

Readability (n = 13) FREa English-language (n = 9)F–Hb Spanish-language (n = 4)	Mean ± SD (range)64.6 ± 15.2 (36.4–87.8)54.0 ± 27.5 (18.9–79.9)
Formatting Characteristics (n = 9)cFold-out length (cm)Fold-out width (cm)Text size (point)Illustrations (n)Average Illustration length (cm)Average illustration width (cm)	Mean± SD (range)32.7 ± 7.3 (23.9–43.3)31.6 ± 9.9 (21.1–46.0)8.3 ± 3.1 (4–12)19.3 ± 9.9 (4–29)2.7 ± 0.7 (1.9–3.8)3.3 ± 0.7 (2.3–4.3)

Flesch Reading Ease (FRE) English-language readability formula is as follows: 206.835 - (1.015 * average sentence length [number of words/number of sentences]) - (84.6 * average number of syllables per word [number of syllables/number of words]).

Fernández-Huerta (F–H) Spanish-language readability formula is as follows: 206.84 – (0.6 * total number of syllables) – (1.02 * total number of sentences).

With the exception of text (i.e., written narrative in each respective language), formatting characteristics of English-and Spanish-language package instructions were identical (e.g., illustrations, text point size). Therefore, our formatting characteristic assessment was limited to nine (n = 9) unique OTC step-by-step rapid antigen COVID-19 package insert instructions.

On average, step-by-step instructions were printed on the equivalent of a tabloid sized piece of paper (length = 32.7 ± 7.3 cm; width = 31.6 ± 9.9 cm). Text size ranged from 4 to 12 point. Step-by-step instructions included between 4 and 29 illustrations, which on average were approximately half the size of a wallet sized photograph (length = 2.7 ± 0.7 cm; width = 3.3 ± 0.7 cm).

All step-by-step instructions avoided use of all-capital letters, italics, specialty fonts, provided steps in chronological order and used active voice throughout (see Table 2 ). Two-thirds (n = 6/9) of step-by-step instructions presented information in an organized visual format, used short and explanatory headings, got to the point quickly, provided action messages first, and incorporated clear messaging. On average, step-by-step instructions included nearly 20 illustrations. The majority of illustrations served a distinct purpose in addition to being clear, realistic, and easy to understand.

Table 2

Layout features of English- (n = 9) and Spanish-language (n = 4) over-the-counter (OTC) step-by-step rapid antigen COVID-19 package insert instructions.

Layout Features (n = 9)a,b	Proportion of work needed to make more user-friendlyLittle (n) Some (n) Much (n)
Avoids all-capital letters, italics, specialty fonts	9	0	0
Ample white space	3	3	3
Short paragraphs	5	4	0
Information is well organized visually	6	2	1
Headings are short and explanatory	6	3	0
Gets to point quickly	6	3	0
Action messages are presented first	6	1	2
Message is likely clear	6	0	3
Sentences are short	2	7	0
Words are familiar/defined	4	2	3
Focuses on Need to Know	5	4	0
Steps in chronological order	9	0	0
Uses active voice	9	0	0
IllustrationsUsed and serve purposeClear and realisticEasy to understand	665	312	022

Based on User-Friendliness Tool and Suitability of Materials Assessment criteria.,

With the exception of text (i.e., written narrative in each respective language), layout features of English-and Spanish-language package instructions were identical (e.g., illustrations, text point size). Therefore, our layout feature assessment was limited to nine (n = 9) unique OTC step-by-step rapid antigen COVID-19 package insert instructions.

Understandability and actionability of OTC rapid antigen COVID-19 test video-based instructions.

Six OTC rapid antigen COVID-19 tests included English-language video-based step-by-step instructions on their product manufacturing website. English-language step-by-step OTC rapid antigen COVID-19 test video-based instructions (n = 6) ranged from 1:04 to 5:41 min in total. Four of six videos scored 100% on the PEMAT-A/V understandability scale (range = 80%–100%), while all videos scored 100% on the PEMAT-A/V actionability scale. As indicated by GQS scores, videos were of high quality (5 videos scored 5/5; 1 video scored 4/5).

Three videos were completely animated. Of these three, two videos included both voice-over and text-based instructions on each new screen. The third animated video did not include voice-over, but instead background music and text-based instructions on each new screen. Among non-animated videos (n = 3), cultural diversity and inclusiveness aspects were as follows: (1) African American woman preparing, performing, and interpreting COVID-19 test independently, (2) Asian woman preparing, performing, and interpreting COVID-19 test independently, and (3) White woman preparing and performing COVID-19 test independently, white woman performing nasal swab on another white woman, and white woman performing nasal swab on African American child.

One COVID-19 test product manufacturing website included Spanish-language video-based instructions (time = 4:59 min; PEMAT understandability = 100%; PEMAT actionability = 100%; GQS = 5). With the exception of Spanish-language narration, video content was identical to the English-language video.

Discussion

While rapid COVID-19 testing offers numerous advantages (e.g., identifying individuals with greatest potential clinical benefit, incentivizes early clinical presentation) for therapeutic antiviral clinical trial design, they also provide an individual with a relatively quick option to determine whether one is potentially infected with COVID-19. Our review of step-by-step OTC rapid antigen COVID-19 test package insert instructions revealed both positive health literacy-related features as well as aspects to improve upon in the future. OTC rapid antigen COVID-19 package instructions included many key established layout features to aid in promoting patient understanding. , Overall, most OTC rapid antigen COVID-19 package inserts needed little or some work to make instructions more user-friendly.

Another positive finding to emerge from our review was illustration use. Previous research has demonstrated that graphics (e.g., pictograms, photographs) promote patient/consumer understanding of instructions. , As such, one of the most encouraging findings to emerge from our study was that OTC rapid antigen COVID-19 package instructions averaged nearly twenty (n = 20) illustrations with the majority of those serving clear and realistic purposes while sized to promote reading ease based upon typical printed piece of paper dimensions (i.e., tabloid). However, consistent with previous studies evaluating reading demands and text point size of OTC package inserts,29, 30, 31 both English- and Spanish-language step-by-step OTC rapid antigen COVID-19 test package insert instructions did not meet the American Medical Association Foundation recommendation of presenting information at or below 6th grade reading level, while ensuring ≥12 point text size use as well.

As evidenced by PEMAT-A/V and GQS scores, six OTC rapid antigen COVID-19 tests included high quality English-language video-based step-by-step instructions on their product manufacturing website. Among the three non-animated videos, cultural diversity and inclusiveness was represented through presence of individuals of different ages, racial and ethnic backgrounds. This finding is emboldening as individuals portrayed in non-animated videos are reflective of the general US population.

One important finding to emerge—across both step-by-step OTC rapid antigen COVID-19 test package insert and video-based instructions—was lack of language availability beyond English. While four OTC rapid antigen COVID-19 tests included a Spanish-language step-by-step package insert, just one Spanish-language instructional video was available. Given the vast number of Spanish speakers in the US, this finding is significant and potentially limits accessibility to OTC rapid antigen COVID-19 testing.

Limitations

Our findings should be considered within the context of several limitations. Our review included a sample of OTC rapid antigen COVID-19 tests available for purchase, in the US, between September and December 2021. It is possible that some OTC rapid antigen COVID-19 tests were overlooked during this time period. However, all OTC rapid antigen COVID-19 tests were purchased from the leading US retail e-commerce company16. and several tests included in our review were identified as a best available at-home option. Three (27.3%) OTC rapid antigen COVID-19 tests in our sample did not have FDA EAU at the time of our review. One OTC rapid antigen COVID-19 test included in our review was recalled from the FDA due to potential false positive results. While two COVID-19 tests included English-language, graphic-based step-by-step instructions via a downloadable Smartphone application (app) exclusively, critical review of app-based instructions was beyond the scope of this study.

We did not directly assess consumer understanding instructions and/or demonstration of nasal swab collection and results interpretation. Instead, consistent with previously conducted studies, , 29, 30, 31 , we used established health literacy assessment tools (i.e., FRE, F–H, UFT, SAM, PEMAT-A/V) to gauge likelihood of consumer understanding and proper use of OTC rapid antigen COVID-19 tests. Direct assessment of patient/consumer comprehension of instructions—semi-structured interviews and/or focus groups—could potentially pinpoint content areas and formatting characteristics in need of refinement as future OTC COVID-19 test instructions are developed.

Conclusion

Overall, OTC COVID-19 test step-by-step materials—both package inserts and video-based instructions—included several positive features shown to foster patient/consumer understanding and ultimately facilitate proper use and results interpretation. In the future, greater attention needs to be placed on expanding both Spanish-language and video-based OTC COVID-19 test material availability to improve accessibility across diverse populations.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or non-for-profit sectors.

CRediT authorship contribution statement

Amy A. Fulmer: conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft preparation, writing—review & editing. George A. Abboud II: conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft preparation, writing—review & editing. Lorraine S. Wallace: conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft preparation, writing—review & editing, supervision.

Declaration of competing interest

Authors declare no known conflicts of interest.