Consumer Understanding, Preferences, and Responses to Different Versions of Drug Safety Messages in the United States: A Randomized Controlled Trial.

INTRODUCTION: As part of its mission, the US Food and Drug Administration (FDA) communicates with the public regularly about the benefits and risks of prescription and over-the-counter (OTC) drugs. Effectively communicating risk, however, is a significant public health challenge.
OBJECTIVE: To better understand how different populations understand information communicated by the FDA about drug safety, we conducted a randomized experiment to examine comprehension and other measures of effectiveness of drug safety messages that occurred in a post-market surveillance phase.
METHODS: We used an Internet panel survey of 1244 consumers, of whom 58% used prescription drugs in the past year. Half of the sample panel was randomized to read a previous FDA Drug Safety Communication (DSC) with the drug name changed, and the other half was randomized to read a revised version of the same DSC. We examined how making certain modifications to the way drug risk information is communicated has an impact on comprehension and behavioral intentions, including the user's likelihood of discontinuing the drug. We also studied how comprehension varied by respondent characteristics, health literacy skills, risk perceptions, and trust in the message.
RESULTS: Based on a five-item comprehension index, the revised version of the message was associated with significantly greater comprehension of the information relative to the standard version (63 vs 52% correct, p < 0.001). Significantly more respondents found the revised version to be clear (82 vs 73%, p < 0.000), while fewer in that group reported learning something new (78% vs 84%, p = 0.015). No significant differences emerged between the two groups in terms of the message being informative, convincing, or helpful. We found no significant differences between the two groups in terms of behavioral intentions, risk perception, and trust.
CONCLUSIONS: We found that making plain language changes to the DSC significantly increased consumers' level of comprehension of its content, providing support for ongoing use and further exploration of these strategies in pharmacovigilance communication research. The study findings have important implications for future drug safety and other communication messages related to prescription drugs.

RCT Entities:   Population Interventions Outcomes

Key Points

Introduction

Risk communication has been described as “one of the greatest challenges facing any public health agency” [1]. Risk communication has been used to communicate about risks related to public health topics such as periodic outbreaks of food-borne illnesses, the dangers of using tobacco products, and environmental degradation [2-4]. The process of communicating risk involves altering individuals’ perceived risk of negative consequences (or their mental models of risk) associated with adopting new behaviors, reducing risky behaviors, and increasing screening and treatment-seeking behaviors [5-9].

Risk communication is a core strategic function of the US Food and Drug Administration (FDA). The FDA’s mission is to promote, protect, and advance public health; in part, by assuring the safety, efficacy, and security of human drugs. The FDA serves the public interest best when consumers and healthcare professionals (HCPs) have timely and understandable information on the benefits and risks of marketed drugs needed to arrive at optimal treatment decisions [10]. The FDA focuses on communicating frequently and clearly about benefits and risks to help guide citizens in making choices about prescription and over-the-counter drugs, foods, medical devices, cosmetics, tobacco, and other products.

The public needs appropriate and timely communications about prescription drugs and the benefits and risks associated with their use, including risks that arise after the drugs are on the market and being prescribed. About 49 % of Americans have used at least one prescription drug in the past month, 31 % have used two or more drugs, and more than 76 % of people 60 years or older have used two or more drugs [11, 12]. To help ensure continued safe and effective use after the FDA approves a drug, the Agency continues to evaluate its benefits and risks. Much information is communicated about a drug throughout the product’s lifecycle. Some of these communication activities include prescribing information, product labeling, and package inserts; drug advertising and product claims; warning letters to manufacturers that violate laws governing advertising and product claims; adverse event reports made by consumers and HCPs; and postmarket Drug Safety Communications (DSCs) from the FDA.

A primary risk communication tool is the FDA DSCs, which provide emerging postmarket safety information about approved drugs, or ‘public pharmacovigilance communications’ [13]. DSCs are electronic communications in a standardized format that are posted in English and Spanish on the FDA’s website. DSCs are targeted to the general public, patients, and HCPs. Typically, they summarize the nature of the safety risk being addressed; present facts, including established indications and benefits of the drug; recommend actions for patients and HCPs; and summarize new data that the FDA has reviewed or is in the process of investigating. They are disseminated through multiple channels, including MedWatch Safety Alerts, targeted stakeholder e-mails and calls, FDA updates for Health Professionals and Drug Information listservs, FDA social media accounts, drug safety podcasts, and through other outlets. Some sources of information that feed into the DSCs include clinical trials and observational studies, systematic reviews, pooled analyses and meta-analyses, and spontaneous reports [14].

A systematic review of 49 published studies found that although some FDA drug risk communications (including advisory and label changes) had immediate and strong impact, many had either delayed or no impact on healthcare utilization or health behaviors [15]. Advisories recommending greater monitoring did not appear to lead to large and sustained changes in patient or prescriber behavior; and in some cases, the speed of adopting warning information varied depending on whether it was for new or continuing medication users [15]. In addition, the potential for unintended consequences from drug risk communications is an ongoing concern, with notable public health implications from messaging related to certain medications, such as birth control pills [16].

Several factors may impact the effectiveness of risk communications related to emerging drug safety issues. For example, a substantial proportion of Americans have low health literacy [17, 18], and people with low numeracy often err in their understanding of the benefits and risks of treatment, typically overestimating the benefits of treatment [19]. Certain strategies can facilitate more effective communication of risk information, including presenting information using absolute (versus relative) risk (or providing both), using frequencies (versus percentages), paying careful attention to time frames associated with the data (e.g., 10 years), and supporting data with visual cues such as graphics [19].

The primary objective of the present research is to understand how adults in the US differ with respect to their comprehension of, as well as their need and preferences for, emerging safety information about prescription drugs. We examine the types of information about prescription drugs consumers prefer and report using, and to what extent they are able to understand benefit and risk information in general, as well as in the context of a specific sample communication they received. The outcomes of exposure to the sample drug safety message we assessed include the extent to which consumers are likely to talk with their healthcare professional, report symptoms, and discontinue taking the medication. Using a randomized design, we also tested how making modifications in the way risk information about a drug is communicated using clear communication and health literacy principles can have an impact on comprehension, risk perceptions, behavioral intentions, message receptivity, and trust.

Data and Methods

Study Design

The study used an Internet panel survey to collect measures of consumer perceptions about emerging drug safety information and the effects of modifying a sample DSC on a specific drug safety issue. Half of the study sample was randomized electronically to receive a previously developed drug safety message about a fictitious drug used for smoking cessation; the other half of the sample was exposed to a revised version of the same message (see electronic supplementary material 1 for additional information about the sample allocation process). Study investigators were blinded to the randomization process. Inclusion criteria included ensuring that at least half of participants or a family member used a prescription drug in the past year and that at least one-third of the participants had no more than a high-school education. These and other sociodemographic characteristics were also considered in developing the weight variable. Sample size was determined based on power calculations for the primary outcomes with at least a 90 % power for detecting small effects (Cohen’s d = 0.2). Human subjects approval was obtained for this research. No changes were made to the study design after the study commenced.

Intervention

For the previous version of the drug safety message (‘Standard version’), we used a DSC released online in a single-page, long-form format before FDA instituted the multi-tabbed DSC format, with more plain language and other general content modifications. When creating the revised version, the drug name was changed to ‘Smoquit,’ a fictitious name. For the revised version of the drug safety message (‘Revised version’), we applied plain language [20], clear communication [21], and health literacy principles to test whether the changes would improve readability and uptake of the information, or influence other measures. The reading level of the Standard version was grade 11 and for the Revised version it was grade 8, using the Simple Measure of Gobbledygook (SMOG) test [22]. Table 1 presents a side-by-side comparison of various elements of the Standard and Revised versions of the message.

Table 1

Side-by-side comparison of Standard version and Revised version of 2012 FDA Drug Safety Communication (DSC)

Standard version of the DSC	Revised version of the DSC
Title: FDA Drug Safety Communication: Safety review update of Smoquit and risk of cardiovascular adverse events	Title: Talk with your health care professional if you are taking Smoquit and have new or worsening symptoms of heart or blood-vessel disease
What is Smoquit?	What is Smoquit?
A prescription drug used to help adults quit smoking that works by blocking the effects of nicotine from smoking on the brain.It increases the likelihood of abstinence from smoking for as long as one year compared to treatment with a placebo	Smoquit is a non-nicotine prescription medicine that—along with quit smoking materials and/or programs—helps people 18 and older stop smoking
	Looking at the best evidence
The U.S. Food and Drug Administration (FDA) is informing the public about the results of a large, combined analysis, also called a meta-analysis, of clinical trials that compared patients who received the smoking cessation drug Smoquit to patients who received a placebo, which is a treatment with no drug in it	The Food & Drug Administration (FDA) asked the drug company that makes Smoquit to review all of the large and well done studies of Smoquit
FDA required the manufacturer of Smoquit to conduct the meta-analysis to further evaluate the cardiovascular safety of the drug, and believes it is important to let health care professionals and patients know about the results of this study	FDA wanted to better understand the effect Smoquit has on heart and blood-vessel health, also called cardiovascular health
FDA first notified the public about a possible increased risk of cardiovascular adverse events with Smoquit in its June 2011 Drug Safety Communication (DSC)	All of the studies compared people who were taking Smoquit to people who were taking a sugar pill that contains no drug, also known as a placebo
	What did they find?
A higher occurrence of major adverse cardiovascular events was observed in patients using Smoquit compared to placebo. Major adverse cardiovascular events were defined as a combined outcome of cardiovascular-related death, nonfatal heart attack, and nonfatal stroke	Looking at the combined results of all the studies, people taking Smoquit were more likely than people taking placebos to have had one or more of the following heart-related problems:• death related to cardiovascular problems;• non-deadly heart attacks; and• non-deadly stroke
These events were uncommon in both the Smoquit and placebo groups, and the increased risk was not statistically significant, which means it is uncertain whether the excess risk for the Smoquit group was due to the drug or due to chance	The chance of having a heart-related problem was rare in both groups. The chance of someone having a heart-related problem if they took Smoquit was 31 in 10,000 (0.31 %). A person taking a placebo had a 21 in 10,000 chance (0.21 %) of having a heart-related problem. This difference in having heart-related problems could be due to chance
However, the data were analyzed many different ways and consistently showed a higher occurrence of events in patients using Smoquit, which makes it seem more likely that it is related to the drug and not purely a chance finding	But FDA suspects that these heart problems may be due to Smoquit. FDA believes this because people taking Smoquit were consistently more likely to have these heart problems than people taking placebos
The meta-analysis findings of cardiovascular risk are similar to the findings in the smoking cessation clinical trial of patients with stable cardiovascular disease that was described in FDA’s June 16, 2011 DSC. The Warnings and Precautions section of the Smoquit label has been updated to include the results of the meta-analysis	The makers of Smoquit have updated the Warnings and Precautions on the medicine’s label to include this new information
	How does this affect me?
Patients taking Smoquit should contact their health care professional if they experience new or worsening symptoms of cardiovascular disease, such as chest pain; shortness of breath; calf pain when walking; or sudden onset of weakness, numbness, or difficulty speaking	Patients: The health benefits of quitting smoking are immediate and substantial. Talk to your health care professional if you are taking Smoquit and have any new symptoms of heart and blood-vessel disease, or if your condition seems to be getting worse. These symptoms include:• chest pain;• shortness of breath;• calf pain when walking; or• suddenly feeling weak, numb, or having difficulty speaking
Report any side effects you experience to your health care professional and the FDA MedWatch program. Patients should also contact their health care professional if they have any questions or concerns about Smoquit	Report any side effects that you have to your health care professional and the FDA MedWatch program. You should also contact your health care professional if you have any questions or worries about Smoquit
Health care professionals are advised to weigh the risks of Smoquit against the benefits of its use. It is important to note that smoking is a major risk factor for cardiovascular disease, and Smoquit is effective in helping patients to quit smoking and abstain from it for as long as one year	Health Care Professionals: Help your patients weigh the potential risks and benefits of using Smoquit. Smoking is a major risk factor for cardiovascular disease
The health benefits of quitting smoking are immediate and substantial. Report adverse events involving Smoquit to the FDA	Smoquit can help patients to quit smoking and keep from smoking for as long as one year. Report problems involving Smoquit to the FDA
Data Summary. Overall, there was a low incidence of major adverse cardiovascular events occurring within 30 days of treatment discontinuation (Smoquit 0.31 % [13/4190] vs. placebo 0.21 % [6/2812]) in the trials included in the meta-analysisNote: This section called ‘Data Summary’ was under a tab directed at Health Care Professionals and was not included in the Consumer/Patient tab

Briefly, the Revised version retained the same general content as the Standard version but used the active voice, made behavioral recommendations more action oriented, used less complex language, ‘chunked’ information with additional subheadings, and provided both quantitative and qualitative data with appropriate context and explanation. For example, the text describing a meta-analysis was revised to use simpler terminology and given a plain language subheading called ‘Looking at the best evidence.’ Language used in the Standard version was reworded in the Revised version and was also added to the main safety announcement to explain to patients the likelihood of experiencing an adverse event when taking the medication. The Standard version reported these numbers only in the data summary section aimed at healthcare professionals, stating that “there was a low incidence of major cardiovascular events occurring within 30 days of treatment discontinuation (Smoquit 0.31 % [13/4190]).” The Revised version was reworded to state “the chance of someone having a heart-related problem if they took Smoquit was 31 in 10,000 (0.31 %).”

Data Collection and Panel Survey

Data were collected using a KnowledgePanel® [23] probability-based Internet sample that is designed to be representative of US households. GfK, the developer of KnowledgePanel, uses address-based sampling of a computerized file updated every 2 months that contains all delivery point addresses serviced by the US Postal Service (over 125 million records with 97 % coverage of US households). This approach reduces sampling biases that are introduced through the use of random digit dialing and Internet-based sampling methods because it includes non-telephone and non-Internet households.

Individuals who agree to be on the panel can use their own computers connected to the Internet to take surveys; netbooks and Internet access are provided to panel members living in non-Internet households. Panel ‘case managers’ provide telephone support to households that require help connecting their computers to the Internet, accessing their e-mail, and accessing and responding to Internet surveys. KnowledgePanel consists of about 50,000 US residents aged 18 or older, and includes people living in cell-phone-only households. Numerous internal and independent assessments of the representativeness of KnowledgePanel samples have been conducted; no evidence of selection bias or bias attributable to time in the panel were identified [24].

The survey was fielded between July 24 and August 20, 2013.

Measures

The comprehension and behavioral intention questions were specifically developed to align with the intervention for this study. The health literacy items are from a validated scale. All measures were prespecified and were not changed once data collection commenced.

Dependent Variables

Comprehension We measured respondents’ comprehension of the Smoquit DSC using five survey items (α = 0.70) developed specifically for this study and based on the content included in the DSC. The questions examined respondents’ awareness and knowledge about the following: (1) How common are major cardiovascular or heart-related events? (2) Who is most likely to have heart-related problems? (3) When should patients taking Smoquit contact their healthcare professional? (4) What is the likelihood of experiencing an adverse event? and (5) What are FDA’s recommendations for patients taking Smoquit? Each survey item had a single correct answer; consequently, we created a five-item ‘comprehension index’ reflecting the proportion of correct answers to these questions. Respondents were told at the end of the survey that the message was about a fictitious drug.

Message Assessment, Risk Perception, and Trust of the Source Respondents were asked five questions about the utility of the DSC; specifically, whether it was (1) clear, (2) informative, (3) convincing, (4) helpful, and (5) did they learn something new. These questions used a 1–4 response scale, with 1 = Strongly disagree and 4 = Strongly agree. We summed the responses to these items to create a 5-item ‘message assessment’ scale (α = 0.91). We also asked respondents about how much they trust the information in the DSC and how much risk they think there is for a person with heart or blood vessel disease who is taking Smoquit (using a 4-item response scale of 1 = None, 2 = Some; 3 = A fair amount; 4 = A lot, with a ‘Don’t know’ option). The focus for this last question was on individuals who already have heart or blood vessel disease because they are at increased risk.

Behavioral Intentions The survey included six questions about respondents’ intended behaviors/actions as a result of being exposed to the DSC; that is, how likely would they be to (1) talk with their HCP about Smoquit, (2) report any symptoms described in the message to their HCP, (3) report any symptoms described in the message to the FDA, (4) look for more information about the medicine, (5) discontinue taking the drug. The responses to these questions each had a 1–7 scale with end points labeled 1 = Strongly disagree and 7 = Strongly agree. The sixth question asked if a respondent would take Smoquit to try and stop smoking if their doctor prescribed it; responses were on a 1–4 scale with end points labeled 1 = Strongly disagree to 4 = Strongly agree.

Independent Variables

Sociodemographic, health insurance, and geographic data were collected from respondents as part of their participation in KnowledgePanel. Participants were classified by GfK as to whether they had Internet access or were given it as a part of their panel participation.

Sources of Prescription Drug Information and Information Preferences To assess what information sources people consult about prescription drugs, we asked the following question: “Before taking a prescription drug, what information source(s) would you use to learn about it?” A total of 20 different response options were provided in the following categories: (1) People (such as doctor or pharmacist), (2) Traditional media (such as radio or magazine), (3) Internet or mobile sources, and (4) Other.

To assess what kind of information people want about prescription drugs, we asked the following question: “When you are looking for information about prescription drugs, what kind of information do you want?” Response options included ‘General use of the medicine,’ ‘Safety information,’ ‘Possible side effects,’ ‘Dosage information or how much to take,’ ‘What it is best used for,’ ‘I don’t look for that information,’ and ‘Other.’

Health Literacy To evaluate health literacy level, we used a five-item version of the Health Literacy Skills Instrument (HLSI) [25, 26], a computer-based instrument that measures a range of health literacy skills in the general population (α = 0.75). This five-item version of the HLSI focuses largely on numeracy-related skills. Respondents use real-world stimuli such as charts, tables, maps, and other images to answer questions about a range of health-related topics. Each question has only one correct response option, and scale scores are computed as the percentage of correct responses ranging from 0 to 100.

Prescription Drug Utilization Prescription drug use was assessed with one item: “In the past 12 months, have you taken a prescription drug? Examples of prescription drugs include antibiotics, antidepressants, and insulin.”

Statistical Methods

Chi-square tests were used to determine if the two DSC groups (Standard version versus Revised version) had comparable demographic characteristics. Responses between the two groups on all dependent and independent variables were compared using Chi-square tests for categorical variables and t-tests for continuous variables. Finally, a linear regression model was used to compare comprehension index scores based on DSC condition while controlling for other factors that may have an impact on comprehension, including demographics (i.e., gender, age, education level, race/ethnicity, income, insurance, and geographic region), household Internet access, prescription drug use in the past 12 months, health literacy, risk perceptions, and trust in the message. We also ran an analysis to determine the proportion of individuals who did not accurately respond to the question about overall risk of Smoquit and who also did not answer the question correctly about the overall risk of Smoquit. Analyses were conducted using SAS version 9.3 survey analysis procedures, and survey weights were applied to represent the national population and adjusted for differential nonresponse. The difference between weighted and unweighted frequencies for all variables was no more than 4 percentage points.

Results

Participants

The survey completion rate was 56 %. A total of 1244 participants completed the survey (n = 620 Standard version; n = 624 Revised version) (see Table 2); their demographic characteristics were similar to those of the US population. The majority of respondents were female (51 %) and White (66 %). Respondents varied in their level of education and mirrored the US population: 12 % had less than a high school education, 31 % had a high school education, 29 % had some college, and 29 % had a college education or more. More than half of the participants (58 %) reported using a prescription drug during the past 12 months (though recall bias may exist), and about three quarters (76 %) of them had access to the Internet other than what was provided by being part of the KnowledegePanel®. At some point in time, 12 % had tried a smoking cessation drug. The mean health literacy score was 78.4 (SD = 29.1), which is similar to the population mean found in previous studies [25, 26]. The health literacy index had good internal consistency, with a Cronbach’s alpha of 0.75. No significant differences in demographic characteristics were detected between respondents by DSC condition.

Table 2

Demographic characteristics of respondents to the Smoquit survey, by type of message condition

Characteristic	Original message		Revised message		p value
	N	Weighted %	N	Weighted %
Male	318	48	314	48	0.996
Age (years)
18–29	107	21	104	21	1.000
30–44	138	25	138	26
45–59	178	27	192	27
60+	201	26	186	26
Education
Less than high school	63	12	70	12	1.000
High school	201	31	200	31
Some college	171	29	166	29
College	189	29	184	29
Race/ethnicity
White	454	66	426	66	1.000
Black	67	12	74	12
Hispanic	66	15	87	15
Other	37	8	33	7
Income
<US$30,000	147	24	129	24	0.890
US$30,000–US$59,999	160	26	181	28
US$60,000–US$99,999	152	24	156	22
US$100,000+	165	26	154	27
Region
Northeast	111	18	109	18	1.000
Midwest	141	21	134	21
South	238	37	242	37
West	134	23	135	23
Household internet access	458	76	472	76	0.989
Medication use in past 12 months	373	58	373	59	0.620
Ever taken smoking cessation drug	62	10	85	15	0.019
Health literacy, mean (SE)	75	30	76	30	0.712

Prescription Drug Information Seeking

When seeking information about prescription drugs, 80 % of respondents reported that they looked for information about possible side effects, 70 % for dosage information, 63 % about safety information, 60 % about general use of the drug, 57 % for what the drug is best used for, 5 % wanted some other kind of information, and 8 % would not look for information (see Fig. 1).

Fig. 1

Types of information consumers want about prescription drugs (N = 1244)

The most frequently cited sources for information about prescription drugs were doctors or HCPs (75 %), pharmacists (61 %), and drug package label or insert (51 %) (see Table 3). The least frequently cited sources included social network sites (1 %), the radio (2 %), and newspapers (5 %). A total of 9 % of participants reported that they would go to a government agency website for information about prescription drugs.

Table 3

Most and least frequently cited sources for information about prescription drugs

Most frequently cited	% (n)	Least frequently cited	% (n)
A doctor or healthcare professional	75 (944)	Social networking sites	1 (15)
A pharmacist	61 (769)	Radio	2 (26)
Medicine package label or insert	51 (663)	A newspaper	5 (68)
Medical/health website	35 (437)	A nonprofit organization website	6 (70)
Internet search	31 (368)	Online forum or discussion group	6 (78)
A family member	22 (249)	A magazine	6 (75)
A friend or coworker	14 (153)	A government agency website	9 (109)

N = 1244

Differences in Comprehension by Drug Safety Communication Condition

The Cronbach’s alpha for the 5-item comprehension index was 0.70, and the index scores followed a normal distribution. Based on the bivariate analyses, respondents who received the Revised version of the DSC answered more comprehension questions correctly (63 %) compared with respondents who received the Standard version (52 %) (p < 0.001). The results of a series of chi-square tests comparing correct answers to each individual knowledge question indicated that respondents who received the Revised version were significantly more likely than respondents who received the Standard version to correctly identify the number of people having heart-related problems after taking Smoquit if the chance was 0.31 % (63 vs 28 %), who was most likely to experience heart-related problems (56 vs 46 %), and how common major cardiovascular problems are for people taking Smoquit (59 vs 48 %) (see Table 4).

Table 4

Comprehension, message receptivity, and behavioral intentions by message condition

Item	Original message	Revised message	Revised vs original
Comprehension	% correct	% correct	RR (95 % CI)
Overall comprehension index score, mean (SE)	52.46 (1.49)	63.20 (1.66)	10.74 (6.37–15.11)
How common are major cardiovascular or heart-related problems for people taking Smoquit? Very common Somewhat common Somewhat rare Very rare Don’t know	48	59	1.21 (1.07–1.37)
Who is most likely to have heart-related problems? People taking a placebo (a sugar pill) People taking Smoquit People taking a placebo and those taking Smoquit are equally likely to have heart problems There is not enough information to answer this question Don’t know	46	56	1.21 (1.06–1.38)
People taking Smoquit should contact their healthcare professional if they have… (check all that apply) Knee pain Arm pain Calf pain Back pain Don’t know	59	56	0.95 (0.84–1.06)
The chance of someone having a heart-related problem if they took Smoquit was 0.31 %. How many people does this mean had heart-related problems after taking the medicine? 31 in 100 31 in 1,000 31 in 10,000 1 in 31,000 Don’t know	28	63	2.22 (1.88–2.62)
What is the recommendation for people taking Smoquit? Continue taking the medicine but cut back on the dosage Talk to your healthcare professional if you have new or worsening symptoms of cardiovascular or heart disease Look for more information to see if you should continue taking the medicine Contact the makers of Smoquit to see if you are at risk of developing cardiovascular disease Don’t know	79	79	1.01 (0.94–1.09)
How much risk do you think there is for a person with heart or blood vessel disease who is taking Smoquit?*			0.92 (0.81–1.03)
None	2	0
Some	37	41
A fair amount	29	26
A lot	21	21
Don’t know	12	11
How much do you trust the information in the message?*			0.98 (0.88–1.10)
None	7	6
Some	28	30
A fair amount	38	37
A lot	17	17
Don’t know	11	10

* Relative risk compares participants reporting a lot/a fair amount vs none/some

Respondents recognized there was risk involved in taking Smoquit for those with heart or blood vessel disease, with 21 % in each condition perceiving ‘a lot’ of risk (see electronic supplementary material 2 for additional information about the subset of respondents reporting ‘a lot’ of risk). Trust levels varied, with 17 % of respondents indicating that they trusted the information in the DSC ‘a lot.’ No statistically significant differences existed in terms of risk perceptions or trust between the two experimental conditions.

Respondents who received the Revised version of the DSC scored, on average, 10 percentage points higher on the comprehension index than respondents who received the Standard version after controlling for other factors (p < 0.001; see Table 5). Greater comprehension was associated with being White compared with being African American (p < 0.014), having no health insurance (p = 0.004), higher health literacy scores (p < 0.001), and greater trust in the Smoquit message (p < 0.001). Lower comprehension was associated with higher risk perceptions for heart or blood vessel disease. Gender, age, education, income, geographical region, Internet access, and medication use were not significantly associated with greater comprehension.

Table 5

Regression model of comprehension index score

Variable	Comprehension
	Coefficient (SE)	p value
Message
Original message	REF
Revised message	10.11 (1.78)	<0.001
Male	−0.19 (1.84)	0.918
Age
18–29	REF
30–44	−3.07 (2.94)	0.297
45–59	1.12 (2.89)	0.699
60+	0.90 (3.17)	0.776
Education
Less than high school	REF
High school	2.10 (3.70)	0.571
Some college	3.55 (4.00)	0.376
College	4.84 (4.01)	0.227
Race/ethnicity
White	REF
Black	−7.92 (3.21)	0.014
Hispanic	1.81 (3.25)	0.579
Other	0.50 (3.84)	0.897
Income
<US$30,000	REF
US$30,000–US$59,999	0.99 (2.86)	0.730
US$60,000–US$99,999	3.33 (3.08)	0.280
US$100,000+	4.19 (3.16)	0.185
Insurance
Medicaid	−0.21 (3.18)	0.948
Medicare	0.17 (2.77)	0.951
Employer	REF
Other insurance	−3.96 (5.89)	0.501
No insurance	9.14 (3.17)	0.004
Unknown	−1.12 (3.04)	0.712
Region
Northeast	−1.25 (2.91)	0.668
Midwest	2.91 (2.65)	0.273
South	1.63 (2.43)	0.503
West	REF
Household Internet access	1.79 (2.66)	0.501
Medication use in past 12 months	2.99 (2.03)	0.142
Health literacy	0.40 (0.04)	<0.001
Risk perceptions	−6.30 (1.20)	<0.001
Trust in information	4.47 (1.05)	<0.001
Drug to stop smoking	1.78 (2.55)	0.487
Knowledge index	–	–

Comprehension model (N = 1004; R 2 = 0.33)

REF reference category

Differences in Message Assessment, Risk Perceptions, Trust, and Behavioral Intentions by DSC Condition

The five-item message assessment scale had a Cronbach’s alpha of 0.91, suggesting good reliability. On a scale of 1–5, the mean score was 2.9 for both DSC conditions. Among respondents, 82 % in the Revised version group agreed that the message was clear as compared with 73 % in the Standard version group (p < 0.001). However, significantly fewer respondents who received the Revised version (78 %) said they learned something new from it, compared with 84 % of respondents who received the Standard version (p = 0.015). No significant differences were found between respondents who viewed the Standard version compared with the Revised version with regard to the message being informative (86 %), convincing (74 %), or helpful (84 %).

No significant differences between groups were found on any of the behavioral intentions measures, as shown in Table 4. Respondents reported that they were fairly likely to talk to their HCP about the drug after seeing the message (mean of 5.4), report any symptoms described in the message to their HCP (mean of 5.9), and look for more information about the drug (mean of 5.4). Across both groups, respondents were less likely to report symptoms described in the DSC to the FDA (mean of 4.3) and similarly rated their likelihood of discontinuing taking the drug after reading the DSC (mean of 4.7). Both groups were also equally likely to rate their likelihood of taking Smoquit if they were trying to stop smoking and their doctor prescribed it (mean of 2.5 out of 4.0).

Discussion

This study examined how changes to a complex risk communication message about drug safety (DSC) can have an impact on consumers’ ability to understand and use the information. The strategies included using plain language, best practices in health literacy, and clear communication principles, such as simplifying the reading level of the DSC by using shorter sentences and words with fewer syllables; using format and design modifications such as subheading levels and more white space; and including numeric information in sections geared toward lay audiences and providing assistance with interpreting it. We found that making these changes to the DSC significantly increased consumers’ level of comprehension of its content, providing support for ongoing use and further exploration of these strategies in pharmacovigilance communication research. Comprehension among respondents exposed to the Revised version of the DSC about a smoking cessation medication was higher regarding potential safety issues, side effects, and who is more likely to experience potential adverse events.

While the changes made to the DSC had a significant impact on consumers’ comprehension level, they had a limited or no impact on respondents’ assessment of the message or behavioral intentions. Participants found the revised message to be significantly clearer; this may be due to the reduced reading level and elimination of passive voice, in particular. Additional changes to the format and content of the message may be needed to have a greater impact. While the content was the same in both versions, respondents perceived that they learned more from the Standard version, perhaps due to its use of more complex terminology.

A consumer’s health literacy level was a key factor in respondents’ level of understanding of the DSC examined in this study. This may be because the measure of health literacy that was used in this study, the HLSI, includes measures of numeracy that may have been particularly relevant because the online survey also asked participants to answer questions containing numeric information, including one question about the chance that someone would have a heart-related problem if they took the medication. The question involved interpreting how many people are in a fractional percentage (0.31 %). Respondents who viewed the Revised version, which explained what that percentage means using a frequency (31 out of 10,000), were more than two times as likely to correctly answer the question (63 vs 28 %) compared with those who viewed the Standard version. Natural frequencies are better understood than percentages in the context of diagnostic or screening tests [27], and this may be the case for other risks as well. The Standard version included the numeric information only in the Data Summary section geared to HCPs because of concerns at the time of its development about the prevalence of low numeracy levels among general health consumers. The present findings suggest that consumers can understand numerical information when adequate context is provided. Other research has also found that people better understand probabilities when they are presented with words and/or visuals that match and reinforce the meaning of the numbers versus when numbers are presented alone [21, 28], but fractional percentages should be avoided [29]. Visual aids and absolute risk formats can improve patients’ understanding of probabilistic information, whereas ‘numbers needed to treat’ can lessen their understanding [30]. However, graphics can be subject to interpretation, sometimes leading to overestimation of risk [31].

An unforeseen finding was the higher-than-expected overall mean likelihood (mean = 4.7 on a 7-point scale) of stopping the drug regardless of the DSC version to which they were exposed (this is in comparison to mean = 2.5 on a 4-point scale of those who reported they would take Smoquit if trying to stop smoking). The respondents were hypothetical users of this fictitious medication and a majority were non-smokers; however, this finding supports prior research suggesting that a better understanding is needed about the extent to which, and for what other types of behaviors, pharmacovigilance communications might unintentionally lead to drug nonadherence and other issues, and how to overcome these potential consequences. For example, further research is needed to assess if consumers would talk to their HCP about their concerns before discontinuing a drug after seeing or hearing safety messages from different sources.

Both the Standard and the Revised versions of the communication stated that smoking is a risk factor for cardiovascular disease and that Smoquit is used to help people stop smoking and it increases the likelihood of abstinence from smoking for as long as 1 year. However, neither version directly presents other health risks of smoking or the other benefits of quitting, or quantifies the cardiovascular risk of smoking. Adherence to a medicine is likely to be motivated by total risks and total benefits associated with the medicine. Total benefits from taking Smoquit include those associated with taking the medicine as well as those associated with quitting smoking. However, such total analyses of risks and benefits are not included in most methodologies to assess drug safety and efficacy [32]. Lay audiences understand risk in multifaceted ways involving statistical probability, with most viewing it as “hazard or peril [33]”. Understanding of risk can be influenced by “whether the risk is incurred voluntarily (e.g., smoking) or involuntarily (e.g., contaminants in the environment), by how emotions color perceptions, and by how optimistic or pessimistic the person is” [34, p. 86]. People also often use heuristics or shortcuts to make judgments and base decisions such as whether to talk to a doctor about side effects or discontinue taking a medication [35]. Our findings provide empirical support for the recommendation that message developers should seek to reduce cognitive burden by presenting messages with a modest amount of information and key points that can be easily identified within an organized and clean layout.

The randomized design and large sample size are key strengths of the study. However, limitations of the study are that we used a single hypothetical message for a single drug with a population that was not actually taking the medication; as a result, there were no behavioral outcomes to assess. With actual patients taking the drug under study and the ability to measure behaviors, not just intentions, over time, the findings may have been different. The data are based on self-report, respondents could have been influenced by social desirability bias, and the absolute change in understanding was small. These small changes may be reliable, or they could be the result of ceiling effects, as between 74 and 86 % endorsed how informative, convincing, or helpful the information was for them. The behavioral intention survey measures were developed based on the information in the DSC but may be somewhat generic, and the question about how ‘convincing’ the information was could be viewed as vague, especially given the messages did not promote any specific recommended immediate-term actions. The direction of the effect in the comprehension model cannot be determined; that is, did risk and trust affect comprehension or did comprehension affect risk and trust? Taken together, these limitations can have a potential impact on the internal validity and generalizabilty of the study.

Information on US government health websites is required to adhere to plain language principles [36]. Undertaking the process of transforming scientific information into plain language is complicated and requires collaboration among subject matter experts, legal experts, and communication specialists, as well as assurance that those with the authority to conduct the final review of the content do not ‘undo’ the plain language implemented by those who contributed to it before them. Some federal agencies are implementing guidelines and processes for ensuring that their public-facing content is understandable [21]. Despite guidelines, developing plain language content can be challenging and every word can matter. For example, when revising the ‘What did they find’ section of the Smoquit DSC, the word ‘consistently’ may only make sense if one knows that there were various analyses conducted, a fact the Revised version did not explicitly state but was included in the Standard version. Changes in expression, syntax, or grammar in the search for greater simplicity and broader understanding could also potentially influence meaning or emphasis and consequently the outcomes.

Approval of a new drug is given only after rigorous review; however, new research and other information that becomes available after a drug is approved and in wide use may reveal additional side effects, new indications for its use, potential links of use of the drug with adverse events (fatal and nonfatal), restrictions in populations that should use it, or changes in prescription dose that must be communicated to patients, caregivers, and HCPs. Responding to the changing science and evidence for the safety and efficacy of approved drugs is a major challenge for public pharmacovigilance communications because this involves crafting relevant, comprehensible, and actionable drug safety messages for various stakeholders. Consequently, the FDA and similar agencies worldwide must continually raise awareness and provide reasonable guidance to patients, caregivers, and HCPs that informs rather than causes unnecessary anxiety. For example, Health Canada, the federal department responsible for maintaining and improving the health of Canadians, is identifying, assessing, and communicating safety information to Canadians. LeBrun and colleagues [37] recently found that implementing a revised Public Advisory template using plain language principles reduced the literacy burden based on the Suitability Assessment Test. Likewise, the FDA’s Center for Drug Evaluation and Research is funding research aimed at better understanding the needs of various audiences and using this and other evidence-based information to enhance the effectiveness of its drug-related communications.

Conclusions

This study provides quantifiable evidence that incorporating clear communication and health literacy principles into risk communication information can significantly increase consumers’ understanding of drug safety information. This suggests that greater effort should be made to apply such modifications to highly technical scientific and regulatory topics, especially when safety issues are involved. Additional research is needed using qualitative and quantitative techniques to assess ongoing changes to drug safety messages and how communication activities may be dependent on contextual and personal variables of the consumers, patients, HCPs, and other stakeholders. Messages about drug safety are clearly different than other types of risk communications related to natural disasters or other man-made risks and should be approached with this caveat in mind.

Supplementary material 1 (PDF 88 kb)

Supplementary material 2 (PDF 43 kb)

More use of plain language, and clear communication changes to drug safety communications may increase consumers’ level of comprehension of the content.

Consumers’ health literacy levels were a key factor in respondents’ level of understanding of the drug safety communication.