Public Perspectives of Using Social Media Data to Improve Adverse Drug Reaction Reporting: A Mixed-Methods Study.

INTRODUCTION: Information on suspected adverse drug reactions (ADRs) voluntarily submitted by patients can be a valuable source of information for improving drug safety; however, public awareness of reporting mechanisms remains low. Whilst methods to automatically detect ADR mentions from social media posts using text mining techniques have been proposed to improve reporting rates, it is unclear how acceptable these would be to social media users.
OBJECTIVE: The objective of this study was to explore public opinion about using automated methods to detect and report mentions of ADRs on social media to enhance pharmacovigilance efforts.
METHODS: Users of the online health discussion forum HealthUnlocked participated in an online survey (N = 1359) about experiences with ADRs, knowledge of pharmacovigilance methods, and opinions about using automated data mining methods to detect and report ADRs. To further explore responses, five qualitative focus groups were conducted with 20 social media users with long-term health conditions.
RESULTS: Participant responses indicated a low awareness of pharmacovigilance methods and ADR reporting. They showed a strong willingness to share health-related social media data about ADRs with researchers and regulators, but were cautious about automated text mining methods of detecting and reporting ADRs.
CONCLUSIONS: Social media users value public-facing pharmacovigilance schemes, even if they do not understand the current framework of pharmacovigilance within the UK. Ongoing engagement with users is essential to understand views, share knowledge and respect users' privacy expectations to optimise future ADR reporting from online health communities.

Key Points

Introduction

An adverse drug reaction (ADR) has been defined as “an appreciably harmful or unpleasant reaction, resulting from an intervention related to the use of a medicinal product” [1]. Adverse drug reactions constitute a significant problem, causing up to 10% of all hospital admissions across Europe [2] and affecting the well-being and quality of life among many more.

In the UK, post-marketing drug safety is monitored and regulated by the Medicines and Healthcare products Regulatory Agency (MHRA). One important source of drug safety information is the MHRA’s Yellow Card scheme [3], which collects information on suspected ADRs voluntarily reported by healthcare professionals and members of the public. Information about the scheme is provided via promotional materials, training and patient information leaflets. Similar systems exist in other countries [4].

Despite the Yellow Card scheme being open to the public since 2005, there still remains significant under-reporting of ADRs. Low public awareness may be part of the problem [5, 6]. A study in 2009 found that fewer than one in ten people had heard of the scheme and, even when made aware, people expressed uncertainty about the value of their reports to the MHRA [7]. Additionally, the process, submitting reports can be done online via the website [3] or using a mobile application [8], by telephone or via a paper form, may be too time consuming [6]. Consequently, a significant proportion of adverse reactions is likely to go unreported, missing a vital learning opportunity.

Increasingly, researchers have been exploring whether social media data can be used as a pharmacovigilance tool, including detecting ADRs [9, 10]. Social media data arguably offer much promise as a real-time ‘naturalistic’ source of data about experiences of health and medication. Indeed, one survey found that 35% of American adults reported searching the internet for help with health-related concerns and 16% tried to connect with others with similar concerns [11]. Tapping into such information therefore has the potential to provide valuable information about ADRs.

A growing number of studies, particularly in the computer science field, have demonstrated the feasibility of identifying ADRs from social media posts using automated text mining techniques [12-15]. While certain types of social media data might be considered publicly accessible, there have been important ethical debates regarding privacy expectations, user autonomy and the potential benefits and harms of usage [16-18]. Furthermore, it is not yet known whether implementing text mining techniques, alone or alongside other methods, would be acceptable to social media users.

The study aimed to: (a) determine experiences and levels of knowledge regarding medication safety and reporting among users of health-related social media platforms and (b) explore public views on using health-related social media data to increase reporting of ADRs for pharmacovigilance purposes via various means, including using automated methods that involve text mining.

Methods

A sequential explanatory mixed-methods design was used [19]. A quantitative online survey was used to assess the breadth of understanding, opinions, and experiences relevant to pharmacovigilance and ADRs among a large cross-section of online health discussion forum users. This was followed by a series of qualitative focus groups to explore the opinions and experiences expressed in the survey and address some of the more complex concepts, including text mining.

Online Survey

The survey was conducted in collaboration with HealthUnlocked, a large, internationally used, online social network for health. Reporting an average of 4.5 million visits per month, HealthUnlocked provides access to resources, support, and communities relevant to health conditions and well-being concerns [20]. Users can join over 700 condition-specific communities within the platform that focus on different conditions and patients. Communities vary in size; some of the larger communities have memberships in the tens of thousands [20], particularly those associated with common conditions or run by major health charities, for example, the National Rheumatoid Arthritis Society (N > 29,000), Asthma UK (N > 16,000) and the Anxiety and Depression Association of America (N > 50,000). The platform has also been used for scientific purposes, including conducting research into the dynamics among online communities for people with long-term conditions [21], describing the representativeness of users compared to wider patient populations [22] and involving patients in the drug development process [23].

A 20-item questionnaire was drafted based on conversations with HealthUnlocked staff, clinicians and the MHRA. We worked with the HealthUnlocked team and a local Manchester-based musculoskeletal patient involvement group to iteratively develop appropriate phrasing and format. The final version comprised four sections relevant to ADRs and medicines regulation: health and experience with medications; reporting side effects; the Yellow Card scheme; and demographics. Multiple choice, binary (yes/no) and free text items were included (see Appendix 1 of the Electronic Supplementary Material [ESM]).

The survey was uploaded to five communities on HealthUnlocked for people with the following health conditions: rheumatoid arthritis, fibromyalgia, respiratory conditions, thyroid conditions and/or mental health problems. We chose these to represent a variety of physical and mental health conditions, whilst targeting large communities with active membership. Users who logged into HealthUnlocked with a UK IP address were randomly selected via a pop-up request to participate in the survey. Potential participants were diverted to an information page where they were asked to consent to a series of statements to access the survey. The survey ran online from October to December 2017.

Survey responses were collected anonymously. As participants logged into HealthUnlocked to access the survey, we were able to place restrictions to ensure each user could only complete the survey once. Participation in the survey was voluntary and no incentives were offered. Participants could skip questions or exit at any time if they wished. We ran analyses using (a) all valid responses per question and (b) only for respondents who completed surveys in full; proportionally, responses did not materially differ so for ease of reporting and interpretation, only surveys that were completed in full (i.e. completers) were included in the final analysis. Demographic data were collected in the final section, thus characteristics of completers and non-completers could not be compared. Descriptive statistics (counts and proportions) and Chi squared tests (p < 0.05) were used to analyse responses. Separate results for men and women were reported in all tables and in the text, where percentages were under 20% or sex differences differed by more than 5 percentage points.

Focus Groups

We ran five qualitative focus groups across Greater Manchester during February 2019, after the survey results had been analysed. Following the analysis of survey results, we thought it unlikely that members of the public would have detailed knowledge about side-effect reporting and anticipated possible reluctance to participate in individual interviews with this narrow focus. In light of their many benefits [24], we instead chose to run focus groups and adopted a broader lens on health-related social media use to reduce pressure on individuals, gain contextual information and benefit from group interaction, for example, via comments, questions and even challenges between participants, with the intention of generating more detailed data about experience, attitudes and behaviours as well as understanding views on adverse-event reporting. We advertised on online health forums, social media platforms (Facebook and Twitter), and through local networks of condition-specific community groups and patient-facing recruitment organisations (e.g. http://www.researchforthefuture.org), many of whom had previously collaborated with the university. Eligibility was restricted to people who lived with at least one long-term condition of interest (the same range of conditions used as in the survey) and had used social media or online forums in relation to their health. Whilst our original intention was to select participants purposively based on key demographic and experience-based characteristics and to continue data collection until saturation was reached, ultimately convenience sampling was used to recruit people, based on interest, availability and resources.

Focus groups were conducted by two researchers experienced in using qualitative methods (LH, SG) with support from a third post-graduate student on two occasions where either one could not attend. Focus groups were held in various community and university venues. Three groups focused on a single health area (fibromyalgia, mental health and respiratory conditions). To accommodate participant schedules, the remaining two focus groups mixed participants from multiple conditions, namely: fibromyalgia and rheumatoid arthritis; and thyroid and respiratory conditions. Participants completed demographic questionnaires prior to discussions. A semi-structured topic guide was used to guide discussions (see Appendix 2 of the ESM), which followed up on topics addressed by the survey, including reasons for using social media to discuss health and medication; awareness of medication safety reporting mechanisms; and, after a short information session about the MHRA’s Yellow Card scheme, opinions about reusing social media for safety reporting purposes, including reporting ADRs to the MHRA. Researchers moderated the groups to ensure key questions were addressed, whilst allowing for a natural flow of conversation and balancing participation between members. Discussions were audio recorded, with consent from participants, and transcribed verbatim. Participants were paid £20 each for participating, plus travel expenses.

Data were analysed thematically [25], aided by qualitative software (NVivo). First, a basic coding framework was developed in advance, based on the expected range of responses; our previous reading and experience of conducting and analysing the survey data informed code development. Codes were intended to represent the semantic content of the data, rather than more ‘latent’ interpretations. Following data collection, two members of the research team coded transcripts, each taking the lead on coding half the transcripts and then cross-checking each other’s work to ensure reliability. Codes were added or discarded where needed and then grouped into themes (see Appendix 3 of the ESM). Accompanying theme names and descriptions were developed and refined iteratively, with constant reference to the original transcripts. We selected quotes that we considered representative of themes to ‘give a flavour’ of the raw data, illustrate the analytic narrative and add a layer of transparency to the analytical process. A patient and public involvement group (N = 4) with experience of long-term conditions and using social media advised on the study design (e.g. discussion topics) and advertisements for participants ahead of data collection. At a results-focused workshop, they commented on preliminary results, with the benefit of viewing anonymised data extracts, thus assisting our interpretation of the findings.

Results

The material presented in this paper focuses on data and findings most relevant to reusing social media data for improving ADR reporting. To support greater data integration or ‘mixing’ under our mixed-methods approach, a long-standing problem in mixed-methods research [19], we have assimilated the results of the quantitative survey and themes yielded by the qualitative focus group under pragmatically defined headings, rather than reporting them sequentially and separately. This strategy was intended to give equal weight to quantitative and qualitative components; however, the focus on overlapping material inevitably means that certain survey data and qualitative themes were deemed outside the scope of this paper (though may be reported elsewhere, in future).

The qualitative analysis yielded three main themes: knowledge about ADRs and reporting methods (theme 1), use of online forums (theme 2) and attitudes towards linking with the Yellow Card scheme (theme 3). Appendix 3 of the ESM provides an overview of how the three main qualitative themes derived were mapped against relevant survey questions.

Response Rate

Overall, 1791 surveys were started and 1359 (75.9%) of these were completed in full. Table 1 describes the characteristics of the 1359 completers and the 20 focus group participants.

Table 1

Characteristics of participants who (a) completed the online survey in full and (b) participated in focus groups

Characteristic	Online survey, % (N)			Focus groups, N (%)
	Women	Men	All
Gender
Women	100 (1160)	0 (0)	85.4 (1160)	75.0 (15)
Men	0 (0)	100 (192)	14.1 (192)	25.0 (5)
Other	0 (0)	100 (0)	0.2 (2)	0 (0)
Missing/prefer not to say	0 (0)	100 (0)	0.4 (5)	0 (0)
Age group (years)
16–18	0.1 (1)	0.5 (1)	0.2 (2)	0 (0)
18–24	1.29 (15)	1.0 (2)	1.3 (18)	10.0 (2)
25–34	4.6 (53)	1.6 (3)	4.2 (57)	20.0 (4)
35–44	8.7 (101)	7.8 (15)	8.5 (116)	15.0 (3)
45–54	26.0 (302)	20.3 (39)	25.2 (342)	10.0 (2)
55–64	35.8 (415)	28.1 (54)	34.8 (473)	40.0 (8)
65+	23.5 (273)	40.6 (78)	25.8 (351)	5.0 1)
Ethnicity
White	93.1 (1080)	93.2 (179)	92.9 (1262)	90.0 (18)
Black	0.9 (10)	0.5 (1)	0.9 (12)	0 (0)
Asian	1.3 (15)	1.6 (3)	1.3 (18)	5.0 (1)
Mixed	1.0 (12)	1.0 (2)	1.0 (14)	5.0 (1)
Other	0.3 (4)	0.5 (1)	0.4 (5)	0 (0)
Missing/prefer not to say	3.4 (39)	3.1 (6)	3.5 (48)	0 (0)
Employment status
Employed	32.6 (378)	22.9 (44)	31.4 (427)	30.0 (6)
Unemployed	11.1 (129)	9.9 (19)	10.9 (148)	35.0 (7)
Retired	51.6 (599)	65.1 (125)	53.4 (726)	20.0 (4)
Other	4.2 (49)	2.1 (4)	3.9 (53)	15.0 (3)
Missing/prefer not to say	0.4 (5)	0 (0)	0.4 (5)	0 (0)
HealthUnlocked community/main conditiona
Fibromyalgia	19.8 (230)	14.1 (27)	18.9 (257)	25.0 (5)
Mental health conditions	5.2 (60)	5.7 (11)	5.2 (71)	15.0 (3)
Respiratory conditions	18.2 (211)	46.9 (90)	22.3 (303)	20.0 (4)
Rheumatoid arthritis	18.4 (213)	16.7 (32)	18.0 (245)	15.0 (3)
Thyroid conditions	38.4 (446)	16.7 (32)	35.5 (483)	25.0 (5)
Number of health conditions
1	35.7 (414)	30.7 (59)	35.0 (476)	5.0 (1)
2–3	44.1 (511)	37.5 (72)	43.0 (585)	50.0 (10)
4+	20.3 (235)	31.8 (61)	21.9 (298)	45.0 (9)
Total	100 (1160)	100 (192)	100 (1359)	100 (20)

aSurvey participants self-reported their main condition. For survey participants, this indicates the relevant HealthUnlocked community participants were drawn from: Fibromyalgia Action UK, Mental Health Support, British Lung Foundation, National Rheumatoid Arthritis Society and Thyroid UK

The majority of survey respondents were women, and in particular white women (79%). Most (86%) respondents were aged 45 years and over and half (53%) were retired. Among women, over a third (38%) were visiting the thyroid community webpage, 5% were visiting mental health pages and the remainder were spread equally (18–20%) between the rheumatoid arthritis, fibromyalgia and respiratory groups. Among men, almost half of respondents (47%) were visiting respiratory group webpages, 5% were visiting mental health pages and the remainder were spread equally (14–17%) between the fibromyalgia, rheumatoid arthritis and thyroid groups. Participants reported a median of two (interquartile range: 1–3) health conditions from the checklist provided. Commonly reported conditions among both women and men included depression (29%, N = 337; 30%, N = 58), hypertension (27%, N = 314; 32%, N = 61) and back pain (23%, N = 272; 28%, N = 53). Men who participated were more likely than women to report heart conditions (22% vs 7%; × 2 = 44.90, df = 1, p < 0.001), lung conditions (40% vs 19%; × 2 = 41.97, df = 1, p < 0. 001) and diabetes mellitus (14.6% vs 5.17%; × 2 = 22.45, df = 1, p < 0.001). Although not listed as a pre-specified checklist option, women were more likely than men to spontaneously report thyroid problems using the free text box provided to note other conditions (26% vs 6%; × 2 = 36.10, df = 1, p < 0.001).

Focus group participants were mainly white and female, with a mean age of 46 years. Groups included a range of three to ten participants. Thyroid and fibromyalgia conditions were the most common main conditions reported among participants, followed by respiratory conditions, rheumatoid arthritis and mental health conditions. Focus group participants were generally engaged and enthusiastic, with the mean duration of groups lasting 108 min (range: 90–120 min).

Pre-Existing Beliefs, Awareness and Understanding of ADRs and Pharmacovigilance

Among survey respondents, prevention of similar side effects occurring among other patients was viewed as the most important reason for reporting side effects, garnering 88% agreement overall (Table 2). Over two-thirds of participants also agreed that reporting was important to understand the prevalence of side effects (82%) and to be recorded in patient records (75%).

Table 2

Survey responses to questions about side-effect reporting beliefs and proposed methods to improve reporting to the Yellow Card scheme

Survey question	Agreed, % (N)
	Women	Men	All
1. Who do you think should be responsible for reporting drug side effects? Select all that apply.
(a) Your doctor or nurse	83.1 (964)	82.8 (159)	83.1 (1130)
(b) Pharmacists	56.5 (655)	50.5 (97)	55.6 (756)
(c) Drug companies	45.1 (523)	47.9 (92)	45.6 (620)
(d) Patients taking medications	72.0 (835)	70.8 (136)	71.8 (976)
(e) Don’t know	4.1 (48)	6.25 (12)	4.42 (60)
(f) Other	5.0 (58)	6.25 (12)	5.2 (71)
2. Which of these do you think are important reasons to report and monitor drug reactions? Select all that apply.
(a) To find other people with similar reactions	54.5 (632)	56.8 (109)	54.9 (746)
(b) To learn how common such reactions are, or find out how many other people have had the same problem	82.0 (951)	81.2 (156)	81.8 (1111)
(c) To help prevent similar side effects happening to other patients	88.2 (1023)	85.4 (164)	87.7 (1192)
(d) To help doctors make a diagnosis	46.9 (544)	59.4 (114)	48.6 (661)
(e) To be recorded within my health record	75.5 (876)	74.5 (143)	75.3 (1023)
(f) Other	7.0 (81)	6.8 (13)	7.0 (95)
3. Would you be happy for the MHRA to use content posted on HealthUnlocked communities to help monitor side effects. Only de-personalised information would be used.	94.1 (1092)	98.4 (189)	94.6 (1285)
4. Would you be happy for researchers to use content posted on HealthUnlocked communities to help monitor side effects? Only de-personalised information would be used.	94.7 (1098)	98.4 (189)	95.0 (1291)
5. Which of the following would make it easier for you to complete Yellow Card submissions online? Select all that apply.
(a) Link to the Yellow Card website from HealthUnlocked	69.7 (809)	73.4 (141)	70.2 (954)
(b) Suggestion to complete a Yellow Card if your post describes a side effect of medication via a promoted link to the Yellow Card website	35.6 (413)	38.0 (73)	35.8 (487)
(c) Suggestion to complete a Yellow Card if your post describes a side effect of medication via a pop-up Yellow Card form in HealthUnlocked	32.4 (376)	31.2 (60)	32.3 (439)
(d) Suggestion to complete a Yellow Card if your post describes a side effect of medication via a pop-up Yellow Card form in HealthUnlocked, with the form partially filled in automatically from details of your post	35.8 (415)	37.5 (72)	35.9 (488)
(e) Other	4.1 (47)	5.7 (11)	4.3 (59)

MHRA Medicines and Healthcare products Regulatory Agency

Survey participants were asked to indicate their level of understanding about various aspects of monitoring medication safety using a five-point scale (Fig. 1). Overall, a third of people reported they had no knowledge about methods of monitoring medication safety. Knowledge levels decreased further when asked about specific aspects of medication safety, with over half indicating no knowledge of the MHRA or the Yellow Card scheme. Men and women indicated similar levels of knowledge, with one exception; women were more likely than men to report any level of knowledge with regard to how to complete Yellow Card forms (29% vs 21%; × 2 = 4.57, df = 1, p = 0.033).

Fig. 1

Levels of understanding about aspects of drug safety monitoring by survey respondents

Focus group participants were asked about their understanding of what constituted an ADR (theme 1). In two groups, such questions prompted comparisons of side effects due to medication, compared with symptoms of different conditions, drug–drug interactions and allergic reactions. For example:

“It’s very hard to know the difference, because I think the main things are if you’ve got skin rashes, and vomiting, that’s like an allergic response.” (P9, Respiratory, Theme 1)

Only two focus group participants had previously heard of the Yellow Card scheme specifically before we brought it up; both had respiratory conditions and had previously reported to the scheme themselves. Two others attending the mental health focus groups had a general level awareness that systems were in place to monitor medication safety, but could not describe the specifics. As one explained:

“I knew there was a way to report it and I knew that if I wanted to report something, I have to look on the leaflet with my medication, there’s a section there that says it.” (P1, Mental health, Theme 1)

Experience of Medication Side Effects and Reporting

The vast majority (89%, N = 1215) of survey participants reported having personally experienced a side effect from their medication. Experience of side effects ranged from 85% to 96% among different groups, with participants affiliated with the rheumatoid arthritis group indicating the highest rates. Over half (56%, N = 761) stopped taking their medication because of side effects and 9% (N = 128; 9% of women and 13% of men) were consequently admitted to hospital. Focus group participants recited personal experiences of side effects, leading to a range of negative outcomes including more serious health symptoms (e.g. difficulty breathing), hospitalisation and having to retire because of ill health.

Survey participants were asked who they thought had a responsibility to report ADRs. Most frequently, respondents agreed that doctors or nurses had this responsibility (83%), although 72% agreed that patients who took medicines also shared this responsibility (Table 2). Among those who had experienced side effects, 74% (N = 1009) did report it to a general practitioner or hospital doctor; just 3% (n = 44; 4% of women and 1% of men) reported it to the MHRA.

Focus group participants reported that medication and side effects were popular topics of discussion online. Several participants described how online communities shared knowledge and experiences of side effects and strategies to overcome them. For example:

“Because there’s no one size fits all treatment for it, everybody with fibro is given a different combination of medication and treatment … we have people coming on saying I’ve been taking such a tablet, I’m really ill. And we give the advice, well, you should also be taking this other tablet to combat it. And again, we’re giving more advice out than some doctors.” (P2, Fibromyalgia, Theme 2)

Two participants with respiratory conditions had personal experience of reporting to the Yellow Card scheme; one explained her reasoning:

It’s maybe to help the drug companies see how many people are reporting side effects. I don’t know if that’s correct or not. But that’s how I see it. So, I always think well, it’s beneficial to do that every time. And particularly if it’s a severe side effect. I don’t think I’ve done it if it’s been mild side effects. But if I’ve noticed things that are maybe not on the list as well, on the packet, I’ve done it then. (P9, Respiratory, Theme 1)

Engaging Users of Health-Related Social Media Platforms

Well over 90% of survey participants said they would be happy for researchers or the MHRA to use content posted on HealthUnlocked to help monitor side effects (Table 2). Focus group participants gave a range of reasons for being supportive in principle of encouraging users of online forums to report side effects to the Yellow Card scheme (Table 3). One strong reason for support that was independently raised at three focus groups was the potential to raise awareness about the Yellow Card scheme in general among a large and distinct population, likely to be engaged in health issues (Table 4).

Table 3

Benefits and risks of proposed new methods to improve reporting to the Yellow Card scheme (YCS), as provided by focus group participants

	Groupa
	FB	Resp	MH	Thy/Resp	FB/RA
Benefits and advantages
Using social media to promote the YCS could help to raise awareness of the YCS among a large section of the population	✓	✓		✓
It could directly link patients with the MHRA, rather than patients having to rely on health professionals	✓			✓
It could help to provide timely access to information for patients by signposting links at the time patients are searching for and/or discussing side effectsa	✓	✓			✓
It could be a more efficient, modern way of supporting researcha	✓	✓		✓	✓
Partial completion of forms could reduce the effort for patients involved with reporting to the YCSa	✓	✓		✓
Risks and disadvantages
The amount of personal information required to complete a report is not necessary and off-putting	✓	✓	✓		✓
The focus on reusing online content could exclude the experiences of people without digital skills or access to the internet	✓	✓	✓
Mistrust of how the data might be used for future medicines safety purposes				✓	✓
Automated methods risk further erosion of privacy and civil rightsa			✓		✓
Mistrust of pop-ups for security reasons, including use of ad blockersa	✓	✓			✓
Pop-ups can be annoying or distressinga			✓		✓
Automated methods may take patients’ comments out of context, be prone to bias and/or errorsa		✓	✓

FB Facebook, MH mental health, MHRA Medicines and Healthcare products Regulatory Agency, RA rheumatoid arthritis, Resp respiratory, Thy thyroid

aReason applies specifically to one or more automated methods. NB—Exemplar quotes are provided in Appendix 4 of the ESM

Table 4

Characteristics of survey respondents, by type of agreement with proposed reporting methods to improve reporting to the Yellow Card scheme

Characteristics	(a) Links	(b) Automated methods	(c) Links and automated methods	(d) None of these
Sex
Women	34.7 (402)	27.6 (320)	35.1 (407)	2.7 (31)
Men	36.5 (70)	21.9 (42)	37.0 (71)	4.7 (9)
Age group (years)
Under 55	32.3 (172)	27.6 (147)	37.9 (202)	2.3 (12)
55 and over	36.7 (303)	26.4 (218)	33.5 (277)	3.4 (28)
Ethnicity
White	34.5 (435)	26.9 (340)	35.7 (450)	2.9 (37)
BAME	32.7 (16)	22.4 (11)	40.8 (20)	4.1 (2)
Employment status
Employed	34.7 (148)	26.2 (112)	36.1 (154)	3.0 (13)
Not employed	35.1 (327)	27.1 (253)	34.9 (325)	2.9 (27)
HealthUnlocked community
Fibromyalgia	33.5 (86)	28.0 (72)	35.0 (90)	3.5 (9)
Mental health conditions	31.0 (22)	22.5 (16)	40.8 (29)	5.6 (4)
Respiratory conditions	39.6 (120)	24.8 (75)	33.0 (100)	2.6 (8)
Rheumatoid arthritis	33.9 (83)	28.6 (70)	35.9 (88)	1.6 (4)
Thyroid conditions	34.0 (164)	27.3 (132)	35.6 (172)	3.1 (15)
Number of health conditions
1	35.5 (169)	27.3 (130)	33.8 (161)	3.4 (16)
2–3	34.7 (203)	27.2 (159)	35.9 (210)	2.2 (13)
4+	34.6 (103)	25.5 (76)	36.2 (108)	3.7 (11)
All	35.0 (475)	26.9 (365)	35.2 (479)	2.9 (40)

BAME Black, Asian and Minority Ethnicities

“I do [think it’s a good idea], yeah, because there’s people are already saying they’ve got side effects, and plainly the two do need to be linked because those people may not know about the Yellow Card, like we didn’t.” (P4, Thyroid/respiratory, Theme 3)

“I suppose it would create awareness for the younger generation that might not know about it.” (P3, Respiratory, Theme 3)

Nonetheless, their support was not unconditional and several reasons to be cautious were outlined (Table 3). Reservations about preserving privacy and sharing personal data online with a hitherto unfamiliar organisation (i.e. the MHRA) were raised in four focus groups. Concerns about the consequences of excluding the experiences of people without digital skills or access to internet forums was also common. A few also questioned the intentions and integrity of parties monitoring side effects and discussed the implications of surveillance.

“They’re asking for email, phone number, home address, full name, and then they’re asking what medication you’re taking. I don’t think they need to know all that information.” (P5, Fibromyalgia, Theme 3)

“I suppose the information could be massaged – for want of a better word – to try and sort of decrease the importance of the impact of the side effects.” (P6, Thyroid/respiratory, Theme 3)

Attitudes Towards Automated Reporting Methods

Survey respondents were presented with four pre-specified, non-mutually exclusive proposals for making it easier for HealthUnlocked users to report side effects to the Yellow Card scheme (Table 2). Two-thirds (70.2%) of survey respondents agreed that a link to the Yellow Card scheme from the HealthUnlocked site would make it easier for them to report side effects. Respondents were also asked whether specifically targeting posts that discussed side effects would be beneficial, whether via promoted links, pop-ups (a small window that appears on screen) or pop-ups including partially completed forms with details automatically extracted from the posts. These three methods, henceforth referred to as ‘automated methods’, proved less popular among respondents, with roughly a third agreeing to each of them individually (Table 2), and 62% (N = 844) agreeing to any of these automated methods.

There were no substantial demographic differences between participants who preferred simple links, automated methods, both types of methods or none of them (Table 4). A Venn diagram was generated (see Appendix 5 of the ESM) to display the overlap of responses among participants who agreed that any or both methods might be useful. This shows that while a link was the most commonly selected single option, preferences did vary among participants with no particular combination of choices garnering a majority.

The opinions of focus group participants were also divided regarding which methods might work best. In four focus groups, participants discussed the benefits of some degree of automation. This was generally deemed acceptable and consistent with tailored marketing approaches elsewhere online, a sign of “moving with the times”.

“So from an IT perspective, yeah, it’s a brilliant idea. Because if you use cookies to log what people are saying, you could very easily pop up an advert just like what Facebook does.” (P8, Fibromyalgia/Rheumatoid arthritis, Theme 3)

A few people particularly welcomed the ease of a partially completed form, providing it was editable and could be checked for accuracy. Common reasons for rejecting this option were security concerns and mistrust of unknown organisations (including the MHRA) and/or pop-ups.

“I think you’d have to be sure that it wasn’t some rogue pop-up, because that’s the main thing. And I think people would be wary of it.” (P9, Respiratory, Theme 3)

“Because they’re just going to see it as a scam. Somebody trying to get your details off you.” (P8, Fibromyalgia/Rheumatoid arthritis, Theme 3)

Whilst participants attending the mental health-themed focus group did not overtly disagree with the principal of engaging social media users, they were the most critical of automated methods. In particular, there was doubt about the ability of algorithms to adequately deal with subtleties, bias and context.

“I think it would be quite difficult because you can just film it and put it on your story for 24 h and delete it if you want to … that’s going to take a very complex algorithm to realise all the different accents, all the different ways of saying depression or antidepressant … the context.” (P7, Mental health, Theme 3)

“I’m a bit sceptical of these types of things online … I think technology has limited how intelligent it can get, whereas a person, if you’re describing symptoms to a person, side-effects, whatever, I think people, and doctors especially, I think they have some sort of a sensitivity.” (P10, Mental health, Theme 3)

Discussion

Summary of Findings

Our findings indicate low levels of knowledge among users of health-related social media platforms about pharmacovigilance measures in general and the Yellow Card scheme specifically. This was despite 89% of survey respondents having previously experienced side effects because of taking prescribed medicines.

Participant responses from both the survey and the focus groups indicated a strong willingness to engage with the monitoring of side effects and to share health-related social media data about ADRs with researchers and regulators. HealthUnlocked users strongly supported the option of enabling users to submit ADR reports from within the HealthUnlocked platform using an online form, with two-thirds indicating that a link to the Yellow Card website from HealthUnlocked would be helpful. However, they were less united in their support when it came to proposing automated methods of surveillance to monitor posts in real time and actively suggest that users report suspected ADRs. None of the automated methods proposed, including promoted links, pop-ups or partially completed forms, gained majority support, even among demographic subgroups. Follow-up focus groups revealed a range of perceived risks and disadvantages associated with introducing measures to increase reporting among social media users that could explain this caution, some of which (but not all) were specific to automated methods.

Comparison with Existing Literature

Our results are consistent with previous studies that have shown that public awareness and uptake of the Yellow Card scheme is low generally [5-7]. Yet, both survey and focus group participants mostly agreed that patients had a significant role to play in reporting ADRs and, when made aware about the Yellow Card scheme, responded positively. This indicates that patients view public facing pharmacovigilance schemes as important, even if they do not understand the current framework of pharmacovigilance within the UK.

Our results suggested that social media users were, in principle, generally happy for data they generate to be shared with the MHRA and researchers for pharmacovigilance purposes. However, they differed in opinion about what methods they would prefer and which would work best. A previous qualitative study by Golder et al. also found that the majority of participants were broadly supportive of the use of social media to monitor ADRs but lacked awareness around the MHRA and Yellow Card scheme [26]. This was confirmed in our study; the focus groups further revealed that this lack of familiarity fed into a specific reluctance to respond to pop-up invitations on security grounds. For these reasons, our findings support continuing work towards public engagement efforts, examples of which can be found not only in the UK but worldwide, such as the ‘take and tell’ campaign [27], to normalise side-effect reporting and increase that transparency of pharmacovigilance efforts.

Golder et al. also reported some concerns about privacy and the need for autonomy over data uses [26]. Insights from our study also showed that participants worried about sharing personal information and wanted control over what was being shared, not just to protect privacy but also to assess accuracy. Some focus group participants, particularly those attending the mental health session, raised doubts about relying on potentially biased social media reports and the ability of automated methods to distinguish ‘true’ ADRs. Though this was not proposed to focus group participants, some of their more minor concerns regarding automatic detection of ADRs (e.g. detecting misspellings, duplicate posts or decoding acronyms) could perhaps be satisfactorily addressed by modern text analytic techniques [14, 15, 28]. Providing patients with the opportunity to formally trial the performance and usability of proposed systems, including viewing and amending information automatically extracted from sample social media posts, could possibly be a useful avenue for future development to gather even more informed views. Notably, a recent citizens’ jury study [29] showed that patients were supportive of using free text data in patient records, once more familiar with text mining methodologies. Even so, concerns about bias and accuracy cannot be ignored and may indeed be shared by experts: currently, the performance of ADR detection algorithms is complex [14] and the quality of different sources of social media data for research and pharmacovigilance purposes has been questioned [10, 28, 30, 31].

Strengths and Limitations

This study benefited from surveying a large sample of users, even in comparison to other surveys using HealthUnlocked [32, 33], of a leading online health social networking platform. The sequential mixed-methods design took time to complete, but allowed both breadth and depth of analysis, enhancing our ability to probe uncertainties and develop a more accurate picture of user perspectives.

Compared to the general population, white middle-aged women were over-represented among both our survey and focus group participants. Though representative of HealthUnlocked members (our target audience), a recent annual user survey reported a mostly female (72%) and older membership, with 60% of users surveyed aged 60 years and over [34], we cannot comment on the generalisability of findings among patients who use other social media platforms. Furthermore, selection bias may have affected our study as people who had heard of the Yellow Card scheme may have been more likely to participate. An inadequate response rate to study advertisements meant we relied on convenience sampling to maximise recruitment to focus groups, accepting anyone who met the eligibility criteria and was available. Whilst those who did participate engaged fully and provided detailed responses, qualitative data collection ceased before true saturation was reached, meaning that we cannot be certain that we did not miss important perspectives. These are limitations of the study. Although we relied on conducting face-to-face focus groups, on reflection, online methods may have encouraged greater diversity and participation.

Because of the complexity of the automated ADR detection techniques involved, it is possible that participants did not fully understand the methods and options presented to them, particularly in the survey where space was limited. By providing information about the Yellow Card scheme and the details of the proposals for improving ADR reporting in focus groups, we may have addressed gaps in knowledge to allow participants to give more informed views; however neutral we strived to be, this may have introduced bias. Future studies on such topics may wish to consider deliberative designs [35], which formally allow for both information provision and discussion, with procedures to reduce bias. Resources and our preference for maximising participation influenced our choice of study design on this occasion.

Conclusions

Public participation in formal ADR reporting schemes is low, yet many people with long-term health conditions are willing to discuss their health, including medication side effects, with their peers on social media platforms. The use of social media as a source of insights of value for pharmacovigilance purposes is still in an early stage of development. Studies show it is technically feasible to extract data about ADRs from free text discussions on social media platforms using computer algorithms. Yet, there are unanswered ethical questions about the relative benefits and risks. Our study shows that HealthUnlocked users were supportive in principle of promoting awareness of ADR reporting and sharing social media data with both regulatory bodies and researchers for pharmacovigilance purposes. However, they were cautious about accepting the use of automated methods using text mining techniques. In tandem with technical developments, ongoing engagement with users is also essential to understand views, share knowledge and respect users’ privacy expectations to optimise future ADR reporting from patients within these settings.

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 763 kb)

Text mining methods can be used to automatically detect reports of adverse drug reactions (ADRs) in social media discussions; however, the acceptability of applying such methods is unknown.

We used a large online survey and qualitative focus groups to understand views among social media users about using automated data mining methods for improving ADR reporting.

Participants were willing to share social media data about ADRs with researchers and regulators, but were more cautious about accepting automated methods of detecting ADRs.

To optimise future ADR reporting, ongoing engagement with users is essential to understand views, share knowledge and respect users’ privacy expectations.