Social Media Research Strategy to Understand Clinician and Public Perception of Health Care Messages.

INTRODUCTION: Despite the potential of social media to influence public health and generate insights, the process of monitoring and analyzing the dissemination of health care messages on social media has been described as difficult and in need of improvement.
OBJECTIVES: The objective of this study was to describe and demonstrate a reproducible methodology for cataloging and analyzing health care-related social media comments and provide insight into how clinicians and members of the general public respond to health care messaging on social media.
METHODS: We collected social media comments related to the American Dental Association's 2016 "Evidence-Based Clinical Practice Guideline for the Use of Pit-and-Fissure Sealants" between April 10, 2017, and October 31, 2017, from Facebook, Twitter, LinkedIn, Reddit, and online message boards for the New York Times, FiveThirtyEight, and Dentaltown. Using data provided in the comments, we conducted engagement analysis as well as content, network, and sentiment analysis across 8 categories.
RESULTS: We collected 671 comments. Among our findings, Facebook (472 of 671) was the most popular platform among commentators; almost half of all comments (335 of 671) aligned with the recommendations of the 2016 American Dental Association sealants guideline; clinicians were more likely than the general public to like a comment that suggested an improvement to the guideline; and >75% of comments (521 of 671) were supported by anecdotal evidence.
CONCLUSION: As the prevalence of anecdotes on social media suggests, the likelihood of falsehoods spreading on social media is high. Insights gleaned from the methodology described in this research could help combat the spread of such misinformation by providing disseminators of health care messaging with insight into their target audiences. Armed with this knowledge, disseminators can craft health care messages that more effectively engage clinicians and the general public. KNOWLEDGE TRANSFER STATEMENT: The methodology used in this research provides a reproducible strategy for tracking social media engagement with health care messages. Engagement results can assist future delivery of health care messages to key stakeholders and ensure better implementation and adoption of these communications.

Introduction

Social media has become an important information dissemination tool for health care organizations. Today nearly 70% of US adults use social media platforms (Pew Research Center 2018), such as Facebook, Twitter, LinkedIn, Reddit, and online message boards, to access, generate, and share content (Kind et al. 2010; George et al. 2013; Korda and Itani 2013; Fung et al. 2015; Callum Staff et al. 2016; El Bialy and Ayoub 2017). As a result, these platforms provide health care organizations with easy access to a public eager to use the Internet in its medical decision making—an eagerness demonstrated by the fact that 80% of all US adult Internet users search online for health information (Fox 2011) and these users often receive medical opinions from the web before they consult a physician (Gualtieri 2009).

To leverage the public’s online engagement, health care organizations have flocked to social media. At least 60% of all state public health departments (Thackeray et al. 2012) and around 40% of all local public health departments (Harris et al. 2013) in the United States have social media accounts. Prominent organizations, such as the Centers for Disease Control and Prevention and the World Health Organization, are active on Facebook, Twitter, YouTube, and Instagram (Jones 2011). In academic settings, almost all accredited US medical schools have a Facebook account (Kind et al. 2010), and 85% of all North American dental schools use either a Facebook page or Wikipedia entry to promote their offerings (Oakley and Spallek 2012). In addition, 52% of dental practices in the United States use social media for marketing and promotional purposes (Henry et al. 2012).

The health care community’s embrace of social media has resulted in a variety of success stories: social media has proven effective at connecting organ donors with transplant candidates (Henderson 2018), offering anxiety-relieving support to patients (Attai et al. 2015), and facilitating learning among medical students (El Bialy and Ayoub 2017). Social media has also been shown to have a positive influence on people’s health (Maher et al. 2014; Laranjo et al. 2015), successfully encouraging the adoption of such behaviors as increased physical activity (Valle et al. 2013), improved weight loss among the obese (Turner-McGrievy and Tate 2013), and responsible resource use during emergencies (Jones 2011). In addition, social media provides health care organizations with a wealth of information. For instance, the monitoring of social media comments can provide insights into such public health issues as disease incidence, resource needs during crises, and reactions to health care messaging (Fung et al. 2015). In fact, a whole field of analytics called infodemiology has emerged to study the wealth of online public health data now available (Eysenbach 2009).

Yet, despite the significant role that social media can play in influencing public behavior and generating valuable insights, its study in the context of health promotion has been limited (Korda and Itani 2013). Researchers have noted how difficult it can be to monitor and analyze the social media discourse surrounding health care messages and have suggested the need for improvements (Thackeray et al. 2012; George et al. 2013; Moorhead et al. 2013; Fung et al. 2015; Sinnenberg et al. 2017). Proprietary tools, such as the Social Media Research Foundation’s NodeXL, exist for researching social media comments, but their price tag can serve as an impediment to adoption. While handbooks on social media research methods also exist, they often overlook tailoring such research to the realm of health care (Sloan and Quan-Haase 2017).

To address these concerns, this research seeks to accomplish 2 goals. First, it describes and demonstrates a reproducible methodology for cataloging and analyzing health care–related social media comments. Second, it provides insight into how clinicians and members of the general public respond to health care messaging by describing how these 2 audiences responded on social media to the release of the 2016 American Dental Association’s (ADA’s) “Evidence-Based Clinical Practice Guideline for the Use of Pit-and-Fissure Sealants” (Wright et al. 2016; hereafter, the 2016 ADA sealants guideline).

Methods

Data Identification

The first step toward gauging perception of the 2016 ADA sealants guideline involved identifying social media comments for analysis. We established inclusion criteria to determine if a comment was appropriate for analysis. These criteria stated that a comment had to appear on a primarily text-driven social media platform (e.g., Facebook, Twitter, and LinkedIn) as opposed to a primarily visuals-driven social media platform (e.g., Instagram, YouTube, and Snapchat) whose components would have been difficult to analyze. Inclusion criteria also stated that a comment had to have been posted in response to the release of the 2016 ADA sealants guideline and not have been posted as a general comment on sealants overall.

To find as many social media comments as possible that met these criteria, the ADA’s social media team compiled a list of all the messages that the ADA posted on its Facebook, Twitter, and LinkedIn accounts to promote the guideline’s publication; this made it possible to identify social media comments that were posted in response to the ADA’s promotional efforts.

We also compiled a list of all online media outlets that wrote about the guideline and notified the ADA about their stories; this made it possible to identify online media outlet message boards from which to collect comments. Overall, 2 online media outlets—the New York Times (Frakt 2016) and FiveThirtyEight (Hobson 2016)—wrote about the guideline and notified the ADA, and both outlets’ websites housed message boards from which comments were identified. Additionally, the social media teams of the New York Times and FiveThirtyEight responded to emails from our research team asking them to provide a list of all posts that they made on their Facebook and Twitter feeds to promote their stories on the 2016 ADA sealants guideline. By learning of these Facebook posts and tweets, we further identified social media comments that were posted in response to stories from the New York Times and FiveThirtyEight.

Finally, we conducted a search of the Reddit and Dentaltown message boards to identify any comments that had been posted to these platforms regarding the 2016 ADA sealants guideline. We identified Reddit comments by typing “ADA sealants” into the website’s search engine. Typing the same phrase into the search engine for Dentaltown’s message boards produced too many hits, and as a result, “ADA sealants guideline” was instead used to identify comments on these message boards.

Data Extraction

We assembled a team of comment collectors to transfer identified comments to a master Excel spreadsheet. Our team had previously designed, tested, and finalized the spreadsheet during a calibration process (described later). The ultimate purpose of the spreadsheet was to catalog not only each comment’s text but also related data. This included explicit data that could be identified by simply looking at a comment, such as its date of publication, social media channel of origin, and URL. In addition, we conducted engagement analysis, or an assessment of the activity surrounding a social media comment, by documenting explicit data concerning the number of likes/recommendations that a comment received (Sinnenberg et al. 2017).

Our spreadsheet also cataloged extractable data generated through content, network, and sentiment analyses (Sinnenberg et al. 2017) of each comment across 8 categories (Table 1). We conducted a content analysis to assess a comment’s subject matter, a comment’s method of supporting its assertions, whether a comment was on- or off-topic, and whether a comment was in alignment or not with the 2016 ADA sealants guideline’s recommendations. We conducted a network analysis to assess whether a commentator mentioned the ADA and whether a commentator was a clinician or member of the general public. Finally, we conducted a sentiment analysis to assess a comment’s type and tone.

Table 1.

Type of Analysis, Category, and Classifications for Generating Extractable Comment Data.

Type of Analysis	Category	Classifications
Content analysis	What subject matter(s) did the comment discuss?	1 = Firsthand success with sealants 2 = Firsthand failure with sealants 3 = Caries 4 = Application of sealants 5 = Context provided for clarification 6 = One’s prior knowledge of sealants 7 = Wording of the guideline 8 = Frustration with the public 9 = Frustration with dentistry10 = Frustration with the ADA11 = Dental procedures besides sealants12 = Sealants use in adults13 = Financial matters14 = Toxicity15 = Sealants use in kids16 = Materials that make up sealants 0 = N/A
	How did the commentator support their comment’s assertions?	1 = Anecdotal evidence2 = Citing a reputable source3 = Citing a nonreputable source0 = N/A
	Was the comment on- or off-topic?	1 = On-topic2 = Off-topic0 = N/A
	Did the comment align with the guideline’s recommendations?	1 = Yes0 = No2 = Can’t tell
Network analysis	Did the guideline mention the ADA?	1 = Yes0 = No
	Was the comment written by a dental professional?	1 = Definitely no2 = Probably no3 = Probably yes4 = Definitely yes5 = Written by a health professional in a field other than dentistry0 = Can’t tell
Sentiment analysis	What type of comment was it?	1 = Discussion2 = Question3 = Critique to the clinical practice guideline4 = Tag of another person5 = New clinical recommendation6 = Graphic/link7 = Correction
	What was the tone of the comment?	1 = Showstopper2 = Big problem/frustration3 = Minor frustration4 = Positive feedback or support5 = Suggestion for improvement0 = N/A

ADA, American Dental Association; N/A, not applicable.

Our research team decided that these 4 types of analysis (engagement, content, network, and sentiment) would serve as a good framework for our methodology, given that these 4 types of analysis were identified by a systematic review as being essential in analyzing health care comments on social media (Sinnenberg et al. 2017). Our research team also decided that analyses for generating extractable data should occur across 8 categories because, as described in the Coding subsection, after our collection of social media comments, we discussed which categories would most succinctly and comprehensively summarize the data contained within our collection. This discussion led to the creation of the 8 categories depicted in Table 1.

To organize comments, we devised a nomenclature and gave each comment a unique identification number. Comment collection took place over several sessions between April 10, 2017, and October 31, 2017. These efforts occurred after the ADA had already promoted the guideline on its Facebook, Twitter, and LinkedIn feeds between September 20, 2016, and November 3, 2016. Collecting comments after these dates ensured that our team collected all comments that directly resulted from the ADA’s promotional efforts on social media.

Coding

After we cataloged comments and their associated explicit data in the spreadsheet, we generated extractable data by analyzing each comment. Prior to this step, we asked those participating in the comment collection process to read the comments as they were transferred to the spreadsheet and to think about different categories that could be used to analyze the comments. This reflection resulted in the identification of the previously mentioned 8 categories for content, network, and sentiment analysis. Each category focused on a particular question to answer, and underneath each category, different numbers (or classifications) were used to designate an answer to a category’s question. Table 1 lists all categories and classifications.

To demonstrate how this categorization and classification system worked, the following is an illustration of how a comment would be evaluated for the category “Was the comment on- or off-topic?” If a comment was deemed on-topic, it would be classified as a “1.” If a comment was off-topic, it would be classified as a “2.” If a comment was unable to be classified as either on- or off-topic (e.g., for cases in which a commentator’s social media post consisted of simply posting another social media user’s name), that comment would be classified as a “0.” This coding system assigned meaning to each comment across all 8 categories and made quantitative analysis possible.

Calibration

To help analyze all comments across these 8 categories in a consistent manner, we created instructions summarizing the coding system. The instructions identified the number associated with a particular category classification and provided a description of that classification and a sample comment that exemplified it. We developed these instructions after an iterative process in consultation with our research team.

For calibration, we gave each member of our team the same 15 comments to analyze independently. We then met as a group to compare analyses, talk through discrepancies, and reach consensus on how best to classify each comment for a particular category. We performed this discussion under the supervision of a lead researcher who clarified analytic techniques and facilitated consensus when there was disagreement. We reached consistency in how the comments were judged after a second round of calibration in which each member of our team analyzed an additional 15 comments independently before meeting to compare analyses and reach consensus under the supervision of a lead researcher.

Data Analysis

After calibration, we grouped team members into pairs. We assigned each pair a set of 50 comments, which each team member would first analyze independently before meeting with the other to discuss their analyses and reach consensus. In the rare case in which a pair was unable to agree about a particular analysis, a third researcher adjudicated. After consensus was reached, the pair’s members would work together to transfer data for their 50 assigned comments to the master spreadsheet; collaborating on this data transfer helped to minimize the possibility of data entry error. We then assigned the pair another set of 50 comments to analyze. We repeated this process until all comments were analyzed.

Possible errors in data entry were verified against the original posts. We cleaned and quantitatively analyzed data with SAS 9.4 statistical software (SAS Institute Inc). We calculated descriptive statistics to assess the distribution of comment characteristics among the sample. We summarized the comment characteristics of those who were clinicians and members of the general public with chi-square tests for categorical variables and 2-sample t tests for continuous variables. We used Fisher’s exact methods in instances of sparse data. We considered results statistically significant at P ≤ 0.05, except in instances of multiple testing. To account for multiple comparisons of the 16 topics listed in Table 1 (n = 16 tests) in Table 3 and Appendix Tables 1 and 2, we adjusted the significance level with a Bonferroni-corrected P value of P ≤ 0.003

Results

We collected 671 comments across social media platforms. These comments were posted between August 4, 2016, and September 21, 2017. Figure 1 shows that a majority of the comments were posted on Facebook (472 of 671). The New York Times message boards were the second-most popular platform (107 of 671). Clinicians accounted for 44% of the comments on Facebook, while 28% came from the general public and 28% came from those in other health professions or unclear professional backgrounds. However, the general public posted comments more often than clinicians on the New York Times message boards (69%), FiveThirtyEight message boards (65%), and Twitter (62%).

Figure 1.

Prevalence of comments from clinicians and the public across social media platforms, August 4, 2016, to September 21, 2017 (n = 671). a “Miscellaneous” comprises comments from LinkedIn, Reddit, and Dentaltown. b “Clinician” is defined as that classified as a “4” (definitely yes) or “3” (probably yes) for the category “Was the comment written by a dental professional?” in Table 1. c A member of the general public is defined as that classified as a “2” (probably no) or “1” (definitely no) for the category “Was the comment written by a dental professional?” in Table 1. d “Other” is defined as that classified as a “5” (written by a health professional in a field other than dentistry) or “0” (can’t tell) for the category “Was the comment written by a dental professional?” in Table 1.

Figure 2 shows that almost half of the collected comments (335 of 671) aligned with the 2016 ADA sealants guideline’s recommendations, while 21% of comments (141 of 671) did not. Regardless of whether a comment aligned with the guideline, anecdotal statements were the most popular means of lending support to one’s comments: 78% of all comments (521 of 671) offered anecdotal support, while 4% (25 of 671) offered support from reputable sources (which we defined as respected third parties, such as peer-reviewed journals and established news outlets) and 1% (5 of 671) offered support from nonreputable sources (which we defined as noncredible or difficult-to-verify third parties).

Figure 2.

Comment frequency and supporting source according to guideline alignment (n = 671). a A comment that aligns with the 2016 ADA sealants guideline was classified as a “1” (yes) for the category “Did the comment align with the guideline’s recommendations?” in Table 1. b A comment that does not align with the 2016 ADA sealants guideline was classified as a “0” (no) for the category “Did the comment align with the guideline’s recommendations?” in Table 1. c A comment that was unable to be classified as in alignment or not with the 2016 ADA sealants guideline (e.g., a comment that simply posts another social media user’s name) was classified as a “2” (can’t tell) for the category “Did the comment align with the guideline’s recommendations?” in Table 1. d A comment that cited personal experience or opinion to back up its assertions was classified as a “1” (anecdotal evidence) for the category “How did the commentator support their comment’s assertions?” in Table 1. e A comment that cited information from a credible third party, such as a peer-reviewed journal or established news outlet, to back up its assertions was classified as a “2” (citing a reputable source) for the category “How did the commentator support their comment’s assertions?” in Table 1. f A comment that cited information in a way that leaves the reader questioning the comment’s validity—for example, through the citation of a noncredible third party or the positing of quotes that are not easily verifiable —was classified as a “3” (citing a nonreputable source) for the category “How did the commentator support their comment’s assertions?” in Table 1. g A comment that does not cite personal experience or reputable/nonreputable sources (e.g., a comment that simply posts another social media user’s name) was classified as a “0” (N/A) for the category “How did the commentator support their comment’s assertions?” in Table 1. N/A, not applicable.

Table 2 shows that 48% of all comments (323 of 671) were supportive of sealants and the ADA guideline; the tone of these comments was categorized as either positive feedback or minor frustrations. Twenty percent of the comments (132 of 671) were not supportive of sealants and the ADA guideline; the tone of these comments was categorized as either showstopper or big problem/frustration. Four percent of comments (28 of 671) involved suggestions for improvement to the ADA guideline. The most common tone used by clinicians and members of the general public was positive feedback: 49% of all comments from clinicians (118 of 240) and 41% of all comments from members of the general public (109 of 264) aligned with this tone classification. The tone most likely to generate likes on social media was big problem/frustration (6.95 likes on average), followed by suggestions for improvement (4.00 likes) and then showstopper (2.63 likes). Suggestions for improvement generated the most likes among clinicians (4.50 likes on average), while big problem/frustration generated the most likes among the general public (10.66 likes).

Table 2.

Frequency of Tones and the Reaction They Generated.

Tone	Definition	Example	Frequency[a]	Likes, Mean (95% CI)[b]
Positive feedback	Person expresses support for sealants and/or the ADA sealants guideline.	“With all this evidence around, I don’t understand why anyone would NOT believe in performing sealants on their patients and want to help them maintain their health.”
Showstopper	Person identifies a problem so serious that it prevented him or her from correctly understanding sealants and/or the guideline. Person’s comment leaves reader feeling hopeless.	“I worked in several offices when starting my career. Never saw sealants done properly. Ever. They were always done, quick and dirty, to generate a fee. Immoral.”
Big problem /  frustration	Person demonstrates comprehension of sealants and/or the guideline but still believes sealants are ineffective or the guideline is incorrect. Less provocative than “showstopper” comments.	“I had sealants and I will tell you, that if your teeth are not 100% cavity-free . . . you will have massive amounts of cavities after the sealants wear off. I ended up with cavities in every molar—worse than if I had never had them.”
Minor frustration	Person identifies small problems but still expresses support for sealants and/or the guideline. Less provocative than “big problem/frustration” comments.	“I work in public health too, and this is our protocol. I admit, I have a hard time doing it! I save it for kids with no dental home and a mouth full of existing decay. I’ve done it once in 3 years!”
Suggestion for  improvement	Person expresses upgrades to the guideline.	“This study does not acknowledge or track failure rate or how caries are assessed underneath of sealants.”
Not applicable	Person writes comment without providing insight into their feelings (for example, simply tagging someone) or makes a comment that is off-topic.	“Here’s that article!”

ADA, American Dental Association.

Generated via sentiment analysis. These results were statistically significant at P ≤ 0.05.

Generated via engagement analysis. These results were statistically significant at P ≤ 0.05.

Proportion of all comments (n = 671) that displayed this tone.

Proportion of all clinician comments (n = 240) that displayed this tone; “clinician” is defined as that classified as a “4” (definitely yes) or “3” (probably yes) for the category “Was the comment written by a dental professional?” in Table 1.

Proportion of all comments from a member of the general public (n = 264) that displayed this tone; a member of the general public is defined as that classified as a “2” (probably no) or “1” (definitely no) for the category “Was the comment written by a dental professional?” in Table 1.

Table 3 shows that caries was the most common topic (175 of 671) to appear among the comments. Other procedures to consider in addition to sealants was the next-most popular topic (157 of 671), followed by firsthand success with sealants (113 of 671). Comments concerning financial matters resulted in the highest number of likes among all commentators (5.37 likes on average) as well as the most likes among clinicians (4.20 likes) and the general public (5.62 likes). Caries generated the second-most number of likes among all commentators (3.48 likes on average); however, even though caries generated the second-highest number of likes among clinicians (3.05 likes), other procedures to consider in addition to sealants resulted in the second-most number of likes among the general public (4.91 likes).

Table 3.

Top 5 Most Frequently Mentioned Topics and the Reaction They Generated.

Topic	Definition	Example	Frequency[a]	Likes, Mean (95% CI)[b]
Caries	Person discusses tooth decay (includes comments stating that sealants cover up caries and encourage growth).	“If placed over existing decay, a sealant makes a protective roof that will allow the decay to grow.”
Other procedures	Person cites additional interventions (including diet) that could be used instead of, or in conjunction with, sealants.	“So if sealants become the norm in tooth care we can . . . finally get the fluoride out of municipal drinking water? I’m all for it!”
Firsthand success	Person says the application of sealants has helped them or their patients, or the lack of sealants resulted in cavities.	“As a dentist with 30+ years of experience, I know how wonderful sealants are.”
Context provided	Person attempts to clarify sealants and/or the ADA sealants guideline or provides additional research for consideration.	“Sealants do not cover gaps between the teeth. If yours do, you should get a new dentist.”
Financial matters	Person discusses money, coding, insurance, or Medicaid issues related to sealants.	“Dental sealants were mentioned by our dentist, but not covered by our (otherwise good) insurance. Thus $400 to $500 per kid.”

ADA, American Dental Association.

Generated via sentiment analysis.

Results for firsthand success, context provided, and financial matters were statistically significant at P ≤ 0.003.

Generated via engagement analysis.

Proportion of all comments (n = 671) that mentioned this topic.

Proportion of all clinician comments (n = 240) that mentioned this topic; “clinician” is defined as that classified as a “4” (definitely yes) or “3” (probably yes) for the category “Was the comment written by a dental professional?” in Table 1.

Proportion of all comments from a member of the general public (n = 264) that mentioned this topic; a member of the general public is defined as that classified as a “2” (probably no) or “1” (definitely no) for the category “Was the comment written by a dental professional?” in Table 1.

Appendix Table 1 shows that, among showstopper comments (the most dramatic comments not in alignment with the guideline), the top 3 most popular topics were firsthand failures with sealants (27 of 68), followed by caries (21 of 68) and then sealant toxicity (18 of 68). Appendix Table 2 shows that, among comments expressing big problem/frustration (the next-most dramatic comments not in alignment with the guideline), the top 3 most popular topics were caries (16 of 64) and frustration with dentistry as a whole (16 of 64), followed by sealant toxicity (15 of 64).

Discussion

This study highlights 2 main considerations. First, it describes and demonstrates a reproducible methodology for cataloging and analyzing health care–related social media comments. Second, it provides insight into how clinicians and members of the general public respond to health care messaging by describing how these 2 audiences responded on social media to the release of the 2016 ADA sealants guideline.

The reproducible methodology described here offers many advantages. It provides a framework from which to use social media to observe conversations and to analyze attitudes, behaviors, and perceptions of clinicians and members of the general public regarding health care messages (Callum Staff 2016). Additionally, this methodology allows for the large amount of qualitative data available on social media to be translated into easily understood quantitative data that could inform the future production of health care messages and improve delivery of that information to its intended audience. This method offers an alternative to cumbersome processes used to gain insight into health care communications audiences, such as the completion of surveys and questionnaires (Eysenbach 2009). Another benefit of this research is that its data collection methodology can be reproduced by others to collect data related to future health care messaging on social media. Future efforts should attempt to replicate this methodology in different settings and with a variety of health care messages, as this will help validate the methodology and maximize its impact on the social media research ecosystem (Luke Hutton 2015).

Preliminary evidence of the validity of this methodology can be found in the similarities of findings between our research and a previous survey of sealant practices among general dentists and pediatric dentists (Tellez et al. 2011). This survey concluded that US dentists have not adopted evidence-based clinical recommendations for the sealing of noncavitated carious lesions—a conclusion reflected in Figure 2, which shows that a majority of comments (336 of 671) were classified as either not being in alignment with the recommendations of the 2016 ADA sealants guideline or not capable of being classified as in alignment or not. Further evidence of the validity for this methodology should come from utilization of different methodologies to study the same phenomenon of dental sealants or from this study’s methodology being successfully used in a health care field other than dentistry.

Our research also provides insight into how clinicians and members of the general public react to health care recommendations. For instance, our research found that clinicians are more likely than members of the general public to like a social media comment that suggests an improvement to these recommendations. We also found that members of the general public are more likely than clinicians to like comments concerning interventions that can be used in place of or in conjunction with a particular guideline’s recommendations. Insights such as these could be used to tailor future guideline messaging to these particular audiences, and additional research with the methodology described here could be performed to assess whether such messages garner a similar reaction again.

Limitations

There are limitations to this research. Privacy settings on social media platforms can make it difficult to identify and catalog all relevant comments. For instance, comments can be lost or hidden from public view when a commentator changes the privacy settings associated with his or her social media account or closes the account. Lost or hidden comments may be critical parts of ongoing conversations, and their disappearance can result in an incomplete understanding of online discussions.

There are also ethical considerations that need to be taken into account when conducting research of social media comments. The sheer volume of comments posted in response to the release of the 2016 ADA sealants guideline made it impossible to receive permission from all commentators to collect and analyze their words for this research. As a workaround, we designed our study to align with a previously published framework for ethical research with social media data (Townsend and Wallace 2016), which recommends that commentators’ identities not be revealed in the publication of social media research. This anonymity minimizes the likelihood of commentator identification, in turn minimizing the possibility of commentators being stigmatized for their social media posts (Hunter et al. 2018). The framework also maintains that the analysis of comments posted on public-facing social media channels (as opposed to private) constitutes ethical research. In accordance with this standard, all comments used in our research came from public websites where commentators could reasonably expect others to observe their conversations (Townsend and Wallace 2016).

In continued consideration of privacy, when generating extractable data, our research team made determinations based solely on the information that commentators had chosen to share publicly on social media (DeCamp et al. 2013); no additional online searching was performed. Given this limited information, our team’s data analyzers sometimes made judgment calls that could have been influenced by their individual bias. To minimize the impact of such bias, we made sure that data analyzers were calibrated at the start of our research; we had data analyzers work in pairs throughout the project to reach consensus on extractable data; and we brought in third-party arbitrators to settle disagreements between data analyzers.

Visual media such as emoticons and GIFs were difficult to transfer into our data collection spreadsheet; to ensure that such items were properly recorded, we wrote parenthetical descriptions of these visuals into our master spreadsheet. Additionally, retweets on Twitter were not cataloged in our study, because there is no equivalent on social media platforms such as Facebook and Reddit that tracks the number of times that individual comments are shared; as a result, retweets were not tracked, because we wanted to ensure that comments across all social media platforms were evaluated consistently.

Furthermore, even though social media is a great vehicle for connecting people, the conversations on such platforms can be prone to “groupthink,” which occurs when individuals conform to the majority opinion and hesitate to share their own thoughts or ideas (Tsikerdeki 2013). This means that comments may mirror previous comments, and such mirroring could belie one’s true values and preferences.

Conclusion

Health communication is a burgeoning and vital field of public health care. As a result, it is incumbent on health professionals to promote information through the most effective means. The methodology described in this research provides health communications teams with a framework from which to gather important data related to how clinicians and laypeople react to social media communications. Data collected with this methodology can be segmented in various ways to facilitate the development of insights into particular target audiences. For instance, data can be broken down to examine how the social media behavior of clinicians differs from that of members of the general public, to identify the topics that are most important to those who are in agreement with a health care message, or to understand the tones that are most likely to generate likes from social media users. There are numerous ways to parse the data so that messaging can be better tailored in the future, thereby increasing the likelihood that future health care messaging is understood and adopted.

The need to personalize health care messaging becomes noticeable when we consider our data concerning how people support their comments on social media. As Figure 2 shows, a majority of the comments collected for this study (521 of 671) relied on anecdotes, instead of data from reputable sources, to communicate support or dissent for the 2016 ADA sealants guideline. This heavy reliance on anecdotal evidence demonstrates the large potential for misinformation to spread via social media. Better insight into the online behaviors of target audiences, generated through analyses such as the one demonstrated in this article, could help combat such a spread of false and misleading information (Breland et al. 2017).

Author Contributions

J. Huber, A. Fushi, M.T. Duong, contributed to conception, design, data acquisition, analysis, and interpretation, drafted and critically revised the manuscript; T. Woods, contributed to conception, design, data acquisition, analysis, and interpretation, critically revised the manuscript; A.S. Eidelman, A.R. Zalal, contributed to conception, design, data acquisition, and analysis, drafted and critically revised the manuscript; O. Urquhart, E. Colangelo, A. Carrasco-Labra, contributed to conception, design, data analysis, and interpretation, critically revised the manuscript; S. Quinn, contributed to data acquisition, critically revised the manuscript. All authors gave final approval and agree to be accountable for all aspects of the work.

Click here for additional data file.

Supplemental material, DS_10.1177_2380084419849439 for Social Media Research Strategy to Understand Clinician and Public Perception of Health Care Messages by J. Huber, T. Woods, A. Fushi, M.T. Duong, A.S. Eidelman, A.R. Zalal, O. Urquhart, E. Colangelo, S. Quinn and A. Carrasco-Labra in JDR Clinical & Translational Research