Translating COVID-19 knowledge to practice: Enhancing emergency medicine using the "wisdom of crowds".

In the spring of 2020, emergency physicians found themselves in new, uncharted territory as there were few data available for understanding coronavirus disease 2019 (COVID-19), the disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus. In response, knowledge was being crowd sourced and shared across online platforms. The "wisdom of crowds" is an important vehicle for sharing information and expertise. In this article, we explore concepts related to the social psychology of group decisionmaking and knowledge translation. We then analyze a scenario in which the American College of Emergency Physicians (ACEP), a professional medical society, used the wisdom of crowds (via the EngagED platform) to disseminate clinically relevant information and create a useful resource called the "ACEP COVID-19 Field Guide." We also evaluate the crowd-sourced approach, content, and attributes of EngagED compared to other social media platforms. We conclude that professional organizations can play a more prominent role using the wisdom of crowds for augmenting pandemic response efforts.

INTRODUCTION: ADAPTING TO THE UNKNOWN

Society is pressed for time to share coronavirus disease 2019 (COVID‐19)‐related knowledge during the pandemic. As cases of COVID‐19 began to rapidly spread across the United States, physicians were desperate for information to help them prepare their pandemic response efforts, and because so little was known about this novel virus, traditional sources of knowledge were of limited use. Some individuals used traditional free open‐access medical education (FOAM) platforms, including Twitter, Facebook, blogs, and podcasts, whereas others looked to specialty organizations, such as the American College of Emergency Physicians (ACEP).

Given the need for guidance, ACEP used information gathered from its own platform, called EngagED, to create the Field Guide to COVID‐19 Care in the Emergency Department for practicing emergency physicians (released in April 2020). Few studies assess whether organizations can effectively use social media platforms to facilitate community discussion and information sharing. This paper highlights important psychosocial elements of the “wisdom of crowds,” differences between platforms like Twitter and EngagED, and the successes and limitations of EngagED content. In the end, we use this analysis to support a novel approach for organizations (like ACEP) to plan and manage clinical information during a pandemic, using the wisdom of crowds beyond its membership.

SOCIAL PSYCHOLOGY OF GROUPS

The idea of the wisdom of crowds first originated in 1906 when British statistician Sir Francis Galton, a man who believed “the stupidity and wrongheadedness of many men and women was so great as to be scarcely credible,” set out to conduct an investigation to prove the incompetence of the average individual. His experimental design? A weight‐judging competition at an agricultural fair where hundreds of individuals placed wagers on the total weight of an ox after slaughter. To his surprise, the average weight guessed by the crowd was off by only a single pound, closer than any one individual guess, including those of the “experts”—the butchers and farmers.

In theory, groups make better decisions on account of having more brain power, more information, and diverse perspectives. Yet in practice, we have all been part of groups that we feel did not make a better group decision than could have been made individually. Part of designing a group that achieves remarkable collective intelligence is creating the right circumstances. First, there must be diversity of opinion; each member should have unique experiences and information. Second, there must be independence; one person's opinion should not be determined by the opinions of other group members. Third, there must be decentralization; there should not be pressure from any authority that opinions fit a broader goal. And lastly, there must be aggregation, a preconceived determination about how private judgments will be turned into collective decisions. Without this set of circumstances, groups are less likely to generate the wisdom of crowds and more likely to be vulnerable to groupthink and other undesirable group behaviors.

UNMASKING UNCERTAINTY: AUGMENTING PANDEMIC RESPONSE USING THE WISDOM OF CROWDS

As physicians, we strive to practice evidence‐based medicine. Traditionally, we look to peer‐reviewed journals for the best evidence available to help guide our actions; however, even when the highest quality of evidence is published, the issue of knowledge‐translation—transforming what is known into what is practiced— is a major obstacle that must be overcome.

It has been estimated that evidence‐based practices take an average of 17 years to be widely recognized and incorporated, a time interval that implementation scientists persistently work to shorten. Over the past decade, we have witnessed the rise of free, open‐access medical education (#FOAMed) as a major mechanism to increase knowledge‐translation and shorten the evidence‐to‐practice gap. The uptake of #FOAMed has also increased physician usage of social media for professional activities and created virtual communities of practice that engage in post‐publication peer‐review. Beyond sharing, discussing, and challenging emerging research, social media also can be used by physicians to crowd source new ideas, expand networks around specialized topics, and provide moral support to colleagues. Social media has not only changed the way people communicate during their normal day‐to‐day lives, but also has even played a role during moments of crisis.

As cases of COVID‐19 began to rapidly spread across the United States, physicians were desperate for information to help them prepare their pandemic response efforts, and because so little was known about this novel virus, traditional sources of knowledge were of limited use. The best available knowledge was the collective experiences of colleagues in the communities that experienced early surges of cases. The best available knowledge was the wisdom of crowds, which would come in various forms from individuals and organizations alike.

SOCIAL PSYCHOLOGY AND THE WISDOM OF CROWDS: DESIGNING OPTIMAL GROUPS

Now, we see how organizations can even curate social media‐based discussions to create resource guides that serve the greater community. From a social psychology perspective, there are several areas that influence the reach of crowd‐sourcing platforms and the kind of attention paid by individuals. One area is “process loss,” whereby groups will beat the average member and most members but not always the best member. To reduce process loss, experts can be identified and a multistage procedure followed to ensure integrity.

During the COVID‐19 pandemic, more published literature becomes available, plus health systems are beginning to collect their own internal data sets. How will platforms adjust to discussing these data and shift away from people sharing their opinions? The relationship between facts (that exist) and personal opinion potentially leads to counterproductive measures while trying to advance the knowledge of members.

The prioritization of knowledge is another possible issue. For example, attention is a limited resource that can intuitively follow false guidance—including what gets priority or what information goes unseen. Intuition often leads individuals to reaching tenuous conclusions while using subjectivity measures like our confidence levels and/or personal experiences—not objective facts. This creates a potential problem with trusting information that feels true even though it may, conversely, be false.

Several issues affect how individuals select what information garners their attention. One area is accessibility, how currently presented information that is “top‐of‐mind” captures more attention than absent information, even if the absent information is more meaningful. Another area is negativity dominance, whereby negative information captures attention more than an equivalent degree of positive information.

SUCCESSES AND LIMITATIONS OF THE WISDOM OF CROWDS

The psychology of groups captures only part of our understanding for the wisdom of crowds. Equally important are the extrinsic components of platforms like EngagED, Reddit, and Twitter, plus how they differ from one another. For online platforms, the organization, content, degree of polarization, accessibility, and emotional tone are all important areas to consider when weighing the successes and limitations of each.

Organization

Searching for individual items can be difficult particularly as individuals do not always start new "threads" for new subjects. This issue can be solved by a daily compilation of the major entries, a task that continues into the pandemic. For EngagED, the compilation was nearly 596 pages long in November 2020, but it is catalogued and searchable. For other platforms like Facebook, content is often organized into groups—both public and private—with some being physician‐centric, whereas others remain focused on a variety of discussion topics. Twitter is organized by individual users who build their networks one follower at a time (and conversations happening in public).

Content

Important documents shared on forums can be posted on public websites, requiring permission. Such documents may be used to form other literary sources, such as EngagED, which created the first textbook for emergency care of patients (the Field Guide), which, as of the end of May, had over 5.7 million emails/posts and over 2.5 million views. In addition, these forums could collect data (though not a scientific sampling) on key areas for its membership, allowing for the rapid collection and dissemination of knowledge beyond individual participants. In contrast, the content of Reddit discussion boards remains curated with up‐vote/down‐vote functionality by users. Facebook and Twitter present content based on data‐driven algorithms used on the back end.

Polarization

Online platforms do have some negative aspects. A potentially adverse facet of both discussions about health care and online platforms is that polarizing views and diverse politics can “boil to the surface” across many discussions. Comments may be offensive and counterproductive for educational purposes, requiring checks and balances. For example on Twitter, messages that are well received get multiple “likes”/”retweets,” whereas; with EngagED, messages are either present or not on the entire message board without a built‐in mechanism allowing for the highlighting of high‐yield messages.

Accessibility

Some forums are open access (ie, Twitter, Facebook, Reddit) and others are closed to membership (EngagED), leading to privacy concerns. Information can be accessible to hackers and trolls and posted elsewhere on social media. The volume of material can be difficult to control, monitor, and eventually curate.

Emotional tone

The wisdom of crowds provides insight into the emotional rollercoaster of individuals during pandemics. Postings show a range of emotions, not dissimilar to the stages of dying/grief described by Kübler‐Ross. The initial stage is not denial or shock, but an intense search for knowledge and preparation. There is selfless sharing of information, protocols, and inventions—followed by a brief but intense period of anger, directed to individuals who post, staff who monitor, and the sponsoring organization. Open forums leave room for raw emotions and unedited comments, often hurtful—followed by depression, where individuals begin to grapple with the enormous pandemic‐related toll on jobs, friends, family, and the country. Finally, individuals may enter the phase of acceptance.

LAUNCHING ACEP EngagED COVID‐19 COMMUNITY

The COVID‐19 EngagED Communication Hub was launched on March 4, 2020, to provide ACEP members a secure platform to communicate the challenges, share innovative solutions, and discuss science and new data as they emerged. The communication hub was open to all emergency clinicians, regardless of membership, while ACEP staff curated discussions and used this information to help select the Field Guide chapters plus content.

By November 30, 2020, there were 4022 members, 3573 posts, 803 discussion threads, and 223 items shared. The most significant increase in membership was seen from March 10 to March 18, 2020, with the largest number of new members seen on March 14, 2020 (n = 605) (Figure 1). The Communication Hub had global participation, with 3363 members in the United States and 659 international members. The international members represented 92 countries across the globe (Figure 2a). Activity for EngagED rose dramatically during the early stages of the pandemic in March and April, as evident by the number of emails sent and messages viewed compared to other time points (Figure 3). This forum became a platform for not only sharing information and resources but also providing an opportunity for collective creative thought and the sharing of innovations.

FIGURE 1

Number of new COVID‐19 EngagED communication hub members per day

FIGURE 2

(a) Global geographical distribution of EngagED Members. (b) Domestic geographical distribution of EngagED members

FIGURE 3

(a) Weekly trend for the number of daily and total emails sent using EngagED (during COVID‐19). (b) Weekly trend for the daily and total number of message views on EngagED (during COVID‐19)

SUCCESSES AND LIMITATIONS OF ACEP EngagED

After the launch of EngagED, discussions helped to inform content contained within the ACEP COVID‐19 Field Guide, a potentially useful resource. Key questions arose including what forms of EngagED content were used to craft the ACEP COVID‐19 Field Guide, and what kinds of value did the information provide?

The value of EngagED threads of discussion and their associated content may be assessed in several different ways that also reflect the evolving utility of the crowd sourcing of clinical knowledge and operational practices during a pandemic, such as speed, preponderance or level of interest, expertise, innovation, and collaboration. After several reviews of the threads of discussion and daily summaries, we identified the following illustrative examples of successes and limitations (and their corresponding chapters found in the Field Guide contained within parentheses).

SPEED

Success—External Tents (“Work Safety”): The deployment of external tents to screen, triage, and test patients was one of the initial topics of discussion to rapidly garner attention. Members learned from their Seattle counterparts and others about the rationale for the use, site selection, effective layout, and needed resources tailored to contemporary knowledge of COVID‐19.

Limitation—Intubation (“EMS”): Early endotracheal intubation of moderately symptomatic patients was one of the earlier practices suggested because of the risks posed to both patients and health care personnel. As additional experience and patient outcomes‐related data were acquired, prophylactic intubation was no longer recommended.

PREPONDERANCE/LEVEL OF INTEREST

Success—Masks (“Triage”): The anomalous use of respiratory personal protective equipment (PPE) was the topic that elicited the highest number of responses. There were 77 posts about the prevalence of and techniques to reuse N95 masks and 25 posts related to modifying or improvising masks.

Limitation—Patient Activity‐Related Risk Factors (“Risk Stratification and Evaluation”): There were several posts that generated limited attention and discussion. Two threads that received no responses inquired if nutrition and vaping respectively were relevant patient risk factors.

EXPERTISE

Success— Emergency Medical Treatment and Labor Act (EMTALA) (“Regulations and Liability”): As the COVID‐19 pandemic challenged various aspects of health care operations, members with legal and regulatory expertise answered questions and shared their insight pertaining to the required medical screening examination for asymptomatic patients referred to emergency departments for screening and testing.

Limitation—Infection Control Guidelines (“Triage”): Members discussed the cumbersome nature of donning and doffing, challenges with reprocessing, and fatigue associated with the sustained use of powered and controlled air‐purifying respirators (PAPRs and CAPRs). Within weeks, universal airborne precautions were perceived as untenable and amended for high‐risk patients and procedures. Complications eventually led to contention between health care personnel and hospital administrators for mask compliance standards.

INNOVATION

Success—Patient Intubation Protocols (“Management of Patients With COVID‐19″): The use of intubation boxes and development of intubation team protocols were 2 innovations that evolved and were used by numerous members and institutions. Multiple academic medical centers developed and shared their intubation team protocols, including staffing by anesthesia personnel with higher levels of PPE and specialized equipment.

Limitation—Ultrasound (“Assessment”): The use of ultrasound to diagnose COVID‐19 was one of the earlier potential innovations discussed. Despite the availability of bedside ultrasound and early research conducted in Italy, sufficient data and validation have not been acquired.

COLLABORATION

Success—Nursing Homes (“Facility Changes”): One of the first posts summarized the lessons learned from the response to the first large domestic outbreak, which occurred in a nursing home. Lessons learned have also been shared with multiple state and federal government agencies. ,

Limitation—Quarantine/Return to Work (“Personal Well‐Being and Resilience”): One of the longer threads discussed the management of health care workers potentially exposed to COVID‐19. Despite improving understanding of the transmission and pathophysiology of COVID‐19, multiple posts have continued to reveal a lack of consensus and consistent policy implementation by hospitals regarding monitoring and quarantine, use and timing of testing, and criteria to return to work.

DISCUSSION: ADAPTING TO THE WISDOM OF CROWDS

Several topics discussed, such as infection control, telemedicine, supply chains, and business, may accelerate emergency medicine into a new era. Other topics, including infection control and quarantine instructions, may be limited in scope and not reach proper consensus.

EngagED is just one example of a social media platform that provides networking and information gathering in evolving areas of COVID‐19. Unlike other platforms, ACEP curated the content of its membership using EngagED to help generate a field guide and to expand knowledge to the greater medical community. However, it is still important to consider “best practices” from other platforms.

Platforms, such as Facebook, Twitter, and Reddit, assist the public with adapting to the unknown, especially during a pandemic. Each platform has its limitations and offers unique ways of providing content during our current pandemic. Similar to EngagED, each platform can be evaluated by the manner in which it organizes content, remains accessible, offers collaboration, and leads to innovation between users.

Content is culled from users and prioritized based on user preferences, which can lead to the exclusion of pertinent information. Twitter users can query conversations via keyword or hashtag and “follow” other users, prompting notifications to the follower when the user posts new information. Reddit allows its community to decide the popularity of conversations on its message board through a direct voting system. Both Twitter and Reddit allow users to remain anonymous when expressing opinions toward a post and—as a result—improve accessibility for those who may not wish to identify themselves yet provide room for misinterpretation. EngagED and Facebook often commit users to identifying themselves, which promotes accountability but may also discourage user engagement by limiting anonymity. EngagED requires users to be identified for each post whereas Facebook offers niche groups of individuals (eg, Facebook's private group, “EMDocs”). Both platforms remain open access but organize content around those identified individuals who choose to post it.

Although information can be readily found online, steps must be taken to ensure content integrity. FOAM content is readily available but occasionally difficult to verify. Some platforms have attempted to catch misinformation and push verifiable data; Twitter will often use a blue‐colored “verified” badge to alert users of authentic accounts that may serve the public interest, and EngagED uses Community Administrators to rectify issues and disseminate proper messaging when needed.

Challenges and limitations

Social media platforms face challenges for balancing the needs of society with the demands of a fast‐moving specialty like emergency medicine. During a pandemic, physicians need to balance their acquisition of evidence‐based clinical recommendations with anecdotal and quick, easily available “clinical pearls” found on social media. Although a peer review process may take months to complete, it upholds the scientific method and high‐quality evidence amid fast‐moving pandemics like COVID‐19.

Additional challenges are human behavior and the prioritization of knowledge. Twitter, Facebook, Reddit, EngagED, and other social media platforms crowd source ideas, but physicians overcome inherent biases like negativity dominance to variable degrees—often to their own detriment. Ultimately, more studies are needed to investigate how the wisdom of crowds can lead to knowledge translation when used by professional organizations amid pandemics.

Conclusion

Pandemics place undue pressure upon physicians who will continually seek to improve their knowledge, stay current with innovations, and optimally respond to new challenges using verifiable sources of information. Professional societies can make a novel contribution to society by disseminating the knowledge generated from online discussion forums into practical resource guides available for others. Coincidentally, online collaborative platforms—such as EngagED, Twitter, Facebook, and Reddit—should not be overlooked since they harness the collective wisdom of their members to better serve and care for their patients and communities. Ultimately professional societies, including ACEP, should play a more contemporary role engaging and harnessing the collective wisdom of their members to better serve and care for their patients and communities.

DISCLAIMER

The opinions and assertions expressed herein are those of the author(s) and do not necessarily reflect the official policy or position of Madigan Army Medical Center, Uniformed Services University, Defense Health Agency, or the Department of Defense. All authors are members of ACEP and several hold leadership positions within the organization.

CONFLICT OF INTEREST

Dr. Sandra Schneider is current editor of Emergency Medicine Reports (Relias media). Both Dr. Schneider and Sam Shahid are employed by the American College of Emergency Physicians (ACEP). All authors are members of ACEP and several hold leadership positions within the organization.

TABLE 1

Demographics of those members who joined ACEP's EngagED (through end of November)

Demographics	(n)	(%)
Age
20‐29	303	7.53
30‐39	1254	31.18
40‐49	1144	28.44
50‐59	759	18.87
60‐69	410	10.19
70‐79	118	2.93
>80	5	0.12
(no response)	29	0.72
Gender
Female	1440	35.80
Male	2368	58.88
Other	1	0.02
Transgender	2	0.05
(no response)	211	5.25
Ethnicity
American Indian or Alaska Native	4	0.10
Asian	169	4.20
Black or African American	53	1.32
Hispanic or Latino	108	2.69
Native Hawaiian or Other Pacific Islander	1	0.02
Other	917	22.80
Two or More Races	31	0.77
White	1511	37.57
(no response)	1228	30.53