An Investigation of Misinformation Harms Related to Social Media during Two Humanitarian Crises.

During humanitarian crises, a large amount of information is circulated in a short period of time, either to withstand or respond to such crises. Such crises also give rise to misinformation that spreads within and outside the affected community. Such misinformation may result in information harms that can generate serious short term or long-term consequences. In the context of humanitarian crises, we propose a synthesis of misinformation harms and assess people's perception of harm based on their work experience in the crisis response arena or their direct exposure to crises. © Springer Science+Business Media, LLC, part of Springer Nature 2020.

Introduction

In humanitarian crises where a community faces large scale dangers, the affected people seek information that can help respond to such crises. However, in a short period of time, official and legitimate sources of the governments or news organizations normally cannot offer enough confirmed/verified information, pushing the community to consume information mainly through fast acting social media channels (Oh et al. 2013; h, 2015). Here social media can play a vital role with many active users uploading real time data about the crises (Holdeman 2018). However, social media is often the source of widespread misinformation (Gupta et al. 2013; Holdeman 2018; Maddock et al. 2015; Rajdev 2015). Facebook, WhatsApp and Twitter have been identified as social media platforms that spread most misinformation in crises (Nealon 2017; Pang and Ng 2017).

Research that focuses on misinformation harms has gained attention in the recent past. Agrafiotis et al. (2018), Elliott (2019) and Ohlhausen (2017) discuss misinformation harms, but not including humanitarian crises. Other scholars have addressed disinformation occurring during the COVID 19 pandemic (Love et al. 2020; Motta et al. 2020; Ma et al. 2020). Yet, to the best of our knowledge, there is no research that systematically examines people’s perception of the effects or consequences of misinformation in terms of harm during different types of humanitarian crises and in terms of differences in the perception of affected people or the larger community.

This paper fills the gap by developing a systematic synthesis of harms from misinformation as applied to humanitarian crisis contexts and investigates aspects of such harms. We use the synthesis in two chosen scenarios of crises misinformation. We present a visualization of the harms and test for significant differences between perceptions of harms between those working in the crisis response arena and those who are not as well as those affected by the crisis and those who are not. Thus, this paper contributes to the gap in the literature regarding misinformation harms and perceptions of such harms.

The rest of the paper is organized as follows: The next section reviews the literature. Subsequently, we extract information and conduct surveys and analyze data to examine harms from misinformation during humanitarian crises. In the final section, we include a discussion and conclusion of the paper.

Literature Review

Misinformation and Misinformation Harms

Misinformation is incorrect information which can seem to be legitimate initially (Holdeman 2018) but can mislead and create harmful effects to the individual and the community (Pang and Ng 2017). Love et al. (2020) have identified that misinformation propagation can deepen harmful or deadly effects on people. Motta et al. (2020) have showed that misinformation spreading through right-leaning media shaped public misleading beliefs and eventually lead to distrust in media. They have also reported that “even seemingly innocuous [misinformation] from relied-upon media sources may lead individuals either into a false sense of security or lead others to ignore government recommendations” (p. 336). Misinformation harm has also been shown to range from hundreds of fatalities (Love et al. 2020). We stress on the following definition of misinformation harms: injuries that are a result of damages caused by misinformation (Bostrom 2011; Sandvik et al. 2017).

Prior literature on misinformation harms during humanitarian crises is scarce (Tran et al. 2019). Agrafiotis et al. (2018) created a structural taxonomy of harms in the context of organizations rather than in the context of humanitarian crises. They defined five main categories of harms and their sub categories, including physical or digital harms, economic harms, psychology harms, reputational harms, and social or societal harms. The harms were considered based on the view of organizations. In addition, Ohlhausen (2017) classified five groups of harms (FTC Informational Injury Workshop Report, 2018). Her taxonomy of injuries includes deception injuries, financial injuries, health or safety injuries, unwarranted intrusion injuries, and reputational injuries. Similarly, Elliott (2019) expounded on 5 categories, including physical harms, psychological or emotional harms, financial harms and reputation harms. He also mentioned about short-term and long-term harms. In this paper we draw from this work and adapt it to crisis context.

Misinformation Harms in Humanitarian Crises

Social media is an indispensable part of crisis response. It is utilized by the authorities for reporting real-time developments on the ground through breaking news and headline reporting. Social media has garnered public attention as a communication tool during crisis situations. However, the success of social media has been short-lived owing to the problem of misinformation.

Several studies have investigated the misinformation in crisis context (see Table 1). In the context of health crises, the anti-vaccination misinformation situations that expound on unproven risks and side effects or the inability of the immune system to respond to the viruses and bacteria have damaged public confidence in vaccination resulting in decline in vaccination and letting the community become exposed to diseases such as measles-mumps-rubella, hepatitis B, and H1N1 (Peretti-Watel et al. 2014). In addition, in the crisis of Zika virus in 2016, claims about the cause (genetically modified mosquitoes), severity (Zika virus symptoms are similar to seasonal flu), immunity (Americans are immune to the virus) and prevention (coffee can keep Zika mosquitoes away) caused problems for efforts to fight the dangerous infectious disease that resulted in people’s health being at risk (Ghenai and Mejova 2017, p. 3). Dredze et al. (2016) have attributed misinformation in social media to uncertainty regarding the origin of the message. Jamison et al. (2020) have discussed the role of Facebook ads in shaping misinformed views about vaccination among people.

Table 1

Misinformation harms in crisis situations

Type	Crises	Problems / misinformation	Harms	Details	Source
Health care crises	Zika virus, 2016	Information overload in short period	Confusion about severity, immunity, vaccination	Harmful information overload causes problems for efforts to face dangerous infectious disease and put people’s health at risk	Ghenai et al. 2017
	Anti-vaccination situations	crisis of public confidence in vaccination	Expose to risks of diseases.	The decline of vaccination coverage: MMR vaccine in Europe (2010), hepatitis B vaccine in France, and H1N1 vaccine in many countries (2009), 300% increase of measles (2019).	McNamara 2019. Miller 2019. Newton 2019.
Natural disasters	Hurricane Harvey, 2017	False information	Delayed evacuation because of fear, causing dangers	Rumor on Twitter: officials ask for their immigration status.	Nealon 2017
	Hurricane Irma, 2017			Rumor: survivors would receive generators.	Nealon 2017
	Louisiana Floods, 2016	Information overload	Confused between legitimate and fake information	Misinformation from Facebook messages and posts confused FEMA (March 2016 floods) and The American Red Cross (Summer floods).	Holdeman 2018.

In the context of natural crises, such as hurricane Sandy, Gupta et al. (2013) investigated spread of misinformation-filled messages. They concluded that there were very few original misinformation messages and that the majority of these messages were shared messages. Rajdev and Lee (2015) examined the behaviors of malicious users posting misinformation messages and concluded that malicious users had lesser number of favorite tweets compared to legitimate. Nealon (2017) reported that false information lead to unnecessary fears on the one hand and false expectations on the other, which severely affected evacuation decisions and support from authorities during hurricane Harvey and Irma. Similarly, misinformation about Louisiana floods in 2006 from Facebook messages and posts confused FEMA (March 2016 floods) and the American Red Cross (Summer floods) with information overload (Holdeman 2018).

Prior research has examined 15 types of harms related to crisis misinformation, including life, injury, income, business, emotion, trust, reputation, discrimination, connection, isolation, safety, access, privacy, decision and confusion harms (see Table 2).

Table 2

Types of misinformation harms in humanitarian crises

Harm types	Description	References
Life harm	Life threatening harms and resulted deaths to victims.	Peretti-Watel et al. 2014.
Injury harm	Physical bodily injuries.	Peretti-Watel et al. 2014.
Income harm	Financial or economic damages due to loss of jobs.	Elliott 2019.
Business harm	Financial loss of business and organizational benefits.	Elliott 2019.
Emotion harm	Emotional sufferings such as sadness, anger, fear, stress or depression.	Nealon 2017.
Trust harm	Loss of belief and trust on people or social media.	Nealon 2017.
Reputation harm	Loss due to damaged reputation and related social consequences.	Maddock et al. 2015.
Discrimination harm	Suffering from discrimination actions and attitudes from others.	Maddock et al. 2015.
Connection harm	Suffering from interrupted social connections with family, friends or working partners.	Agrafiotis et al. 2018.
Isolation harm	Suffering due to social isolation from the community.	Agrafiotis et al. 2018.
Safety harm	Exposure to dangers such as identity thefts and consequences	Sandvik et al. 2017.
Access harm	Denied or restricted access to services due to leaked sensitive information.	Sandvik et al. 2017.
Privacy harm	Leaked sensitive or private data, which can lead to physical intrusions or consequences of misused data.	Ohlhausen 2017.
Decision harm	Wrong decisions that may lead to dangers.	Holdeman (2018).
Confusion harm	Loss of reaction time and confusion resulting in delayed decisions.	Holdeman (2018).

There is recent research on Twitter social media users that address certain types of misinformation and their harms, such as the use of house cleaners as COVID 19 virus treatments (Chary et al. 2020) or “vaccine misconceptions” during Zika virus outbreak (Dredze et al. 2016). Additionally, (Motta et al. 2020) have examined “mentions” from online users that are related to misinformation harms on platforms such as Media Cloud in the context of the COVID 19 pandemic. However, few researchers have systematically considered the consequences from misinformation as might be perceived by either people from different backgrounds or different groups of affected people such as “patients” (Love et al. 2020) or “medical students and hospital workers” (Ma et al. 2020) in such a healthcare crisis like COVID 19 pandemic.

To ensure enriched quality of data (Love et al. 2020; Ma et al. 2020), in this paper, we have recruited participants that have knowledge and experience as well as exposure to the actual context of that topic. We have surveyed (Agrafiotis et al. 2018) participants that are victims of the crises (or referred to in this paper as “victims); and (Alexander 2018) participants that have existing crisis related working experience (referred to as “crisis responders”). We believe that victims can provide first-hand organic insight into perceptions of harms based on their own experience facing crises’ hazards and vulnerabilities. In addition, the crisis responders will have in depth knowledge and understanding about harm likelihood and impact as a result of their routine work that aims to mitigate or minimize possible immediate, short term or long term effects of emergencies as well as to help the community of victims and their situations to recover or reconstitute.

While various researchers have recently started addressing misinformation harms in specific contexts, there are several gaps in the literature that need to be considered. First, the harm identified are typically anecdotal in nature (Chary et al. 2020). Second, there are limited studies that systematically investigate different types of harms associated with the humanitarian crises. Third, there is scant research that addresses harm perceptions of people with varying crisis experience. Finally, majority of the research focuses on misinformation harm identification, rather than harm assessment. This paper fills these gaps.

Therefore, this paper (Agrafiotis et al. 2018) establishes a synthesis of misinformation harms that are applicable to the context of humanitarian crises, and (Alexander 2018) examines how people perceive misinformation harms in crises. The findings are expected to not only contribute to the understanding of perceived harms of misinformation during humanitarian crises contexts but also derive practical implications to stakeholders such as crises first responders, governmental organizations and social media platforms in efforts of minimize effects of misinformation toward the victims.

Methodology

Details of Survey

To examine misinformation harms, we conducted a survey to obtain judgements of people regarding their perceptions of harm from misinformation during crisis situations (Park et al. 2006). The survey was approved by IRB (Institutional Review Board) at a southern university in the U. S. It was designed using Qualtrics.1 The survey was distributed through Amazon Mechanical Turk (MTurk).2

We chose two scenarios, anti-vaccination and hurricane, that exhibited the following criteria: (Agrafiotis et al. 2018) popularity and familiarity: scenario details were widely known or were reported in various news outlets; and (Alexander 2018) diversity: the scenarios had different characteristics and captured different types of harms of misinformation in crises. The resulting scenarios are listed in Table 3: (Agrafiotis et al. 2018) anti vaccination crisis with overload of and confusing misinformation, and (Alexander 2018) Hurricane Harvey 2017 disaster with the wrong claim of immigration status check..

Table 3

Misinformation scenarios

Scenario	Name	Type	Misinformation	Source
S1	Anti-vaccination	Healthcare	Information overload and confusing information	Miller 2019. McNamara 2019. Newton 2019.
S2	Hurricane Harvey, 2017	Natural	Wrong immigration checking information	Nealon 2017.

The survey was conducted in three rounds: The first round was a screening survey to filter participants that were victims and/or crisis responders, including those in firefighter departments, police departments, Federal Emergency Management Agency (FEMA), the Red Cross, or hospitals and other healthcare organizations. We ensured the reliability of participants’ claim that they were victims or crisis responders by asking them to list from 3 to 5 key steps they had performed to handle crisis situations. In this round, we asked for 400 responses from participants in the U. S. We retained 273 participants (68.25%) who had appropriate responses.

The second round was constructed to filter the 273 participants based on their familiarity with hurricane Harvey or anti-vaccination crises, or whether they had been involved in similar situations (see Table 3). Familiarity with the scenarios was measured on a Likert scale from 1 to 5. We retained 183 participants (67.03%) who had familiarity with the crisis situations at hand.

In the third round, we sent harm perception questionnaires to the 183 participants. They were asked to give judgements about the 15 harms listed in Table 2 on two aspects: likelihood of happening, and the level of impacts or the perceived damages of the harms. The ratings on likelihood ranged from 0 to the highest level of 10. Out of 183 requested responses, we got 89 responses (48.63%).

Addressing Amazon Mechanical Turk (MTurk) Data Quality Concerns

There have been debates about the quality and validity of studies conducted on MTurk. Although raising concerns about appropriateness and overall quality of MTurk workers’ responses, Cheung et al. (2017) has pointed out that MTurk responses passed various important validity tests. Importantly, most researchers agreed that MTurk workers and their responses are much more diverse than participants from other recruiting methods (Buhrmester et al. 2011; Casler et al. 2013; Heen et al. 2014; Majima et al. 2017; Sheehan 2018). We also applied various quality control measures as detailed below.

We only allowed people qualified as “Master,” someone with more than 90% previous approval rates to join. Further we used attention check questions (ACQ) to ensure that we got the best responses. Our records show high quality regarding those ACQs. All these steps are crucial to guarantee high quality and appropriateness of the research design.

Data Analysis

Overview

As we can see from Table 4, the 89 participants were well distributed between scenarios.

Table 4

Participant distribution

Scenarios	Qualified Participants
	Value	% on Total
Scenario 1 (Anti-vaccination)	49	55.06%
Scenario 2 (Hurricane Harvey)	40	44.94%
Total	89	100.00%

Examining Harms in Scenarios and Quadrants

In order to examine harms perceptions in the misinformation scenarios, we tracked differences on two main dimensions: likelihood and impact. We rescaled the 0–10 scale to −5 to +5 scale by subtracting 5 so that zero is the mid-point. This resulted in four quadrants as shown below:

Quadrant 1: negative likelihood and negative impact

Quadrant 2: negative likelihood and positive impact

Quadrant 3: positive likelihood and negative impact

Quadrant 4: positive likelihood and positive impact

Table 5 summarizes the likelihood and impact, and the associated quadrants for the 15 harms. The ratings vary between the scenarios.

Table 5

Likelihood and impact scores

HARMS	SCENARIO 1 – Anti-vaccination			SCENARIO 2 – Hurricane Harvey
	Likelihood	Impact	Quadrant	Likelihood	Impact	Quadrant
Life	−0.77	0.82	2	0.45	1.83	4
Injury	−1.02	0.7	2	−0.03	1.28	2
Income	−1.81	−0.5	1	−0.45	0.75	2
Business	−1.46	−0.34	1	−0.79	0.55	2
Emotion	−1.13	0	2	0.68	1.2	4
Trust	−0.46	0.35	2	−0.16	0.58	2
Reputation	−1.02	0.12	2	−0.95	0.18	2
Discrimination	−1.58	−0.58	1	0.39	1.03	4
Connection	−1.79	−0.34	1	0.16	0.1	4
Isolation	−1.35	−0.28	1	0.16	0.8	4
Safety	−1.58	−0.16	1	−0.29	1.5	2
Access	−2.06	−0.5	1	−0.29	1.18	2
Privacy	−1.92	−0.96	1	−0.21	0.8	2
Decision	−0.54	0.32	2	0.21	1.53	4
Confusion	−1.1	−0.46	1	0.68	1.13	4

Table 5 yields the visualization as shown in Fig. 1. From Fig. 1, we can see that people perceived lower likelihood of harm for misinformation related to anti-vaccination while higher impact of harm for misinformation regarding hurricane scenario.

Fig. 1

Misinformation harms of two scenarios

The most noticeable difference is in connection harm and isolation harm. Similar is the case with decision harm and confusion harm. This can be expected because confused undocumented immigrants may make wrong decision thereby not evacuating, and therefore may be left isolated. Furthermore, physical harms and emotional harms from scenario 2 are higher, suggesting that generally people care more about life threatening issues in crisis situations. Finally, we can also see that it seems financial harms and certain other harms related to general safety, services access and personal privacy do not have high scores, indicating that in such humanitarian contexts, those financial or safety harms are not prioritized because people care more about physical dangers and emotional harms.

One factor that can help explain this difference is the immediacy of response. In hurricane Harvey context, people are urged to act fast, and postponing evacuation due to misinformation about immigration status checks may lead to higher perception of harm. On the contrary, the need of vaccination can take months or years because its effects take more time to appear.

Post-hoc Analysis

Scenario Differences

In this section we test the significance of pairwise comparisons between the mean values of harm perceptions using Tukey test (Howell 2010). Tukey test investigates the significance of differences in means across the two scenarios in this study. The details of the results can be seen in Table 6. From the total 15 examined harms, we only report the differences that are statistically significant with p-values equal to or less than 5%.

Table 6

Harm differences across the scenarios

Criteria	Related harms	Mean Differences (S2 – S1)
Harm’s Likelihood	Emotion	1.81*
	Discrimination	1.91*
	Connection	1.95**
	Access	1.77*
	Privacy	1.71*
	Confusion	1.78*
Harm’s Impact	Safety	1.66*
	Access	1.68*
	Privacy	1.76*
	Confusion	1.49*

S1 – Anti-vaccination and S2 – Hurricane Harvey; *: p-values ≤ 5%; **: p-values ≤ 1%.

From results in Table 6, we can see that the likelihood and impact of access harm, privacy harm and confusion harm differs between anti-vaccination and hurricane scenario. In addition, the likelihood of emotion harm, discrimination harm and connection harm as well as the impact of safety harm differs between the two scenarios. For example, the immigrants that decide not to evacuate during Hurricane Harvey disaster (S2) were more likely to face emotional harm due to interrupted social connections with friends or family members together with fears from dangers. Moreover, participants worry about general safety concerns owing to vast physical and life devastation potential of a hurricane.

Individual Differences

We further investigate the differences in harm perception between different participants with varying crisis experience and exposure.

We consider whether the perceived harms would be different between those working in the crisis response arena and those who are not. Crises related working experience is the experience dealing with rescuing victims (such as first responders or police officers), treating victims (such as doctors and nurses working for hospitals or clinics), or supporting victims of crises (such as staffs of emergency responses organizations like FEMA). We denote participants with such working experience as W1, and those without such experience as W0.

In addition, we also consider whether the perceived harms would be different between those affected by the crisis and those who are not (i.e. direct victims of the crisis). Direct victims are people who have been directly affected by any kind of crises, including natural and manmade crises. We denote participants that are direct victims as V1, and participants that are not direct victims as V0.

Table 7 showed the details of these considered groups and their distribution.

Table 7

Participants for post-hoc analysis

Criteria	Groups’ codes	Group descriptions	Value	% on total
Related work	W1	Crisis responders	43	48.31%
	W0	Not crisis responders	46	51.69%
Total participants			89	100%
Direct victims	V1	Direct victims of crises	58	65.17%
	V0	Not direct victims of crises	31	34.83%
Total participants			89	100%

Tables 8 and 9 show the significant differences between groups of participants listed in Table 7. We consider the differences in terms of harms’ likelihood and impact. We report only the significant differences in Tables 8 and 9.

Table 8

Analyses comparing means between participants that are crisis responders and not crisis responders

Criteria	Related harms	Mean differences
Harms’ Likelihood	Reputation harm	W1-W0: 1.83*
	Safety harm	W1-W0: 1.42*
	Access harm	W1-W0: 1.36*
	Privacy harm	W1-W0: 1.57*
	Decision harm	W1-W0: 1.41*
Harms’ Impacts	Life harm	W1-W0: −1.26*
	Reputation harm	W1-W0: 1.52*

*: p-values ≤ 0.05; **: p-values ≤ 0.01; ^:p-values ≤ 0.10)

Table 9

Analyses comparing means between participants that are victims and are not victims

Criteria	Related harms	Mean differences
Harms’ Likelihood	Access harm	V1-V0: 1.54*
Harms’ Impacts	Injury harm	V1-V0: −1.24^
	Emotion harm	V1-V0: −1.67*
	Isolation harm	V1-V0: −1.3^

*: p-values ≤ 0.05; **: p-values ≤ 0.01; ^:p-values ≤ 0.10)

We notice that harms’ likelihood is positive across crisis experience groups. This implies that participants with crisis experience judged the likelihood of harms higher than the participants with no crisis experience because crisis responders have worked closely with the community impacted and as a result may be able to better able to identify potential harms that the opposite group may not be able to.

In addition, harms’ impact is negative across crisis exposure groups. This means that crisis victims reported lower impact of harms than their counterparts (non-victim participants). This is because victims have more realistic estimation while non-victims have exaggerated judgements that may be shaped by social media coverage.

We also notice that most of the significant differences in perceived harms belong to emotional or psychological harms such as trust, emotion, reputation or isolation harms because fear of immigration issues and physical damages from the hurricane can generate much higher levels of harms in S2 compared to the vaccination context of S1.

Conclusion

Humanitarian crises are situations in which people seek helpful information to find suitable solutions. Social media can act much faster than official information sources, but it comes with a price: exposure to misinformation that can create serious consequences. Many researchers have tried to tackle the situation by creating misinformation detection systems or algorithms, hypothesizing and testing the roles of behavioral characteristics of involved people, or finding the patterns of how misinformation can successfully spread and influence people. However, not much attention has been placed on categorizing the harms or impacts. This paper assesses misinformation harms in the context of humanitarian crises.

Moreover, by gathering judgements of people that have experience and exposure to crises through various rounds of survey, this study investigated the likelihood and the level of impacts of different harms derived from the literature as well as the individual differences associated with them.

These findings are expected to be beneficial not only for optimizing crisis response and recovery activities for prioritizing the use of resources, but also for future research studies to deepen and broaden such findings. The contributions of this research to both the practical side of benefiting the community or minimizing harms for victims and the academic size of forming a systematic background for humanitarian crises and emergency related researches are clearly significant.

There are certain limitations of this study. While we have tested how participants with and without crisis related working experience or victimization experience responded differently for likelihood and impacts of harms, we have not tested for specific types of working experience (such as police officers, first responders, doctors or nurses) or victimization exposure (such as direct or indirect victims). In addition, we have not examined the effects of demographics information (such as age, gender or income) that may influence the perceived harms. We propose that future research can extend this work in several ways. We recommend testing other types of crises in order to generalize findings reported in this paper.