Health and science-related disinformation on COVID-19: A content analysis of hoaxes identified by fact-checkers in Spain.

A massive "infodemic" developed in parallel with the global COVID-19 pandemic and contributed to public misinformation at a time when access to quality information was crucial. This research aimed to analyze the science and health-related hoaxes that were spread during the pandemic with the objectives of (1) identifying the characteristics of the form and content of such false information, and the platforms used to spread them, and (2) formulating a typology that can be used to classify the different types of hoaxes according to their connection with scientific information. The study was conducted by analyzing the content of hoaxes which were debunked by the three main fact-checking organizations in Spain in the three months following WHO's announcement of the pandemic (N = 533). The results indicated that science and health content played a prominent role in shaping the spread of these hoaxes during the pandemic. The most common hoaxes on science and health involved information on scientific research or health management, used text, were based on deception, used real sources, were international in scope, and were spread through social networks. Based on the analysis, we proposed a system for classifying science and health-related hoaxes, and identified four types according to their connection to scientific knowledge: "hasty" science, decontextualized science, badly interpreted science, and falsehood without a scientific basis. The rampant propagation and widespread availability of disinformation point to the need to foster media and scientific caution and literacy among the public and increase awareness of the importance of timing and substantiation of scientific research. The results can be useful in improving media literacy to face disinformation, and the typology we formulate can help develop future systems for automated detection of health and science-related hoaxes.

Introduction

A great amount of misinformation and hoaxes on matters related to the pandemic emerged in parallel with the COVID-19 pandemic, and spread primarily through social networks. This phenomenon reached such levels that the World Health Organization (WHO) described it as a “massive infodemic,” and warned the world of its dangers as it prevents the public from accessing the much-needed reliable information about the disease [1]. It is well known that many of the hoaxes were focused on scientific and health-related topics [2,3]. However, the relationship between scientific information and the characteristics of these hoaxes has not been elucidated.

For the first time in contemporary history, a pandemic of this magnitude was experienced, and all media outlets across the globe disseminated a huge amount of “express science” that gave rise to a problematic relationship between science and society. A lot of information was based on preprints of scientific publications that had not yet undergone a peer-review process, and this contributed to public misinformation.

Spain was hit hard by the pandemic and suffered a high percentage of infections and deaths [4]. On March 14, 2020, the government announced a state of alarm, which involved a nation-wide lockdown that lasted until June 21. Spain was restrained by a national lockdown, which created a crisis, and citizens were eager to understand the pandemic better and turned to social media to receive immediate information. Spain was thus a perfect case for our study.

The phenomenon of information disorder

History is littered with examples of fabrication and dissemination of falsehoods by people, organizations, and governments [5,6]. Recently, public dissemination of falsehoods has reached unprecedented proportions. Digital networks have transformed traditional public communication processes, and one of their consequences is that incorrect information can now be spread worldwide quickly, and on a massive scale.

Disinformation refers to deliberate deception, whereas misinformation refers to the unintended proliferation of falsehoods. These two categories effectively differentiate between acts of malice (voluntary) and mistakes (involuntary). These two broad categories include multiple modalities and specific terms. Research has explored certain modalities such as conspiracy theories [7], rumors [8], and hoaxes [9].

In journalism, “fake” [10,11] or “false” [12] news phenomenon has been widely investigated. Interest in disinformation within the media has intensified over the last decade, especially since 2016, as a result of the US presidential election [13] and the Brexit referendum [14]. The incidence of false information during these events helped popularize the controversial and ambiguous concept of “fake news” [11,15]. According to Tandoc Jr. et al. [16], “fake news” is a multiform reality that encompasses diverse expressions, such as news satire, news parody, fabrication, manipulation, advertising, and propaganda.

Among the different forms and modes of disinformation, hoaxes play a prominent role. These have been defined as “all intentionally false content that appears to be true, conceived with the purpose of deceiving the public and publicly spread via any social platform or social network” [2]. In our study, we opted to use the hoax concept because the falsehoods investigated not only correspond to content disseminated in news media, but according to the extant literature, it is a concept that designates deliberate falsehoods and targets the general public through any communication channel.

Identified as key challenges of our time [17], misinformation and disinformation have been the subject of many research articles with a wide range of approaches and methodologies [18]. The deliberately misleading nature of false information makes it difficult to study and analyze, and most studies conducted thus far have focused on three aspects: (1) identification of the forms of false content, (2) the dynamics of dissemination, especially on social networks, and (3) the impact on public opinion.

A majority of studies on misinformation and disinformation have focused on politics. Research on fake news has also been conducted in the fields of science and health. Waszak et al. [19] studied a sample of news items in the Polish language from 2012 to 2017 and reported that 40% of the content was misinformation, which was extensively shared. In a study on English-language news coverage of the Zika virus, Sommariva et al. [20] found that rumors were shared on social networks three times more often than verified information. A study conducted in the US concluded that misinformation was a worrying issue, particularly due to the easy accessibility and widespread usage of social media [21].

The COVID-19 pandemic has highlighted the seriousness of the problem posed by the spread of health-related falsehoods. According to studies conducted in Spain, during the early stages of the pandemic, the health crisis created a demand for information in addition to traditional media coverage [22], and an emotional need to share relevant information that would help people make decisions about their behavior during a health crisis [23]. This created the perfect breeding ground for the spread of hoaxes.

Social networks provided the public with an alternative means of searching for information about the pandemic, and this was in line with a pattern of behavior that had already been observed in previous crises. A similar change in consumption habits occurred in China during the SARS epidemic in 2003 when the general public actively sought information from alternative sources and created alternative information channels by being information producers and disseminators [24].

Multinational surveys indicated that misinformation about COVID-19 is a global phenomenon [25], and false or misleading information is common, especially in social networks and messaging applications [26].

Misinformation about the pandemic spread rapidly in several countries, although patterns of diffusion among topics vary [27]. Many citizens in several countries viewed misinformation on this topic as highly reliable information, and it negatively affected people’s willingness to get vaccinated and comply with guidance measures [28]. Exposure to misinformation about COVID-19 had other relevant effects, including greater information avoidance and less systematic processing of information about the pandemic, as revealed by a multinational study [29].

Other regional studies, such as one conducted in India, revealed that misinformation on COVID-19 increased consistently in 2020, which may be because it was a major event that generated uncertainty and even panic in this country, as in other parts of the world [30].

Regarding the distribution of misinformation, a study conducted in India indicated that online media (especially social media) produced 94.4% of fake news, whereas mainstream media produced only 5.6% [3]. Social media became particularly relevant during the pandemic as it provided an outlet for people’s frustrations, and increased their interest at a time when worldwide lockdowns imposed physical constraints on people [31]. Different social media platforms spread different volumes of misinformation because of different interaction patterns and the peculiarity of the audience on each platform [32]. On Twitter, the rate of misinformation was higher among informal individual/group accounts than among other types of accounts [33].

A multinational study on mis- and disinformation about COVID-19 conducted by the Reuters Institute for the Study of Journalism indicated that most of the information disorders published on social media came from “ordinary people,” although those pieces of false information which came from politicians and celebrities achieved more engagement [34]. Another study based on Spain suggested that many hoaxes did not include any indicator of “cognitive authority” (e.g. a source of information that is perceived as authorized, based on its knowledge of a topic) [35]. On Twitter and Facebook, a minority of “super spreaders” exerted a strong influence on each platform, and there was evidence of coordinated sharing misinformation on the pandemic [36]. COVID-19 may have been used as a vector to spread misinformation and disinformation for political purposes [37].

Among the few studies on the actual content of misinformation, research on English-language fact checks found that the bulk of misinformation adopted “various forms of reconfiguration, where true information is spun, twisted, recontextualized, or reworked” [34]. Another study, based in Iran, found that misinformation on COVID-19 disseminated through social media included “disease statistics; treatments, vaccines and medicines; prevention and protection methods; and dietary recommendations and disease transmission ways” [38]. Researchers have also explored conspiracy theories related to COVID-19 (e.g., that it did not exist or was caused by 5G radiation). Evidence suggests that conspiratorial claims are highly unlikely to endure [39].

The current research was based on the results of a previous study on COVID-19 conducted in Spain, which found that one-third of the false information about the pandemic contained falsehoods about scientific and health-related matters and identified four main types of hoaxes: joke, exaggeration, decontextualization, and deception [2].

This study was aimed to analyze the science and health-related hoaxes of COVID-19 that spread during the pandemic. More specifically, our objectives were to (1) identify the characteristics of form and content, and the platforms used to spread science and health-related hoaxes, and (2) formulate a typology that can be used to classify the different types of hoaxes according to their connection with scientific information.

Materials and methods

Study design

We used a qualitative method [40]. More specifically, we conducted content analysis, a common methodology in social science, to obtain descriptive indicators through systematic procedures [41]. Our research objectives and methodology were inspired by previous studies that analyzed and classified misinformation models [34] and the hoaxes that were spread during the COVID-19 pandemic [2].

Data collection

We compiled all hoaxes on COVID-19 published on the websites of Spain’s three main fact-checking organizations (Maldita.es, Newtral, and EFE Verifica) over a three months period (March 11 to June 10, 2020), which started when the WHO declared the pandemic.). These are the only Spanish organizations certified by the International Fact-Checking Network (IFCN), which was created in the United States in 2015 by the Poynter Institute, an entity that assesses the quality of the work of fact-checking organizations worldwide [42].

The sample was manually selected by checking the websites of the three organizations and compiling all the items related to COVID-19. Automated or specific data extraction tools were not used. After removing the hoaxes which were repeated on one or more of these three websites, we set up a database with our sample (N = 533), which consisted of hoaxes identified by Maldita.es (N = 327), Newtral (N = 143), and EFE Verifica (N = 63).

Coding

After classifying the hoaxes in a database, a codebook was developed based on a previous study [2]. After discussing this code in two research team meetings, two independent coders carried out a pre-test in which they jointly coded 5% of the hoaxes to detect compression problems and unify the coding criteria. In the testing phase, the final analysis code was obtained, which included the variables detailed in Table 1 (the coding criteria are explained in detail in the codebook attached as Supporting Information, S1 Appendix: Codebook).

Table 1

List of variables.

1. Subject of the hoax: science/health, politics/government, other.Next, all science/health hoaxes (N = 187) were coded, and variables 2 to 10 were analyzed.

2. Platform used to spread the hoax: networks (in general), Twitter, Facebook, WhatsApp, Instagram, YouTube, and others.

3. Formats used: text, audio, image, video, other.

4. Geographical scope: local, national, international, unspecified/not applicable.

5. Type of hoax: joke, exaggeration, decontextualization, deception.

6. Topic of hoaxes related to science/health: scientific research, scientific policy and health management, advice issued to the public, and others.

7. Topic of hoaxes related to scientific research: origin of the virus, transmissibility, fatality rate, treatments, vaccines, etc.

8. Source type: anonymous, spoofed, fictitious, real.

9. Non-anonymous sources: members of the public, business, government, professional, healthcare/science.

10. Type of healthcare/science sources: researchers, international scientific organizations, national scientific organizations, health professionals, and others.

Once coding was conducted based on the final codebook, an intercoder reliability test was conducted. It consisted of the double blind codification of 126 items, a representative sample of the initial universe of 187 items (50% of heterogeneity, 5% error margin, 95% level of confidence).

The agreement between the two independent coders was tested through a Cohen’s kappa test for the parallel and blind codification of each variable, resulting in an optimal level of agreement for all the variables: Topic 1, 0,96; Topic 2, 0.95; Topic 3, 0.97; Source 1, 0.98; Source 2, 0.95; Source 3, 0.99; Geographical scope, 1; Type of hoax, 0.97 (see S2 Appendix: Statistical analysis).

A chi-squared test was used to analyze the relationship between two categorical variables to contrast the significance of the relationship between them. The results are indicated in the paper where appropriate, and are included in detail in the S2 Appendix: Statistical Analysis Appendix.

Results

Content and platforms

The hoaxes were initially subclassified into three main categories: “science and health,” “politics/government,” and “other.” (See Table 2). The primary interest was in identifying hoaxes related to science and health in the context of other social, political, and governmental issues. This category included several subcategories, such as “scientific policy and health management,” “scientific research,” and “advice issued to the public” related to the COVID-19 pandemic. The “politics” category included content that focused on political parties, party members, and government affairs at international, national, regional, or local level.

Table 2

Hoax topics.

	n	%(*)
Science and Health	187	35.08
Politics	176	33.02
Other	170	31.89
Total	533	100.00

* Percentages are rounded.

A relevant share of the hoaxes fell into the “other” category (see Table 3). Among them, scams were the most prominent. These included fake job offers, fraudulent requests for personal data, the sale or promotion of nonexistent products, and announcements of vouchers and aid for buying food and medicine. Second, hoaxes were also related to public safety. These included actions by security forces (e.g., municipal, regional, and national police, civil guards, and military emergency units) as well as control of communications and state roads, among others.

Table 3

Content of hoaxes in the “other” category.

	n	% (*)
Scams	58	34.12
Public safety	29	17.06
Shocking events	27	15.88
People’s behavior	21	12.35
Demonstrations	19	11.18
Celebrities	8	4.71
Predictions	5	2.94
Other (**)	3	1.76
Total	170	100.00

* Percentages are rounded.

** These hoaxes concerned insurance companies, employment, or pollution.

The highest number of hoaxes was observed during the first month of study. Of these, just over one-third corresponded to “science and health” hoaxes (see Table 4). The distribution of the content of hoaxes by month was not statistically significant (chi-square = 2,413; df = 4 y p-tail = 0,660), which means there was not a specific month in which a significantly higher proportion of hoaxes about unk > “science and health” was spread.

Table 4

General content of hoaxes by month.

Date	Science and Health	Politics	Other	Total
				n	% (*)
Month 1 (March 11 to April 10)	109	91	93	293	54.97
Month 2 (April 11 to May 10)	36	37	30	103	19.32
Month 3 (May 11 to June 10)	42	48	47	137	25.70
Total	187	176	170	533	100.00

*Percentages are rounded.

A total of 187 hoaxes about “science and health” were spread through various platforms, including social networks, the media, and other channels such as SMS, emails, blogs, and non-journalistic websites. Certain hoaxes were found to be spread on more than one platform. This produced a total number of frequencies (N = 218) that was greater than that of the total number of hoaxes in the initial “science and health” sample (N = 187).

Messaging and social media applications (both specified and unspecified) were the most frequently used channels for spreading hoaxes. WhatsApp was the platform used to spread the maximum number of hoaxes, followed by social networks like Twitter and Facebook, the video-sharing platform YouTube, media outlets, and other platforms (see Table 5).

Table 5

Platforms used to spread “science and health” hoaxes.

	n	% (*)
Social media (unspecified)	72	33.03
WhatsApp	54	24.77
Twitter	26	11.93
Facebook	18	8.26
Media outlets	12	5.50
YouTube	12	5.50
Instagram	4	1.83
Other	20	9.17
Total	218	100.00

*Percentages are rounded.

Content of science and health-related hoaxes

The frequencies of the three main topics were relatively similar. The most frequent hoaxes were related to scientific research, which will be further analyzed in detail in the study. This was followed by hoaxes related to scientific policy or health management, which focused on issues such as decisions by the authorities to control the spread of the virus and management of health resources. Erroneous advice on how to avoid the coronavirus was also common and included recommendations such as drinking different beverages, following diets, and engaging in practices such as gargling, inhaling steam, and consuming chlorine dioxide (see Table 6).

Table 6

Content of science and health-related hoaxes.

	n	% (*)
Scientific research	64	34.22
Scientific policy and health management	63	33.69
Erroneous advice issued to the public	51	27.27
Other	9	4.81
Total	187	100.00

* Percentages are rounded.

Most hoaxes on scientific research concerned the origin of the coronavirus; for example, they propagated that the virus was manufactured and released by China or the United States, or connected it to 5G technology. Hoaxes that recommended bogus treatments and vaccines were also common. Hoaxes related to the fatality rate and transmissibility of the virus were also detected, although to a lesser extent (see Table 7).

Table 7

Content of hoaxes relating to scientific research.

	n	% (*)
Origin of the virus	27	42.19
Treatments	16	25.00
Vaccines	10	15.63
Fatality rate	3	4.69
Transmissibility	3	4.69
Other	5	7.81
Total	64	100.00

* Percentages are rounded.

In some cases, the origin of hoaxes was related to studies that were in their initial stages and had not yet produced any conclusive results. For example, social networks were flooded with images of skin lesions caused by the coronavirus, which caused an alarm among the public [43]. Such lesions were not caused directly by the virus and were, therefore, not contagious, but rather a manifestation of the body’s immune response to the virus, or were minor skin lesions associated with inflammation of the skin’s blood vessels, a phenomenon that is still being researched.

Based on preliminary scientific results, it was claimed that smoking protected against coronaviruses, a statement that went viral on Twitter [44]. This idea arose from an article published as a preprint that suggested that the percentage of smokers among hospitalized COVID-19 patients in China was lower than that of the general population, which suggested a certain protective role. Subsequently, another preliminary, unreviewed article written by a group of French scientists suggested that nicotine could have preventive and healing effects against COVID-19. The researchers themselves warned people against jumping to premature conclusions and urged them to make a distinction between nicotine and tobacco, a product that contains thousands of toxic substances. This suggests that nicotine could be an effective treatment for acute COVID-19 in a clinical setting, and does not equate to the claim that tobacco can prevent the disease. However, this article was criticized for its serious methodological shortcomings.

Other hoaxes arose from poorly interpreted preliminary research. For example, the claim that COVID-19 was not being treated properly because it was not pneumonia but thrombosis went viral on social media [45]. This claim was based on the results of the first autopsies performed in Italy, in which disseminated intravascular coagulation was detected. Nevertheless, thrombosis was not a cause of COVID-19, but was a consequence of some of more severe cases.

Science and health-related hoaxes, geographical scope, and format

The most common type of hoax was deception, in which entirely false content was communicated and made credible through various mechanisms. These were followed by hoaxes based on decontextualization, in which the information was placed in a false context. This includes several types of incorrect attributions, such as geographical or chronological displacement. For example, one of the debunked hoaxes stated that during the pandemic, the Spanish prime minister would have a personal medical team of 14 people as a special measure against the coronavirus [46]. In fact, this was a regular medical team assisting the prime minister, not one specially created to protect him during the pandemic. Exaggerations, which accounted for 13.90% of the total, were hoaxes in which facts were represented disproportionately. For example, a headline in a digital publication claimed that singer Shakira had been “hit by coronavirus,” thereby implying that she had contracted the disease. However, the body of the article revealed that she had been suffering because her parents lived in a high-risk city during the pandemic [47], and had not been infected herself.

The few parodies detected in our sample were hoaxes intended as mockery or had satirical purposes. For example, an advertisement from a person who was offering to infect people with the coronavirus for a modest price of 60 euros was spread on social media [48] (Table 8).

Table 8

Types of hoaxes.

	n	% (*)
Deception	116	62.03
Decontextualization	43	22.99
Exaggeration	26	13.90
Parody	2	1.07
Total	187	100.00

* Percentages are rounded.

There were no major differences between the formats used for the different types of hoaxes; the most common was text, followed by photos. Videos and audio clips were less frequent. The only significant difference was that audio clips were used more frequently for exaggerations (chi-square = 45,494**; df = 30; p-tail = 0,035) (see Table 9).

Table 9

Types of hoaxes, according to format (**).

	Deception		Decontextualization		Exaggeration		Joke		Total
	n	%	n	%	n	%	n	%	n	% (*)
Text	66	51.56	31	56.36	19	54.29	2	50.00	118	53.15
Photo	27	21.09	15	27.27	6	17.14	2	50.00	50	22.52
Video	26	20.31	9	16.36	3	8.57	0	0.00	38	17.12
Audio	9	7.03	0	0.00	7	20.00	0	0.00	16	7.21
Total	128	100.00	55	100.00	35	100.00	4	100.00	222	100.00

* Percentages are rounded.

** Some hoaxes used more than one format simultaneously, so the total number of cases in this table (N = 222) was higher than the number of science and health-related hoaxes analyzed (N = 187).

International hoaxes were the most common among all types, apart from parodies. This was followed by hoaxes at the national level and those at the local level. Deception (+), decontextualization (+), and exaggeration occurred more frequently at a global level, than at regional or national level. However, decontextualization and exaggeration occurred more commonly at a national level, than at local level. In contrast, deception occurred more frequently at the local level than at a national level (see Table 10). These differences were not statistically significant (chi-square = 7,256; df = 9; p-tail = 0,6610) and, therefore, none of the types of hoaxes was significantly more prominent in a specific geographical scope.

Table 10

Types of hoaxes according to geographical scope.

	Deception		Decontextualization		Exaggeration		Joke		Total
	n	% (*)	n	% (*)	n	% (*)	n	% (*)	n	% (*)
International	57	49.14	18	41.86	10	38.46	0	0.00	85	45.45
National	23	19.83	14	32.56	9	34.62	1	50.00	47	25.13
Local	27	23.28	10	23.26	5	19.23	1	50.00	43	22.99
Not specified/ not applicable	9	7.76	1	2.33	2	7.69	0	0.00	12	6.42
Total	116	100.00	43	100.00	26	100.00	2	100.00	187	100.00

* Percentages are rounded.

Sources

Our study identified four types of sources for these hoaxes: real, anonymous, spoofed, and fictitious (see Table 11). Real sources were individuals and corporations accurately identified; anonymous sources were those that did not identify themselves; and spoofed sources were those sources to which information was falsely attributed.

Table 11

Types of sources.

	n	% (*)
Real	78	41.71
Anonymous	56	29.95
Spoofed	45	24.06
Fictitious	8	4.28
Total	187	100.00

* Percentages are rounded.

51.90% of the three non-anonymous types of sources were scientists and health professionals. This category was dominated by health professionals (44.12%), researchers (29.41%), and international scientific organizations (17.65%).

The classification of non-anonymous sources according to the type of hoax (Table 12) revealed several significant relationships (chi-square = 27,090*; df = 18; p-tail = 0,077). Healthcare and scientific sources dominated three out of the four types of hoaxes (deception, decontextualization, exaggeration), ahead of government sources, and most of these hoaxes were based on deception. For example, a hoax published on May 7, 2020 [49] claimed that coffee consumption prevented and cured the coronavirus, and falsely attributed the claim to Chinese ophthalmologist Li Wenliang, who warned about the coronavirus outbreak and ended up dying from the illness.

Table 12

Non-anonymous source types by hoax type.

	Deception		Decontextualization		Exaggeration		Joke
	n	% (*)	n	% (*)	n	% (*)	n	% (*)
Healthcare/science	48	57.83	11	40.74	11	55.00	0	0.00
Government	9	10.84	8	29.63	0	0.00	0	0.00
Member of the public	8	9.64	3	11.11	4	20.00	0	0.00
Business	5	6.02	1	3.70	4	20.00	0	0.00
Professional	7	8.43	2	7.41	0	0.00	1	100.0
Other	6	7.23	2	7.41	1	5.00	0	0.00
Total	83	100.00	27	100.00	20	100.00	1	100.00

* Percentages are rounded.

Healthcare/scientific sources accounted for a prominent share of hoaxes based on decontextualization; a fine example would be a particular hoax published on March 20, 2020 [50], which included false information about the effects of the coronavirus, and falsely attributed those claims to Spanish Doctor Quique Caubet. In this case, Dr. Caubet acknowledged that he had shared the message through social networks, although he himself had not written it. Government sources were more often quoted in decontextualization hoaxes than in deception hoaxes.

Healthcare/scientific sources contributed mainly to exaggeration hoaxes, ahead of businesses and members of the public. For example, a disproved claim by Newtral on March 25, 2020 [51] stated that sunbathing for half an hour a day boosted immunity against the virus. This misinformation was attributed to two researchers from the University of Turin, who stated that according to preliminary data from a study, it might be useful to recommend that people expose themselves to sunlight as much as possible. They, however, never claimed that sunbathing could prevent infection. Finally, only one joke or parody hoax was detected from a non-anonymous source (professional).

Discussion

General content and platforms

The increased demand for information about COVID-19 led to the spread of hoaxes on an extensive variety of subjects, including the health situation, research on the new virus, political management, and social behavior in response to the crisis. The data from our study indicated that the hoaxes with scientific and health-related content accounted for a considerable percentage (35.08%) of all false information spread during the first three months of the pandemic. Almost half of all hoaxes (43.7%), including 54.9% of science and health-related hoaxes, were spread in the first month of the pandemic. This suggests that in the early days, the strong public interest in accessing information to adapt to a novel situation, combined with a lack of information, gave rise to numerous hoaxes. However, hoaxes about science and health continued to spread consistently during the three sampling months, which makes it difficult to establish a causal relationship between the diffusion of misinformation and specific events beyond the beginning of the pandemic that coincided with the first few weeks of lockdown in Spain.

With respect to platforms, social networks, including both private messaging applications and open networks, provided the primary setting for the spread of hoaxes (82.9%), way ahead of the traditional media and other interpersonal communication channels. The results of this study are affected by our sample selection method, which limited the sample to hoaxes debunked by the three fact-checking organizations. Our results were consistent with previous reports that indicated that the usage rate of social networks increased during times of crisis because social media provided a platform for emotional communication, and were timely and uncensored by government sources [52].

This increased use of social networks went hand-in-hand with an increase in the spread of hoaxes. One of the first studies on this subject carried out in the context of the pandemic indicated that 88% of false information originated on social media platforms [34]. In fact, Spain was one of the European countries where the use of social networks increased the most during the lockdown [53] and where social media acted as the main channel for spreading hoaxes, as reflected in this research.

The large amount of scientific information hastily produced in the first few months of the pandemic created serious communication problems. Many scientific articles were published in such a short time that scientists themselves and even specialized publishers were unable to process them properly.

As mentioned, hoaxes concerning the origin of the coronavirus were among the most common of all science and health-related hoaxes. One case related to this subject was paradigmatic with respect to the way in which some hoaxes were created. One of the main sources of misinformation about the origin of the coronavirus was probably an article published in a preprint format that suggested that the new SARS-CoV-2 virus was a manufactured combination of HIV and SARS viruses [54]. This preliminary article was withdrawn by the authors within three days of being published after errors were discovered in their bioinformatics analysis and interpretation. However, it was one of the most talked about hoaxes on social networks at the time and promoted the false notion that SARS-CoV-2 was genetically engineered in a laboratory.

On other occasions, disinformation was generated when politics and scientific fraud came together. One of the most prominent cases is the hydroxychloroquine treatment scandal. Preliminary studies had shown that this compound inhibits viral replication in vitro [55]. These findings indicated that hydroxychloroquine was one of the first antivirals to be tested in severe COVID-19 cases. French microbiologist Didier Raoult, advisor to the French government in the fight against the pandemic, promptly claimed that this compound was an effective treatment for COVID-19 in humans. The WHO even included the medication in its international clinical trial. The issue was further clouded by US President Donald Trump’s revelation that he was taking hydroxychloroquine to protect himself from coronavirus. To complicate matters further, an article published in The Lancet warned that hydroxychloroquine was not only useless, but also associated with adverse effects and an increased risk of death [56]. However, the study did not follow experimental protocols and a group of scientists questioned its results. The work could not be independently verified; therefore, The Lancet was forced to retract the article. The efficacy of hydroxychloroquine became a political issue, with some “in favor” and others “against” the treatment, based primarily on ideological rather than scientific reasons.

Types of hoaxes, geographical scope, and format

Science and health-related hoaxes about COVID-19 took all the forms identified in a previous study [2]. The most frequent were deceptions based on simple untruths. Less frequent, although still prominent, were decontextualization and exaggeration hoaxes, which were created based on partially true information that was then distorted to create the hoax.

Decontextualization appears to be a misinformation strategy that is more frequent in health and science-related hoaxes (22.99%), which includes hoaxes on any topic (16.1%). However, our results do not allow us to identify distinguishing characteristics among decontextualization hoaxes in science and health compared to other types of hoaxes, as there are no significant differences in frequencies regarding format, geographical scope, or sources (except for a small difference in governmental sources, which are more frequent).

The most common formats for these hoaxes were those that required the least technical expertise. Text was the most common format, with photos, videos, and audio clips trailing behind. These data seem to indicate that any member of the public could create hoaxes, and that no special technical skills were required.

In addition, it seems that some formats are especially effective for certain types of hoaxes. For example, decontextualization frequently used photos, and exaggerations were particularly suited to audio clips. However, in our study, the format frequency varied slightly according to the type of hoax.

The fact that many of the science and health-related hoaxes were international in scope was consistent with the global nature of the COVID-19 pandemic, which prompted the public to constantly search for information in other countries, either to compare different pandemic situations or to seek valid reference points to cope with the crisis.

Generally, the credibility of hoaxes was based on the source’s epistemological authority (whether alleged or real). In the four types of sources that were identified (real, anonymous, spoofed, and fictitious), the source was usually presented as a person or institution with a good reputation and/or competence to deal with the subject in question. It was unsurprising that more than half of the sources of science and health-related hoaxes identified (51.90%) were scientists or health professionals.

Sometimes the authority of health professionals was underpinned by their status as “eyewitnesses” (whether alleged or real) to the events in question. Meanwhile, the authority of scientists (international or national researchers and scientific organizations) was based on their reputation and competence. The use of such sources made the hoax seem credible because in areas where science plays a key role, the public generally trusts scientists above friends and family as the main sources of information [57].

The classification of sources by hoax type indicates that different types of sources were used to spread different types of hoaxes. This suggests that all types of hoaxes were underpinned by similar arguments to reinforce the epistemological authority of the sources regardless of the type of source in question.

Common characteristics

Based on the study’s results and the diverse nature of the hoaxes, we can highlight some of the most common characteristics of science and health-related hoaxes spread during the COVID-19 pandemic (Table 13).

Table 13

Most common characteristics of hoaxes about science and health.

Topics	Scientific research (origin of the virus) and health management
Platform	Social networks
Format	Text
Type of hoax	Deception
Scope	International
Source	Real

Classification of hoaxes

The results of our study allowed us to identify four types of hoaxes, according to their relationship with scientific knowledge: “hasty” science, decontextualized science, badly interpreted science, and falsehood without a scientific basis. Table 14 lists the typical characteristics of each of these types in terms of origin, source, hoax type, and common topics.

Table 14

Classification of science and health-related hoaxes according to their connection to scientific knowledge.

	Origin	Sources	Type of hoax	Common topics
Hasty science	Provisional results	Real	Decontextualization, exaggeration	Scientific research (origin of the virus, treatments, vaccines)
Decontextualized science	Provisional or definitive results	Real	Decontextualization	Scientific research, scientific policy / health management
Badly interpreted science	Definitive results	Real, spoofed	Exaggeration	Scientific research, erroneous advice issued to the public
Falsehood without scientific basis	Unknown	Spoofed, anonymous	Deception	Scientific policy / health management, erroneous advice issued to the public

Limitations

The main limitation of our study is that it analyzed only those hoaxes identified by three fact-checking organizations in a single country (Spain) spread only during the first three months of the pandemic. The fact checkers followed several methods to identify the hoaxes. In some cases, they checked the information sent directly to them from the audience. In other cases, they selected information previously published by news media. This method has its limitations, as fact-checkers cannot address all the misinformation that is spread over a particular period and may have selection biases [58], and such biases could have transitively had an impact on the accuracy of our results. Another limitation is that the data we obtained cannot be easily compared to those of other studies because of the different criteria used for classification.

Our study also suggests that preprints and decontextualization may play a relevant role in creating hoaxes. A profound analysis of these relationships surpasses the aim and scope of our research and our sampling method does not allow the establishment of hoaxes originating from a preprint nor a more profound analysis of the differential characteristics of hoaxes based on decontextualization.

Conclusion

Our first objective was to analyze the characteristics of the form and content of hoaxes and the platforms used to spread them. Based on the results and discussion presented, we can state that scientific knowledge (whether rigorous or not) played a very prominent role in shaping the hoaxes related to COVID-19. A broad, diverse variety of hoaxes based on scientific knowledge was created and contributed substantially to public misinformation during the pandemic.

Our findings allowed us to identify several characteristics of hoaxes, providing relevant information that can be used as a basis for future research. They can also contribute to a better understanding of how disinformation is spread to the public and, therefore, can help improve media literacy actions to address disinformation about health and science.

Our second objective was to formulate a typology for science and health-related hoaxes. We identified four types according to their connection to scientific knowledge: hasty science, decontextualized science, badly interpreted science, and falsehood without a scientific basis. This typology can serve as a preliminary framework for future research and can help develop systems for automated detection of health and science-related hoaxes.

Many hoaxes (hasty, decontextualized, and poorly interpreted science) have an actual scientific basis, with varying degrees of rigor. In these cases, real or spoofed sources were often used to give the hoax epistemological authority and make it more credible. Sometimes, the origin was a low-quality scientific publication, either in preprint format or in a peer-reviewed article in a prestigious journal.

Decontextualization played a relevant role in health and science-related hoaxes, probably because they were easy to produce, and at the same time, were credible since they were partially true.

Our findings also suggested that preprints might be a prominent source of hoaxes. The spread of hoaxes based on provisional results underlines that scientific research requires time and substantiation, both of which are not feasible during the pandemic. This lack of time to develop comprehensive scientific research was instrumental in contributing to public misinformation.

Certain hoaxes were mere falsehoods without any scientific basis, frequently supported by spoofed or anonymous sources. Theoretically, the public should be able to detect this type of hoax more easily than hoaxes based on actual scientific knowledge. However, this requires a certain level of media and scientific literacy that not all members of the public possess.

The connection between preprints and hoaxes, as well as a deeper analysis of the differential characteristics of hoaxes based on decontextualization, present a desirable topic for future research.

Codebook.

Coding criteria and variables developed to classify the hoaxes.

(DOCX)

Click here for additional data file.

Statistical analysis.

Cohen’s kappa test was used for inter-coder reliability and the chi-square test was used for significance between variables. Coding dataset. Accessible on Zenodo. DOI:10.5281/zenodo.4895047. https://zenodo.org/record/4895047#.YLfDrLczbct.

(DOCX)

Click here for additional data file.

25 Mar 2021

PONE-D-20-35335

Health and science-related misinformation on COVID-19. A content analysis of hoaxes identified by fact checkers in Spain

PLOS ONE

Dear Dr. León,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

I am writing to let you know that your Manuscript ID PONE-D-20-35335 entitled " Health and science-related misinformation on COVID-19. A content analysis of hoaxes identified by fact checkers in Spain" which you submitted to Plos One has now been reviewed.  The comments of the reviewers are included at the bottom of this letter.

Please note that the reviewers have recommended major revisions to your manuscript, which I believe will result in an improved manuscript overall. Particularly, I suggest you attend to clarifying research goals based on a more solid and wide literature review. Therefore, I invite you to revise your manuscript, in conjunction with the comments of the reviewers.

Please submit your revised manuscript by Apr 23 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Chang Sup Park, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments:

I am writing to let you know that your Manuscript ID PONE-D-20-35335 entitled " Health and science-related misinformation on COVID-19. A content analysis of hoaxes identified by fact checkers in Spain" which you submitted to Plos One has now been reviewed. The comments of the reviewers are included at the bottom of this letter.

Please note that the reviewers have recommended major revisions to your manuscript, which I believe will result in an improved manuscript overall. Particularly, I suggest you attend to clarifying research goals based on a more solid and wide literature review. Therefore, I invite you to revise your manuscript, in conjunction with the comments of the reviewers.

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We suggest you thoroughly copyedit your manuscript for language usage, spelling, and grammar. If you do not know anyone who can help you do this, you may wish to consider employing a professional scientific editing service.

Whilst you may use any professional scientific editing service of your choice, PLOS has partnered with both American Journal Experts (AJE) and Editage to provide discounted services to PLOS authors. Both organizations have experience helping authors meet PLOS guidelines and can provide language editing, translation, manuscript formatting, and figure formatting to ensure your manuscript meets our submission guidelines. To take advantage of our partnership with AJE, visit the AJE website (http://learn.aje.com/plos/) for a 15% discount off AJE services. To take advantage of our partnership with Editage, visit the Editage website (www.editage.com) and enter referral code PLOSEDIT for a 15% discount off Editage services.  If the PLOS editorial team finds any language issues in text that either AJE or Editage has edited, the service provider will re-edit the text for free.

Upon resubmission, please provide the following:

The name of the colleague or the details of the professional service that edited your manuscript

A copy of your manuscript showing your changes by either highlighting them or using track changes (uploaded as a *supporting information* file)

A clean copy of the edited manuscript (uploaded as the new *manuscript* file)

3. Please include additional information regarding the data extraction tool used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a data extraction tool as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information, or include a citation if it has been published previously.

4. During the internal evaluation of your manuscript. We note that the current article addresses a similar research question with some overlap in the study variables of a  previous publication:  http://profesionaldelainformacion.com/contenidos/2020/may/salaverria-buslon-lopez-leon-lopez-erviti.pdf. In light of the related article, please cite and discuss the above mentioned related paper in the body of the manuscript (introduction, methods if overlap, and discussion). In particular, please discuss how your research contributes to the basis of academic knowledge in light of the above mentioned publication. Please bear in mind that our publication criteria state " If a submitted study replicates or is very similar to previous work, authors must provide a sound scientific rationale for the submitted work and clearly reference and discuss the existing literature. Submissions that replicate or are derivative of existing work will likely be rejected if authors do not provide adequate justification.

5. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

6. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The paper proposes a classification schema of articles evaluated by 3 Spanish fact checking organizations, dubbed "hoaxes" by the authors. The three-month period considered encompasses the first COVID-19 related lockdowns. The authors annotate the hoaxes for kind of media used, platform it appeared on, geographical scope, source type, and a hierarchical "type" classification. Out of 533 articles, authors consider 187 which are about "science and health", and for these, the paper provides tables of class memberships for the above annotations. The authors also provide content analysis, grouping the topics into "origin of the virus", "treatments", "vaccines", etc. They then present analysis using intersection between some of these variables, such as type of hoax vs. format, and type of hoax vs. geographical scope.

The paper suffers from a vague motivation, from the introduction, to the presentation of results, and conclusions. The Abstract, for instance, presents the results in "The "prototypical hoax"..." sentence as a hodge-podge of observations that are difficult to contextualize, and understand their importance. In the Introduction, the authors claim that the "characteristics of these hoaxes and the forms they take have not yet been brought to light", such as [1] concerning India, and I'm sure many more. Similarly, the related work section does not make a compelling case for performing this study. My point is not that observational studies are not important, but they need to be motivated by more than "nobody has done this before". If the classification system proposed here was aimed at a particular task, such as ways to intersect or detect misinformation, or particular theories about communication, the results may be easier to interpret and contextualize.

I commend the authors on coming up with a coding schema that resulted in a higher inter-annotator agreement, giving me confidence that the labels applied in this study are somehow valid.

The analysis presents some interesting insights. For instance, the fact that these hoaxes are often based on "decontextualization" and that this often happens around healthcare/scientific sources. In fact, the authors state several times that the pace of scientific publishing picked up so much that it's likely that even the publishers "were not able to process them properly". The authors provide several examples of poor-quality and retracted articles that made a splash on social media. It would be very interesting to see a systematic analysis of whether pre-publishing articles resulted in retraction and misinformation, instead of seeing this anecdotal evidence.

The fact that lots of these hoaxes were found on social media channels is emphasized by the authors as a major finding in the paper. However, this is largely affected by the article selection policy of the fact-checking websites used in this study. I am not sure about the Spanish mass media, but in the US (with which I am more familiar), plenty of falsehoods are promoted on the television and radio -- media which still have a huge audience, possibly in demographics only somewhat overlapping with that captured by the current study. I would recommend the authors contextualize their findings in this methodological limitation.

Another limitation is the paper's scope - Spain - making the insights highly localized. This is, of course, also a strength, providing points of view outside US-centric research that is so popular. However, the authors fail to really juxtapose their findings to those in other countries, or strongly contextualize the findings in the peculiarities of the Spanish COVID situation at the time. This is another area where the authors could strengthen the paper. For instance, a time visualization could provide readers insights into when most hoaxes happened, and with what major events they were associated. How the political situation may have differed from that of other countries, or influenced by outside political forces (such as Trump's push for hydroxychloroquine). From where the sources of information came (inside or outside Spain), etc.

The authors need to show statistical testing results when claiming comparison, such as on line 341, especially since the final number of documents examined (n) can shrink quite a bit. In the end, this is a sample of all potential misinformation out there -- if we are comparing, say, 5 to 6 documents, is the strength of this difference enough to claim it can be generalized?

Overall, the paper may present a few interesting insights, but is definitely not groundbreaking. It could be strengthened by motivating the labeling exercise by a particular task (say, automated hoax detection, or science decontextualization monitor), a particular theory (ex: pre-publishing scientific news results in misinformation), or emphasis on the peculiarity of the Spanish situation and its interaction with the ongoing events there and abroad. I believe considering the "why is this interesting" question by the authors may make this observational study much more interesting.

Small remarks:

Abstract - "importance of time" should be "importance of timing"?

[1] Akbar, Syeda Zainab, et al. "Misinformation as a Window into Prejudice: COVID-19 and the Information Environment in India." Proceedings of the ACM on Human-Computer Interaction 4.CSCW3 (2021): 1-28.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Yelena Aleksandrovna Mejova

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

2 Jun 2021

The responses to reviewers are included in the attached document "Response to reviewers"

Submitted filename: Response to reviewers.docx

Click here for additional data file.

19 Oct 2021

PONE-D-20-35335R1

Health and science-related misinformation on COVID-19. A content analysis of hoaxes identified by fact checkers in Spain

PLOS ONE

Dear Dr. León,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

The reviewer points out that the statistical analysis has not been performed rigorously. Also please make clear conceptual distinctions among hoaxes, rumors, misinformation and fake news.

I look forward to receiving your revision.

==============================

Please submit your revised manuscript by Dec 03 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at  https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Chang Sup Park, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

The reviewer points out that the statistical analysis has not been performed rigorously. Also please make clear conceptual distinctions among hoaxes, rumors, misinformation and fake news.

I look forward to receiving your revision.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: No

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: This is an interesting paper on Spanish hoaxes on COVID-19, and I enjoy reading the result. There are some comments or suggestions for further improvement:

Overall comments:

1. The authors have tried to addressed all comments raised in a previous round of review .

the statistical analysis could be improved.

The result is descriptive and lacks knowledge discovery, it would be really nice if the author could consider more comparison or in-depth analysis.

Specific comments：

The statistical analysis has not been performed appropriately and rigorously. For example, for chi-square, which should be used to compare two variable, seems to be used to compare multiple variables at one time (or haven’t been reported properly) as shown in one example below (line 451-453), and p value should not be reported as “p-tail<0,05”:"A significant difference between formats was that audio clips were used more frequently in the case of exaggerations (20.0%) (Table 8) (chi-square= 45,494**; df=30 ; p-tail<0,05). "

The authors use content analysis with Codebook by two coders, there are 3 topics however, in result, only four types are listed, which doesn't match: Healthcare/science,Healthcare/science,Member of the public ,Business.（Table 11. Non-anonymous source types by hoax type.)

I would suggest this research to distinguish the difference between hoaxes, rumors, misinformation and fake news. There have been many studies on misinformation, rumors and fake news, so why choose hoaxes only? Comparison between hoaxes and misinformation and further analysis could be added to enhance the importance of the article.

In the introduction, the author claim to" establish a classification system that can be used to explain the narrative mechanisms that underpin the credibility of this information "(line. 58,59), however, instead of building up a system, this paper demostrates a descriptive analysis on one case only.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

23 Nov 2021

-The result is descriptive and lacks knowledge discovery, it would be really nice if the author could consider more comparison or in-depth analysis.

Our results provide knowledge that goes beyond the specific case study. As we explain the conclusion, we have

1. Identified the most common characteristics of health and science related hoaxes.

These characteristics provide relevant information that can be used as a basis for future research. They can also contribute to a better understanding of how disinformation is spread to the public and, therefore, can help to improve media literacy actions about health and science (p. 32).

2. Formulated a typology of this type of hoaxes, according to their relationship to scientific information.

This typology can work as a basis for future research and can help to develop systems for automated detection of health and science- related hoaxes (p. 34).

-The statistical analysis has not been performed appropriately and rigorously. For example, for chi-square, which should be used to compare two variable, seems to be used to compare multiple variables at one time (or haven’t been reported properly) as shown in one example below (line 451-453), and p value should not be reported as “p-tail<0,05”:"A significant difference between formats was that audio clips were used more frequently in the case of exaggerations (20.0%) (Table 8) (chi-square= 45,494**; df=30 ; p-tail<0,05).

The chi-square test was used to compare two variables but the results were not reported properly. We have now amended this in the main text and Appendix 2.

-The authors use content analysis with Codebook by two coders, there are 3 topics however, in result, only four types are listed, which doesn't match: Healthcare/science,Healthcare/science,Member of the public, Business.（Table 11. Non-anonymous source types by hoax type.).

We have amended the manuscript in order to differentiate correctly the four types of hoaxes (deception, decontextualization, exaggeration, and joke/parody.) and the three topics of Science and health-related hoaxes (Scientific research, Scientific policy and health management, Erroneous advice issued to the public):

The classification of hoax content revealed relatively similar frequencies among the three main topics (p. 17).

Healthcare/scientific sources predominated in three of the four hoax types (p. 24).

- I would suggest this research to distinguish the difference between hoaxes, rumors, misinformation and fake news. There have been many studies on misinformation, rumors and fake news, so why choose hoaxes only? Comparison between hoaxes and misinformation and further analysis could be added to enhance the importance of the article.

We have clarified this in the text. As the manuscript indicates, the conceptualization regarding the forms of falsehood in publicly disseminated information is based on a “theoretical distinction between disinformation and misinformation”. The first concept refers to deliberate deception, while the second covers inadvertent falsehoods. Ultimately, these two categories distinguish between lying (voluntary) and error (involuntary).

Within these two general categories there are multiple concrete expressions. Specifically, research has explored modalities such as conspiracy theories (Craft et al. 2017), rumors (Alkhodair, 2020) and hoaxes (Braun & Eklund, 2019). In the field of journalism and the media, the so-called “fake” (Tandoc et al., 2021) or “false” (Andı & Akesson, 2020) news have also been widely investigated.

In our study, we have opted for using the concept of “hoaxes”, because the falsehoods investigated do not correspond only to content disseminated in news media and because, according to the existing literature, it is a concept that designates deliberate falsehoods targeted to the general public through any communication channel.

References

Alkhodair, S. A., Ding, S. H., Fung, B. C., & Liu, J. (2020). Detecting breaking news rumors of emerging topics in social media. Information Processing & Management, 57(2), 102018.

Andı, S., & Akesson, J. (2020). Nudging Away False News: Evidence from a Social Norms Experiment. Digital Journalism, 9(1), 106-125.

Braun, J. A., & Eklund, J. L. (2019). Fake news, real money: Ad tech platforms, profit-driven hoaxes, and the business of journalism. Digital Journalism, 7(1), 1-21.

Craft, S., Ashley, S., & Maksl, A. (2017). News media literacy and conspiracy theory endorsement. Communication and the Public, 2(4), 388-401.

Tandoc Jr, E. C., Thomas, R. J., & Bishop, L. (2021). What is (fake) news? Analyzing news values (and more) in fake stories. Media and Communication, 9(1), 110-119.

-In the introduction, the author claim to" establish a classification system that can be used to explain the narrative mechanisms that underpin the credibility of this information "(line. 58,59), however, instead of building up a system, this paper demostrates a descriptive analysis on one case only.

We have amended this phrase and clarified the research objectives in the introduction (p. 5):

The overall objective of this study is to analyze the science and health related hoaxes about COVID-19 that were spread during the pandemic. More specifically we aim to (1) identify the characteristics of form and content and the platforms used to spread science and health related hoaxes; and (2) formulate a typology that can be used to classify the different types of hoaxes, according to its connection with scientific information.

Submitted filename: Response to reviewers.docx

Click here for additional data file.

7 Feb 2022

Submitted filename: Reviewers Comments. PONE-D-20-35335R2.docx

Click here for additional data file.

2 Mar 2022

Responses to reviewers have been included in the document "Response to reviewers", previously uploaded to the system.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

7 Mar 2022

Submitted filename: Reviewer 3. Final Comments and Recommendation. PONE-D-20-35335R2.docx

Click here for additional data file.

8 Mar 2022

Response to reviewer’s comments

1. p.6, line 148: Delete the comma after ‘… COVID-19’.

Done.

2. p.7, line 168: Add the word ‘current’, i.e. to read ‘The current research …’

Done.

3. p.8, line 188: Add the word ‘period’ immediately before ‘(March 11 to June …)

Done

4. p.10, line 209: Split last word into two, i.e. to read ‘… intercoder reliability’

Done.

5. p.10, line 209: I don’t think there is a need for the word ‘and’ between ‘double’ and ‘blind’ as this is usually written simply as ‘double blind’

Done.

6. p.11, line 324: Remove comma and second full stop after ‘prominent’

Done.

7. Table 6: Place the label (descriptor) on the line above the table, consistent with all other tables.

Done.

8. p.19, lines 359-60: After the colon, replace the commas with semi-colons, i.e. to read ‘real; …. ; …. ‘ and …’

Done.

9. p.23, line 454: Use a capital ‘T’ for the correct title of The Lancet.

Done.

All Tables containing numbers:

10. Line up the ones under ones, tens under tens and hundreds under hundreds in all columns; and

Done.

11. Remove brackets in the footnotes to the tables, i.e. remove brackets surrounding * and **.

Done.

Tables containing words:

12. Adjust either width of some columns or size of font to ensure that words are not split incorrectly (e.g. across two lines) and that commas appear immediately after the relevant word.

Done.

Submitted filename: Response to reviewers comments.docx

Click here for additional data file.

14 Mar 2022

Health and science-related disinformation on COVID-19. A content analysis of hoaxes identified by fact checkers in Spain

PONE-D-20-35335R4

Dear Dr. León,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Chang Sup Park, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

18 Mar 2022

PONE-D-20-35335R4

Health and science-related disinformation on COVID-19: A content analysis of hoaxes identified by fact-checkers in Spain

Dear Dr. León:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Chang Sup Park

Academic Editor

PLOS ONE