Personal protective equipment in Covid-19: Evidence-based quality and analysis of YouTube videos after one year of pandemic.

BACKGROUND: The correct use of personal protective equipment (PPE) during the Covid-19 pandemic is mandatory to minimize the contagion risk. The current study aimed to evaluate quality information of YouTube videos on PPE use during the pandemic.
METHODS: Using Google Trend tool, the frequency of worldwide YouTube and Google searches for "donning and doffing" was examined. We queried YouTube with terms related to donning and doffing of PPE. Validated quality information assessment tools were used.
RESULTS: From the December 1, 2019 to the January 31, 2021, according to YouTube and Google searches, both peaks occurred in April 2020 (69.5% and 72.0%, respectively). Of all videos, 144 were eligible for the analyses. According to misinformation tool, 90 (62.5%) videos contained inaccuracies. The median DISCERN Section 1 ranged from 3 to 5. The median DISCERN Section 3 was 4. According to Global Quality Score, 8.3% (n = 12), 14.6% (n = 21), 22.9% (n = 33), 30.6% (n = 44) and 23.6% (n = 34) were classified as poor, partially poor, moderate, partially good and excellent quality videos, respectively.
CONCLUSIONS: Nowadays, YouTube may be recommended as a reliable source of information. Nevertheless, a not negligible number of videos contained inaccuracies. Future authors should improve videos contents to provide more complete information.

Introduction

More than 17,000 health care providers worldwide have lost their lives due to Covid-19 infection and its complications, since the beginning of the pandemic. , Currently, one year after the outbreak declaration, it is estimated that approximately 120 million people got infected and more than 2.5 million died in 223 different countries. , The transmission mechanisms of Covid-19 have been largely studied by the scientific community.5, 6, 7, 8 In this regard, international official recommendations primarily focused on the correct use of personal protective equipment (PPE), with the aim to break down the viral spreading.9, 10, 11 However, the use of PPE in health care settings may be highly dangerous if the staff executing the donning and doffing procedures is not adequately skilled and trained to the purpose.

Over the years, online medical resources and instruments have increased. YouTube is the main free video platform, containing over 100 million videos and more than 1.9 billion users in over 100 different countries. , YouTube, as well as other social medias,15, 16, 17, 18, 19 is commonly used as a source of medical information and it is successfully used as a learning and self-training tool, among health care workers as well. , Previous studies have already evaluated the quality of the videos concerning several medical fields.22, 23, 24, 25 To the best of our knowledge, no previous investigators examined the quality information of YouTube videos on the use of PPE for Covid-19 patients care. The aim of the current study was to evaluate the evidence-based quality of YouTube videos on the correct donning and doffing procedures and usage of PPE at one year from the beginning of Covid-19 pandemic.

Materials and methods

Web interest assessment

We evaluated the interest of the worldwide web users on the donning and doffing procedures. We queried Google Trends with the terms “donning and doffing,” using the following search settings: “worldwide,”, “period from December 1, 2019 to January 31, 2021,” “all the categories.” The trends of Google search and YouTube search were independently recorded. To examine changes in search frequency over time, we compared the relative frequency of searches between all 14 months (December 2019-January 2021).

Search strategy, selection criteria, and videographic characteristics

We queried YouTube with 40 combinations of keywords (Supplementary Table 1). Internet cache cleared, the log-out from any personal account and a proxy located in United Kingdom through a free VPN software were set to minimise the search history and the geographic related biases.

The first 50 videos were examined for each of the 40 keywords combinations. A total of 2,000 videos was achieved. The following exclusion criteria were applied: non-English language, live-actions, interviews, news reports, webinars or lectures nontopic related, non-medical content, non-donning/doffing procedures and duplicate videos. Only the videos published after the December 1, 2019 were included. A total of 144 videos were eligible for the analyses ( Fig 1 ).

Fig 1

PRISMA diagram depicting inclusion and exclusion criteria of YouTube videos search.

For each of the 144 videos included, the following variables were collected on the 25th of February 2021: length (seconds), views, persistence time on YouTube (days), thumbs-up, thumbs-down, subscribers, authoring entity (public health institutions [such as public and government health institutions and associations, general health institutions], academic hospitals [such as academic hospitals and academic institutions, universities], nonacademic hospitals [such as nonacademic hospitals and institutions, health care centers, private practice hospitals], alternative medicine channels [such as digital learning platforms, health information websites], individuals and others [such as news channels and PPE producers companies]) and target audience (health care workers, general public, and visitors).

Strategies and instruments for the assessment of videos content

Videos contents were assessed independently by 2 medical doctors, both staff members in the “Intermediate Care Covid-19 Unit” of the University of Naples “Federico II.” A third investigator, a medical doctor staff member from the same Unit, adjudicated any differences and consensus was achieved among all reviewers.

Videos content assessment according to the centers for disease control and prevention (CDC) recommendations

Videos contents were evaluated according to the CDC recommendations, in which the donning and the doffing procedures were reported as step-by-step checklists (Supplementary Table 2). For each of the donning and doffing steps, 5 possible scores from 1 (strongly disagree) to 5 (strongly agree) were assigned.

Misinformation tool

After evaluating the content of the videos according to CDC recommendations, the investigators judged if inaccuracies were present in reporting information. The inaccuracies reasons were categorized according to the following non-exclusive groups, as previously done: (1) Incomplete presentation of data; (2) Inconsistent with CDC recommendations; (3) Weak or inconsistent supporting data; (4) Misrepresentation of data; (5) False or without any supporting data (Supplementary Table 3).

DISCERN instrument

The DISCERN instrument is a validated tool used to evaluate the good quality evidence-based and the reliability of the information given. , The original version of the DISCERN tool, made by The British Library, was used for the study. The instrument is based on a scale of 5 possible answers per question (1 = no; 2 = partially no; 3 = partially; 4 = partially yes; 5 = yes). DISCERN Section 1 (questions 1 to 8 aimed to evaluate if a publication is reliable) and Section 3 (question 16 aimed to evaluate the overall rating of the publication) were used. DISCERN Section 2 was not applicable to the current study, since it aimed to evaluate information on treatment choices. However, to date, the only possible preventive treatment available for people dealing with Covid-19 patients is the correct usage of PPE.

Global quality score (GQS)

The GQS is a validated tool assessing the quality, the feasibility and the clinical utility of each video.32, 33, 34 It was firstly used to assess the internet quality content on Crohn's Disease and Ulcerative Colitis. The original English version of the GQS was used for the porpoise of the study. Five possible scores from 1 (poor quality, poor flow, most of information missing, not at all useful) to 5 (excellent quality, excellent flow, completely accurate information, very useful) were assigned.

Statistical analysis

Descriptive statistics were presented as medians and interquartile ranges (IQR) for continuously coded variables or counts and percentages for categorically coded variables. Chi-square test and Kruskal-Wallis test examined the statistical significance in proportions and medians differences. In all statistical analyses, R software (www.rproject.org) environment for statistical computing and graphics (R version 4.0.0) and Microsoft Excel 2019 were used. All tests were 2-sided with a level of significance set at P < .05.

Results

Web interest

From the December 1, 2019 to the January 31, 2021, an increased user's interest on both Google and YouTube was recorded ( Fig 2 ). Specifically, according to YouTube search, the relative search frequency ranged from 2.4 to 21.6%, with the peak recorded in April 2020 (69.5%). Similarly, according to Google search, the relative search frequency ranged from 3.2 to 22.6%, with the peak recorded in April 2020 (72.0%).

Fig 2

Chart-line plot depicting relative frequency of worldwide search for “donning and doffing” on both Google and YouTube searches, observed between the December 1, 2019 and the January 31, 2021.

Videographic characteristics

Of 144 videos ( Table 1 ), the median length was 298.5 seconds (IQR: 190.0-472.2; range: 25.0-2442.0), the median number of views was 4443.5 (IQR: 1118.8-19484.8; range: 6.0-2675484.0) and the median persistence time on YouTube was 314.0 days (IQR: 268.0-337.0; range: 50.0-381.0). Moreover, the median number of thumbs-up, thumbs-down and subscribers was 27.0 (IQR: 7.0-120.0; range: 0-44000.0), 2.0 (IQR: 0-7.0; range: 0-1600.0) and 9570.0 (IQR: 1000.0-42800.0; range: 10.0-2500000.0), respectively. Of all videos, 36.8% (n = 53), 12.5% (n = 18), 13.2% (n = 19), 22.2% (n = 32), 6.9% (n = 10), and 8.3% (n = 12), were produced by public health institutions, academic hospitals, nonacademic hospitals, alternative medicine channels, individuals and others, respectively. Additionally, 88.2% (n = 127), 10.4% (n = 15) and 1.4% (n = 2) were targeted to health care workers, general public and visitors, respectively.

Table 1

Videographic characteristics of 144 YouTube videos on donning and doffing of personal protective equipment, recorded on the 25th of February 2021

Videographic characteristics		Value
Length, sec	Median (IQR)Range	298.5 (190.0-472.2)25.0-2442.0
Views, n	Median (IQR)Range	4443.5 (1118.8-19484.8)6.0-2675484.0
Persistence time, d	Median (IQR)Range	314.0 (268.0-337.0)50.0-381.0
Thumbs-up, n	Median (IQR)Range	27.0 (7.0-120.0)0-44000.0
Thumbs-down, n	Median (IQR)Range	2.0 (0-7.0)0-1600.0
Subscribers, n	Median (IQR)Range	9570.0 (1000.0-42800.0)10.0-2500000.0
Authoring entity, n (%)	Public health institutionsAlternative medicine channelsNonacademic hospitalAcademic hospitalsIndividualsOthers	53 (36.8)32 (22.2)19 (13.2)18 (12.5)10 (6.9)12 (8.3)
Target audience, n (%)	Healthcare workersGeneral publicVisitors	127 (88.2)15 (10.4)2 (1.4)

Videos content results

Videos content results according to CDC recommendations

According to the donning CDC recommendations step-by-step checklist, the median scores recorded ranged from 3 (for step 1) to 5 (for step 2). According to the doffing CDC recommendations step-by-step checklist, the median scores recorded ranged from 4 (for steps 2-5) to 5 (for steps 1 and 6) ( Supplementary Fig 1).

Misinformation tool results

According to misinformation tool, 90 videos were defined as inaccurate. Specifically, 90.0% (n = 81), 50.0% (n = 45), 46.7% (n = 42), 35.6% (n = 32) and 24.4% (n = 22) were inaccurate for incomplete presentation of data, for inconsistent with CDC recommendations, for weak or inconsistent supporting data, for misrepresentation of data and for false or without any supporting data, respectively ( Table 2 ).

Table 2

Videos containing inaccurate or non-evidence-based claims about donning and doffing of personal protective equipment (PPE) according to Centers for Disease Control and Prevention recommendations, recorded on 25th of February 2021

Inaccuracy reasons or non-evidence-based claims	Videos, n (%)
Incomplete presentation of data	81 (90.0)
Inconsistent with CDC recommendations	45 (50.0)
Weak or inconsistent supporting data	42 (46.7)
Misrepresentation of data	32 (35.6)
False or without any supporting data	22 (24.4)

DISCERN instrument results

The median DISCERN for the Section 1 ranged from 3 to 5. Specifically, the highest median score was recorded for the question 5. Conversely, the lowest median score was recorded for the question 4, 7, and 8. The median DISCERN for the Section 3 was 4. Moreover, statistically significant differences were recorded when DISCERN Section 1 and Section 3 were compared according to the videos authoring entity. Specifically, videos produced by public health institutions obtained a median score ≥4 in 8 out of 9 questions. Conversely, videos produced by individuals obtained a median score ≤3 in 8 out of 9 questions (Table 3 A).

Table 3

DISCERN instrument Section 1 and Section 3 (A) and Global Quality Score (B) of 144 videos recorded on the 25th of February 2021

	Overall	Public health institutions 53 (36.8%)	Alternative medicine channels 32 (22.2%)	Nonacademic hospitals 19 (13.2%)	Academic hospitals 18 (12.5%)	Others 12 (8.3%)	Individuals 10 (6.9%)	P-value
a. DISCERN Instrument – Section 1 & Section 3
1.Are the aims clear?	4	5	5	5	5	4	3	< .001
2.Does it achieve its aims?	4	4	4	4	4	3	2.5	< .001
3.Is it relevant?	4	4	4	4	4	3	2	< .001
4.Is it clear what sources were used to compile the publication?	3	4	3	3	3	2	2	.001
5.Is it clear when the information used or reported was produced?	5	5	5	5	5	5	5	.1
6.Is it balanced and unbiased?	4	4	4	4	4	3	2.5	< .001
7.Does it provide details of additional sources of support and information?	3	4	3	3	3	2	2.5	< .01
8.Does it refer to areas of uncertainty?	3	3	3	3	3	2.5	2	.06
16.Based on the answer to all the above questions, rate overall quality of the publication	4	4	4	4	4	3	2.5	< .01
b. Global Quality Score
1.Poor quality	12 (8.3)	3 (5.7)	2 (6.2)	0 (0)	0 (0)	3 (25.0)	4 (40.0)	< .001
2.Partially poor quality	21 (14.6)	11 (20.8)	4 (12.5)	0 (0)	3 (16.7)	1 (8.3)	2 (20.0)	< .001
3.Moderate quality	33 (22.9)	11 (20.8)	8 (25.0)	3 (15.8)	3 (16.7)	6 (50.0)	2 (20.0)	< .001
4.Partially good quality	44 (30.6)	13 (24.5)	9 (28.1)	13 (68.4)	7 (38.9)	1 (8.3)	1 (10.0)	< .001
5.Excellent quality	34 (23.6)	15 (28.3)	9 (28.1)	3 (15.8)	5 (27.8)	1 (8.3)	1 (10.0)	< .001

GQS results

Of all videos, 8.3% (n = 12), 14.6% (n = 21), 22.9% (n = 33), 30.6% (n = 44) and 23.6% (n = 34) were classified as poor, partially poor, moderate, partially good and excellent quality videos, respectively. According to videos authoring entity, the highest rate of excellent quality videos was recorded in videos produced by public health institutions (28.3%, P < .001) and by alternative medicine channels (28.1%, P < .001). The highest rate of poor-quality videos was recorded in videos produced by individuals (40.0%, P < .001) and by others (25%, P < .001, Table 3B).

Discussion

The current study aimed to evaluate the evidence-based quality and accuracy of YouTube videos on the correct usage of PPE for the care of Covid-19 patients. To the best of our knowledge, no previous investigators examined this topic. We addressed this void and identified several noteworthy observations.

First, from the December 1, 2019 to the January 31, 2021, we observed an increased worldwide interest on the donning and doffing procedures, both on Google (Δ = 19.2%) and on YouTube (Δ = 19.4%). The peak occurred in April 2020 during the first pandemic wave. In consequence, most of the users interested in the topic obtained information from videos uploaded on the web. Thus, it is mandatory to evaluate the quality of information given in the videos, because learning wrong ways of PPE usage may lead to higher contagion risks. In the context of a new sanitary emergency outbreak, videos uploaded on YouTube provide a quicker and easier access to the contents of interest, compared to the information available in the scientific literature at the moment of the search. Indeed, the process of studies publication on the international libraries, such as PubMed or Cochrane Library, takes time due to multiple revision-making steps. For this reason, the contents shown in the videos should be correctly reported.

Second, of all videos examined, approximately 40% were produced by public and government health institutions and approximately 90% were targeted to the health care workers, who are the most exposed professionals to the virus. In consequence, it is important that YouTube videos explaining donning and doffing steps report accurate and high-quality information in order to avoid mistakes. In this regard, we recorded higher medians for all the doffing steps (ranged from 4 to 5), relative to the donning steps (ranged from 3 to 5). It is important to underline how the doffing, more than donning, is the most dangerous procedure, due to the prolonged contact with infected patients during the care. In the future, it will be important for the video authors to maintain good standards on the high-quality represented steps and give more attention to the low-quality represented ones, providing a more complete and exhaustive information.

Third, we recorded inaccurate contents in 90 videos (62.5%), according to the misinformation tool based on the CDC recommendations. Specifically, the main inaccuracy reason was the incomplete presentation of data. For example, we observed that several YouTube videos have given partial information on the step of washing hands, even though it is widely known that this procedure decreases the risk of infection. , Thus, future videos should focus more their attention on the worst represented aspects.

Fourth, according to DISCERN instrument, the median of the question 16, which is a surrogate of the overall quality, was 4. Moreover, videos produced by public health institutions showed higher quality, relative to videos produced by other entities. Furthermore, according to GQS, 78 (54.2%) videos were evaluated at least as good quality. Moreover, we observed that, among 85 (59.0%) videos produced by both public health institutions and alternative medicine channels, more than half were evaluated as good or excellent quality. Conversely, among 22 (15.2%) videos produced by individuals or others, more than half were partially poor quality or below. The above considerations imply that YouTube videos on donning and doffing of PPE during Covid-19 pandemic show a reasonable quality, especially the ones produced by official institutions and alternative medicine channels. In conclusion, internet users should first consider videos published by these kinds of authoring entities, which may represent the most reliable contents sources on this topic.

Taken together, we observed that the interest peak on both Google and YouTube occurred in April 2020, concordantly to the first pandemic wave. It confirms how YouTube was highly used to promptly acquire information on donning and doffing of PPE for Covid-19 patients care. Moreover, overall reliability and quality of YouTube videos on this topic were reasonable, as evidenced by high DISCERN score and high number of good and excellent quality videos. Nevertheless, a not negligible number of videos contained inaccuracies. In the future, it will be mandatory for the authors to give more attention to the low-quality items, maintaining good standards in the high-quality ones, providing the most exhaustive information possible. Finally, we have listed the top-quality videos among all the ones included in the current study as they may be effective tools for health care workers training during the pandemic (Supplementary Table 4).

The current study is not devoid of limitations. First, YouTube search results rely on Google proprietary search algorithms, based on user's previous search activities and location. However, the log-out from any personal accounts and the change of location via VPN proxy were performed. Second, only English-language videos were included in the final sample. Other languages videos could provide different information. Third, some reliable or non-reliable videos might be missed, due to our search terms. However, we used 40 keywords combinations in order to minimize selection errors. Fourth, quality assessment videos were subjectively evaluated. However, 3 investigators were independently involved to analyse video contents. Fifth, YouTube is a constantly expanding multimedia platform and the contents may rapidly change significantly with new updates over time.

Conclusions

During the Covid-19 outbreak, YouTube was used as an easy and quick access source of information on donning and doffing of PPE, especially during the first pandemic wave. According to DISCERN instrument and GQS, YouTube may be recommended as a reliable source on correct usage of PPE for the Covid-19 patients care. In the future, it will be important for the video authors to maintain good standards on the high-quality represented steps and give more attention to the low-quality represented ones, providing a more complete and exhaustive information.