Analysis of YouTube videos as a source of information for myocarditis during the COVID-19 pandemic.

OBJECTIVE: In this study, we aimed to examine the content, reliability, and quality of YouTube video contents concerning myocarditis and its association with the COVID-19 for the first time in the literature.
METHODS: The most viewed 50 videos were included in the analysis. The time since the videos were uploaded, video length, type of image (real/animation), video content, qualify of the uploaders, the number of daily and total views, likes, dislikes, comments and VPI were recorded. The reliability of the videos was determined using the modified DISCERN criteria for consumer health information, while the quality was determined with the GQS.
RESULTS: The mean length of the videos was found as 6.25 ± 5.20 min. Contents of the videos included general information, COVID-19, vaccination, diagnosis, patient experience and treatment. The most common contents were regarding COVID-19 and vaccination by 44%. The uploaders of the videos were classified as physicians, hospital channels, health channels, patients and others. Fourteen (28%) videos were directly uploaded by physicians. The most viewed, liked and disliked videos were uploaded by health channels. The mean VPI score was calculated as 92.89 ± 12.29. The mean DISCERN score of all videos was 3.88 ± 0.77 and the mean GQS score was 3.63 ± 0.85. Reliability and quality of the videos were moderate.
CONCLUSION: YouTube videos on myocarditis have mostly focused on the associations between myocarditis and COVID-19 disease and vaccination. Health-related contents on YouTube should be subjected to peer review and quality assessment.

Introduction

Myocarditis is an inflammatory cardiac disease that is predominantly caused by viruses, but other infectious agents such as bacteria, protozoa and fungi can also lead to this disorder [1]. During the current COVID-19 pandemic, the subject of myocarditis has started to attract more attention. COVID-19 is known to induce myocardial injury. Suggested mechanisms of myocardial injury, including myocarditis, in COVID-19 patients include cytokine storms induced by an imbalance in the inflammatory response of T cells [2]. On the other hand, there are several case reports on myocarditis induced by COVID-19 vaccination [3]. However, there is no scientific evidence to clarify whether myocarditis is related to the vaccination or is only a coincidental occurrence remains unclear [4]. Increasing speculative news in the media and especially social platforms have negatively affected not anti-vaxxers, but people who hesitate to get vaccinated. As a result, many people are increasingly seeking information about myocarditis and its relationships with COVID-19 and vaccination on the Internet, and especially YouTube videos instead asking for professional medical counseling as it is the case in many other health-related issues.

YouTube is one of the most viewed Internet platforms with more than 2 billion users [5] which composed 95% of all internet users [6]. YouTube has significant potential to provide health-related information, but since the videos are uploaded not only by healthcare professionals, but also by lay persons, accuracy, reliability and quality of health-related YouTube videos are questionable [7]. Furthermore, using these videos for medical information has several advantages, such as complex medical language, lack of peer review and disorganized contents [8]. Previous studies on YouTube videos have revealed that one-third to half of the videos pertaining to several diseases provide misleading and/or unreliable information [9].

There are numerous studies in the literature analyzing the reliability and quality of YouTube video contents in a wide range of diseases from the fields of orthopedics, obstetrics, genital cosmetic procedures to emergency conditions, radiology and cardiology [10-13].

The unknown reliability and quality of myocarditis-related YouTube videos raises the need for evaluation of these video contents, especially since they may pose a risk for misleading people about vaccination for COVID-19, which is the vital component of global fight against the disease. Therefore, the objective of this study was to examine the content, reliability, and quality of YouTube video contents concerning myocarditis and its association with the COVID-19 for the first time in the literature.

Materials and methods

Search strategy

This study was designed and executed by two experienced cardiologists as a detailed analysis of YouTube videos concerning myocarditis. Searching terms were determined as “myocarditis after covid vaccine”, “covid vaccine and myocarditis” and “myocarditis symptoms” based on the Google Trends [14]. The search terms were entered to the YouTube video-sharing platform and sorted using YouTube’s relevance based ranking. The first 50 videos were used in the analysis, because an internet analysis revealed that more than 90% of users choose from the searching results listed within the first pages [15].

Exclusion criteria

Videos with news content, live broadcast videos, non-English videos, those longer than one hour, videos without audio and/or visual information, duplicate videos, videos for entertainment purposes and those irrelevant to the study subject were excluded from the study.

Video parameters extracted

After elimination of the excluded videos, the remaining most viewed 50 videos were included in the analysis. Data extracted for each video included: the time since the videos were uploaded, video length, type of image (real/animation), video content, qualify of the uploaders (physicians, hospital channels, health channels, patients, lay persons etc.), the number of daily and total views, likes, dislikes, comments. The video power index (VPI) was calculated using the formula: VPI = (like × 100/[like + dislike]) x (views/day)/100, based on the previous studies [12].

Evaluation of the videos and scoring

The 50 videos included in the analysis were evaluated and scored by the two observers (2 experienced cardiologists) on August 1, 2021. The evaluation was performed at the same time period, but in separate settings to prevent bias and being influenced by each other. The reliability of the videos was determined using the modified DISCERN criteria for consumer health information, while the quality was determined with the Global Quality Scale (GQS).

DISCERN criteria The DISCERN is an instrument used for scoring quality of consumer health information on treatment choices. In this study, we used the short form of the DISCERN that was adapted by Singh et al. [16]. This form of DISCERN questioning tool consists of 5 items evaluated based on a 5-point Likert scale. According to DISCERN, video scores > 3 points indicates good, a score of 3 points shows moderate and scores < 3 points refer to poor reliability and should not be used by patients. Table 1 shows the items of DISCERN scale.

Table 1

DISCERN scale

#	Content analysis	Reliability
1	The explanations given in the vide are clear and understandable	Poor
2	Useful reference sources (publication cited, valid studies etc.)	Poor
3	The information given in the video is balanced and unbiased	Moderate
4	Additional sources of information are mentioned	Good
5	The video evaluates controversial or uncertain areas	Good

DISCERN scores of 1 and 2 indicated poor reliability, 3 moderate reliability and 4 and 5 showed good reliability of the videos evaluated

Global Quality Scale (GQS) GQS was used to assess overall quality of the reviewed videos. GQS has a 5-point Likert structure based on the quality, flow and ease of use of the examined videos. The GQS scores range between 1 (poor quality) and 5 (high quality). Higher points given to the GQS scale indicate higher quality of video contents. The five items of the GQS are given in Table 2.

Table 2

Global quality scale (GQS)

#	Content analysis	Quality
1	Not at all useful for patients: poor quality and flow, missing information	Poor
2	Limited usefulness: generally poor quality and flow, limited information is provided	Limited
3	Moderate usefulness: suboptimal flow, some important information is discussed, but others are poor	Moderate
4	Useful: good quality and flow, relevant information is provided, but some topics are not covered	High
5	Very useful: excellent quality and flow, covering all relevant information	Excellent

GQS scores: 1 indicates poor quality, 2: of limited quality, 3: moderate quality, 4: high quality and 5: excellent quality of the video evaluated

Ethics aspects

Ethics approval was waived as only videos were used in the study, and human or animal subjects were not used. Since all data used in this study was publicly available, no permission was needed from YouTube. The names of organizations/people who uploaded the videos were kept confidential.

Statistical analysis

Statistical analysis was performed using SPSS version 23.0 (SPSS, Statistical Package for Social Sciences, IBM Inc., Chicago, IL, USA) statistical software. Data obtained in this study were summarized as frequencies (n) and percentages (%) for the categorical variables and mean ± standard deviation for the continuous variables. Compliance between the two independent observers was evaluated with a Spearman's correlation analysis and Cronbach α coefficients. p values < 0.05 were considered statistically significant.

Results

Contents of the reviewed videos concerning myocarditis were divided into six categories as general information, COVID-19, vaccination, diagnosis, patient experience and treatment. The most common contents were regarding COVID-19 and vaccination by 44% (n = 22 videos). Distribution of the evaluated videos by contents is given in Fig. 1.

Fig. 1

Contents of the videos concerning myocarditis

The uploaders of the videos were classified as physicians, hospital channels, health channels, patients and others. Fourteen (28%) videos were directly uploaded by physicians. Distribution of the videos according to the qualification of the uploaders is shown in Fig. 2.

Fig. 2

Distribution of the video uploaders

Of all videos, 12 (24%) were animated and 38 (76%) were real images. The total number of views was 16,201,765. The most viewed, liked and disliked videos were uploaded by health channels. The most viewed video was regarding COVID-19 vaccination as expected, by 5,720,289 views. The video was uploaded by a hospital channel on June 12, 2021. The highest number of comments was found in the videos uploaded by physicians. The number of views, likes, dislikes and comments according to the general characteristics of the videos is given in Table 3.

Table 3

The distribution of views, likes, dislikes and the comments of the videos according to the general features

	n (%)	View	Like	Dislike	Comment
Image type
Real	38 (76)	6.510.763	79.670	14.205	19.115
Animation	12 (24)	4.542.741	33.604	821	2.577
Uploaders
Physician	14 (28)	3.899.884	37.863	3.114	14.140
Hospital channel	10 (20)	1.097.636	8.318	313	2.324
Health channel	22 (44)	5.968.036	60.985	11.531	4.610
Patient	2 (4)	42.664	6.025	61	598
Other	2 (4)	45.284	83	7	20
Video contents
COVID-19	6 (12)	332.698	7.555	218	1.175
Vaccination	16 (32)	4.424.978	56.928	13.487	16.877
General info	18 (36)	4.811.376	37.988	1.043	2.952
Diagnosis	2 (4)	7.792	152	2	14
Patient experience	4 (8)	1.383.139	10.435	265	652
Treatment	4 (8)	93.251	216	11	22

The mean length of the videos was found as 6.25 ± 5.20 min and the mean number of daily views as 6.897 ± 25.543. The mean VPI score was calculated as 92.89 ± 12.29. The videos uploaded by hospital channels had the highest number of daily views. In addition, the videos with vaccination contents had the highest number of daily views among all videos. The mean length, number of daily views and VPI scores according to the video characteristics are explained in Table 4.

Table 4

The mean length, daily views and VPI scores of the videos based on the general features

	n (%)	Length (min)	Daily views	VPI (%)
		Mean ± SD	Mean ± SD	Mean ± SD
Image type
Real	38 (76)	6.25 ± 5.20	13.297 ± 36.895	93.41 ± 10.66
Animation	12 (24)	6.25 ± 04.40	1.427 ± 2.668	94.65 ± 6.33
Uploaders
Physician	14 (28)	9.21 ± 05.03	8.175 ± 12.485	93.67 ± 5.83
Hospital channel	10 (20)	5.21 ± 04.49	22.255 ± 58.726	94.54 ± 5.94
Health channel	22 (44)	5.38 ± 05.07	8.425 ± 36.556	93.02 ± 10.93
Patient	2 (4)	1.33 ± 02.18	13.04 ± 11.44	99.49 ± 0.50
Other	2 (4)	2.50 ± 05.16	7.58 ± 5.10	91.62 ± 4.44
Video contents
COVID-19	6 (12)	3.28 ± 04.53	191 ± 327	93.60 ± 6.75
Vaccination	16 (32)	9.36 ± 05.05	31.940 ± 59.774	89.25 ± 16.54
General info	18 (36)	6.46 ± 05.07	523 ± 1.218	95.99 ± 4.84
Diagnosis	2 (4)	1.33 ± 02.18	4.6 ± 2.3	98.66 ± 1.33
Patient experience	4 (8)	3.21 ± 04.55	186 ± 302	98.76 ± 1.68
Treatment	4 (8)	2.00 ± 05.44	11 ± 7	94.54 ± 4.65

The reliability and quality of the videos about myocarditis were evaluated and scored independently by two experienced cardiologists. Accordingly, the mean DISCERN score of all videos was 3.88 ± 0.77 and the mean GQS score was 3.63 ± 0.85. The mean DISCERN scores given by the Observer 1 and Observer 2 were found as 3.86 ± 0.76 and 3.90 ± 0.79, respectively. The mean GQS scores given by the Observer 1 and Observer 2 were found as 3.64 ± 0.88 and 3.62 ± 0.83, respectively. The DISCERN and GQS given by the observers according to the general characteristics of the videos are given in Table 5.

Table 5

DISCERN and GQS scores of the videos according to the general features

	n (%)	DISCERN	GQS
		Mean ± SD	Mean ± SD
Image type
Real	38 (76)	3.76 ± 0.80	3.46 ± 0.85
Animation	12 (24)	4.25 ± 0.77	4.17 ± 0.85
Uploaders
Physician	14 (28)	4.32 ± 0.77	3.89 ± 0.83
Hospital channel	10 (20)	3.80 ± 0.80	3.00 ± 0.85
Health channel	22 (44)	3.82 ± 0.78	3.84 ± 0.87
Patient	2 (4)	2.75 ± 0.84	2.50 ± 0.97
Other	2 (4)	3.00 ± 0.87	3.75 ± 0.89
Video contents
COVID-19	6 (12)	3.42 ± 0.80	3.17 ± 0.85
Vaccination	16 (32)	4.21 ± 0.83	3.78 ± 0.87
General info	18 (36)	4.06 ± 0.77	3.81 ± 0.86
Diagnosis	2 (4)	3.50 ± 0.86	3.75 ± 0.85
Patient experience	4 (8)	2.88 ± 0.82	3.00 ± 0.90
Treatment	4 (8)	3.63 ± 0.83	3.50 ± 0.94

Based on the DISCERN scores, reliability of the contents was found as poor in 2 (4%), moderate in 7 (14%) and good in 41 (82%) videos. When GQS scores were examined, the content quality was found as poor in 1 (2%), limited in 5 (10%), moderate in 22 (44%), high in 19 (38%) and excellent in 3 (6%) videos. The videos uploaded directly by physicians were of good reliability and moderate quality. Compliance analysis of the DISCERN and GQS scores given by the two observers showed an excellent agreement (Table 6).

Table 6

Correlation between the observers in terms of the mean DISCERN and GQS scores

	Mean ± SD	p	r	Cronbach α
DISCERN 1	3.86 ± 0.76	< 0.01	0.952	0.935
DISCERN 2	3.90 ± 0.79
GQS 1	3.64 ± 0.88	< 0.01	0.961	0.894
GQS 2	3.62 ± 0.83

Discussion

Today, while there is a global fight against the COVID-19 pandemic, there is an ongoing debate on the effects of the disease on myocarditis. Furthermore, some COVID-19 vaccines have been associated with the development of myocarditis. Diaz et al. associated myocarditis with some vaccines, including MRNA vaccines [17]. The Center for Disease Control and Prevention recently declared a possible association between COVID-19 mRNA vaccines and myocarditis, especially in young patients with an incidence of 4.8 per 1 million cases [18]. Some studies have reported that there is no definitive scientific evidence on this association, and postvaccination adverse effects are known to be benign and self-limited [3], while others suggested that myocarditis is one of the complications that manifest following COVID-19 infection [18, 19]. The potential benefit of global vaccine campaigns against COVID-19 is much more greater than self-limited myocarditis. However, especially, speculative news on media, social platforms and the Internet pose a risk of misleading people who hesitate to being vaccinated and contribute to the infodemic, which is defined as “the dangers of misinformation phenomena during the management of an outbreak [21]. As a result, these news and grapevine lead people to increasingly seek answers for their health-related problems on the Internet, and particularly YouTube.

YouTube is the most commonly used video-sharing website worldwide. Any individual is allowed to upload videos at free cost after registration. YouTube has become a great social media platform and a generous source of medical information for patients [7]. The view counts of YouTube videos have been dramatically increased during the pandemic [22]. In the present study, we tried to determine the reliability and quality of YouTube videos concerning myocarditis. In our study, the most commonly viewed video content was general information regarding myocarditis followed by COVID-19 vaccination and myocarditis.

General performance of a video may be measured with the view, like dislike and comment counts. These parameters can provide basic information about the performance of a video. In the present study, the mean number of views was found as 324,035, likes as 3022, dislikes as 336.52 and comments as 434. Undoubtedly, these numbers show great differences between the studies depending on the subject of the research.

The qualification of the uploaders has a significant effect on the reliability and quality of YouTube videos. As is known, anyone can freely upload videos on this platform that has still no a regulation or peer review process for the uploaded videos. In the present study, most videos (44%) were uploaded by health channels followed by physicians (28%) and hospital channels (20%). Only 8% of the videos were uploaded by lay persons. In a study by Cetin et al. examining YouTube videos on hyperglycemia, the videos were most commonly uploaded by health channels (54%) [12]. In another study by Fode et al., 42% of the YouTube videos pertaining to erectile dysfunction were uploaded by health channels [23]. In this regards, studies have similar results indicating that health channels are the most common video uploaders. On the other hand, most of the videos uploaded by health channels about any medical topic include physician(s) as narratives. Again in our study, the highest view and like counts were found in the videos uploaded by health channels.

In the present study, the mean Video Power Index that indicates the popularity of videos based on daily views, like and dislike counts was calculated as 92.89. The highest VPI index score was found in the videos uploaded by patients (99.49) and those with patient experience (98.76). In a study by Yurdaisik et al. including YouTube videos concerning breast cancer, the mean VPI was reported as 94.10 [13]. In a study by Kuru et al., YouTube videos on rotator cuff tears were examined and the mean video power index value was found as 90.6 [10]. In another study by Cakmak et al. on the YouTube videos concerning umbilical hernia, the mean VPI was found as 90.25, while the videos uploaded by non-doctors had a higher VPI compared to the videos uploaded by doctors [24]. Several studies have reported that YouTube videos uploaded by lay persons are of poor quality, although these videos have higher rates of like and VPI values [25]. We attributed this result to the fact that video contents uploaded by healthcare physicians may have medical language that may not be understood by everyone, making the videos including patient experience that are narrated by lay persons more popular.

In our study, the reliability of YouTube videos pertaining to myocarditis was measured using the modified DISCERN criteria. Accordingly, the mean DISCERN score of the overall videos was found as 3.88. When DISCERN scores were analyzed according to the general characteristics, the highest mean DISCERN score was found as 4.32 for the videos uploaded directly by physicians followed by 3.82 for the videos uploaded by health channels. The lowest DISCERN score was 2.75 for the videos uploaded by patients. Based on the DISCERN scoring, reliability was found as poor in 4%, moderate in 14% and good in 82% of the videos about myocarditis. Based on the contents, the highest DISCERN score (4.21) was given to the videos containing the association of myocarditis and COVID-19 vaccination, indicating that these videos contain reliable information to guide and motivate people for being vaccinated.

Because the vast majority of health-related YouTube studies in the literature have been conducted in a wide range of diseases and medical conditions, making a direct comparison for reliability score is challenging. Besides, some studies have used 30-item DISCERN scoring, while the others have utilized the modified 5-item DISCERN as in our study. Nevertheless, looking at the several studies in the literature; the mean DISCERN score of YouTube videos on erectile dysfunction was found as ≤ 3 with the highest score given to the videos uploaded by medical institutions [23]. The mean DISCERN score was reported as 2.35 by Akpolat et al. [26], 1.91 by Wu et al. [27], and 3.00 by Onder et al. [28]. In general, the above-mentioned studies have reported higher DISCERN scores for the videos uploaded by healthcare professionals. Our higher DISCERN scores compared to the other study might be caused by the fact that association of myocarditis and COVID-19 is a hot topic and especially health channels and other professionals are trying to provide more reliable information to promote people for COVID-19 vaccination.

In the present study, the mean GQS score was found as 3.63, which indicates moderate quality. Again the highest GQS score was given to the videos uploaded by physicians (3.89) followed by health channels (3.84), while the lowest score was given to those uploaded by patients (2.50), which shows poor quality. Based on GQS scoring, the quality of the viewed YouTube was found as moderate. The mean GQS score was reported as 4.00 by Onder et al. [28], 3.08 by Ustdal et al. [29], and 2.46 by Oztermeli et al. [30] with higher GQS scores were given to the videos uploaded by health professionals. Again our higher GQS scores might be attributed to the fact that the majority of the videos were uploaded by health professionals.

In our study, the reliability and quality of YouTube videos concerning myocarditis were found as moderate. However, the videos uploaded by health professionals, including physicians and health channels, received higher scores compared to the videos uploaded by hospitals (may include indirect ads) and lay persons.

Study limitations

This study has several limitations. First, evaluation of the videos is subjective and there is still no validated tool for objective evaluation. Second, YouTube search results are dynamic, and change when new contents are uploaded. Therefore, this study shows the reliability and quality of the videos at a specific time.

Conclusion

The YouTube videos concerning myocarditis were found to have moderate reliability and quality. YouTube videos on myocarditis have mostly focused on the associations between myocarditis and COVID-19 disease and vaccination. Although most videos were uploaded by healthcare professionals, it seems that the lack of a regulation especially for health-related content limits the usefulness of the videos. Health-related contents on the Internet and particularly on YouTube should be subjected to peer review and quality assessment before they can be published.