Educational quality of YouTube videos on musculoskeletal ultrasound.

INTRODUCTION/
OBJECTIVES: A progressively growing number of rheumatologists have integrated musculoskeletal ultrasound (MSUS) as a prized tool in their daily clinical practice over the past two decades. Lately, YouTube is often used by healthcare professionals as an educational tool. This study aimed to analyze the quality of educational information available on YouTube pertaining to MSUS.
METHOD: We performed a search using the keyword "musculoskeletal ultrasound" on YouTube in January 2021. Of the first 200 videos screened, 147 met the inclusion criteria. The quality and reliability of the videos were evaluated using the global quality score (GQS) and modified DISCERN score, and their popularity was assessed using the Video Power Index (VPI).
RESULTS: Of the 147 videos evaluated, 58 (39.5%) were of high quality, 30 (20.4%) were of moderate quality, while 59 (40.1%) were of low quality. The modified DISCERN score and the duration of the videos were significantly higher in the high-quality videos than the moderate- and low-quality videos (p < 0.001 for all). The median number of views per day, likes, comments, and the popularity of the videos assessed with VPI were significantly higher in the high- and moderate-quality group compared to the low-quality videos (p < 0.001, p < 0.001, p = 0.007, and p < 0.001, respectively).
CONCLUSION: The quality of YouTube videos as an educational source on MSUS is unsatisfactory. Professional organizations and physicians should consider creating more high-quality MSUS videos. YouTube should consider filtering out poor-quality videos in collaboration with experts in the field. Key Points • Although rheumatologists are increasingly using ultrasound as an extension of their exam, accessing musculoskeletal ultrasound training is still facing some challenges. • YouTube is commonly used as an educational source by medical students and healthcare professionals. • Approximately 60% of the videos related to musculoskeletal ultrasound on YouTube revealed high and moderate quality, while the remaining 40% were of low quality and not suitable for medical education. • A filtering mechanism is required to improve the quality of YouTube content on MSUS.

Introduction

Ultrasound (US) is a non-invasive, dynamic, and relatively inexpensive imaging modality that allows a multi-planar evaluation of the musculoskeletal system [1]. Although traditionally performed mainly by radiologists, this imaging technique has become an essential tool for many rheumatologists in their clinical practice in the last two decades [2]. The musculoskeletal US (MSUS) has empowered rheumatologists to diagnose, predict, and observe disease outcomes by linking clinical manifestation with imaging in a more advanced way [3]. With the increasing use of MSUS, the need for training has emerged [4]. Guidelines have been developed and educational courses have been organized through regulated efforts by professional organizations, such as the European League Against Rheumatism (EULAR) and the American College of Rheumatology (ACR) [5]. Despite all these efforts, the MSUS training of rheumatologists is still not at the desired level [1].

In a study conducted by the British Society for Rheumatology, 75% of rheumatologists stated that the reason why they did not use ultrasound was the lack of training on MSUS [1]. As MSUS is a highly applicator-dependent imaging method, appropriate training is extremely important [6]. Some rheumatologists learn how to perform MSUS from an expert. Guidance by a mentor is considered an appropriate form of US training. Regrettably, only a restricted number of experts have adequate time to train [1]. In addition, considering that the education sector has been disrupted by the ongoing COVID-19 pandemic, similar to all the other sectors, e-learning may represent an alternative and a complementary form to traditional MSUS training [6, 7]. The online MSUS course organized by EULAR is a striking example for online education, which has focused on teaching basic skills related to MSUS since 2012 and assists hands-on MSUS learning [1].

The internet and social media have become an indispensable part of daily life in today’s digital world. YouTube is one of the most popular social media sites that first started to serve in 2005 [8]. As of March 2021, more than 500 h of video were uploaded to YouTube per minute and viewers watched over a billion hours of video every day [9]. In recent years, medical students and physicians have increasingly turned to YouTube for medical education. Nonetheless, since videos posted on this platform have varying quality and reliability, there is a potential risk of spreading inaccurate information, which can pose a significant challenge in providing optimum healthcare [10].

The quality of YouTube videos for various types of medical information has been analyzed previously [11]. However, to our knowledge, the quality of online video content regarding MSUS has not yet been analyzed. Therefore, the current study was designed to assess the quality of educational information of YouTube videos related to MSUS, which is easily accessible and potentially an alternative resource.

Materials and methods

Search strategy

A YouTube search (www.youtube.com) was performed on January 15, 2021. The default relevance mode on YouTube was used for screening, as most viewers do. We accessed the YouTube website using Google Chrome’s incognito mode to filter prior search history. As most viewers often view the first few pages of search results [12], the first 200 videos were included in the study using the keyword “musculoskeletal ultrasound.” Previous studies have also used a similar path [13, 14]. We also performed an additional analysis using the search term “rheumatological musculoskeletal ultrasound” to assess the differences between the original and the additional search results. All the videos were watched and analyzed by two independent, EULAR MSUS–certificated rheumatologists (O.Z. and M.E.O.). Any divergence between the authors was solved by reconsideration and consensus.

Exclusion criteria

Only the videos in English language were included in the analysis. Videos that were not related to MSUS, duplicate videos, music videos, and those with no audio were excluded. These exclusion criteria were applied under the guidance of previous studies [14, 15]. In addition, we excluded videos exceeding 1 h considering that they were too long for the most viewers to watch, as also described by previous studies (15,16).

Data collection

Total days on YouTube, number of views, views per day, number of likes and dislikes, number of comments, and the duration of the videos were recorded for further analyses. The contents of the videos were also recorded as shown in Table 1.

Table 1

Contents of the videos included in the sample

MSUS overview	37 (25.2%)
Introduction to MSUS	13 (8.8%)
Advantages of MSUS	2 (1.4%)
Artifacts and pitfalls in US	2 (1.4%)
US-guided injections	5 (3.4%)
Muscle US	1 (0.7%)
Tendon US	4 (2.7%)
TMJ US	1 (0.7%)
Shoulder US	21 (14.3%)
Elbow US	11 (7.5%)
Wrist and hand US	23 (15.6%)
Hip US	3 (2.0%)
Knee US	10 (6.8%)
Ankle and foot US	14 (9.5%)

MSUS musculoskeletal ultrasound, US ultrasound, TMJ temporomandibular joint

Video popularity

Video popularity was assessed using the Video Power Index (VPI), which was calculated based on the proportion of the number of views, likes, and dislikes over the number of days since upload: (likes × 100/(likes + dislikes) × (views/day)/100 [17].

Evaluating educational value with reliability and quality

Reliability was rated using a 5-point modified DISCERN tool adapted by Singh et al. [13] from the original DISCERN tool developed by Charnock et al. [18]. Higher scores denoted better reliability. The quality of the videos was rated with 5-point Global Quality Scale score, which was developed by Bernard et al. [19] to evaluate the quality of information presented on websites. Scores of 1–2 points were considered to indicate low quality, 3 points moderate quality, and 4–5 points high quality [20].

Statistical analysis

All statistical analyses were conducted using SPSS v. 22.0 (IBM SPSS Statistics for Windows; Armonk, NY: IBM Corp). The Shapiro–Wilk test was used to check the normality of data distribution. Continuous variables were presented as median (interquartile range) values and categorical variables were given as number or percentages for the non-normally distributed data. The Kruskal–Wallis test was performed to compare more than two independent groups. Pairwise comparisons were undertaken using the Mann–Whitney U test if a significant difference was found in the Kruskal–Wallis test. The inter-reviewer agreement was determined with Cohen’s kappa coefficient.

Results

Of the 200 videos screened, 17 were determined to be irrelevant, 15 were advertisements, nine had a duration longer than 1 h, four were in languages other than English, four were music videos, two did not have sound, and two were duplicates, and therefore they were excluded from the study. The remaining 147 videos that met the inclusion criteria were included in the sample and analyzed (Fig. 1).

Fig. 1

Flowchart of the screening done for the 200 videos

Basic characteristics of the videos

The distribution of the analyzed videos over the years is shown in Fig. 2. In general, there was an increase in the number of uploaded videos over the years, with a peak in 2020. The median number of days on YouTube was 2413 (range, 1292–3229). The median length of the videos was 4.21 (range, 2.0–14.42) minutes. The median number of views and views per day were 3418 (range, 1206–9716) and 1.73 (range, 0.71–4.71), respectively. The median GQS score was 3.0 (range, 2.0–4.0). The median modified DISCERN score was 3.0 (range, 2.0–4.0). Cohen’s kappa score representing interobserver agreement was 0.876 for the GQS score and 0.916 for the modified DISCERN score. The median VPI score was 1.37 (range, 0.55–4.49). The main characteristics of the videos are demonstrated in Table 2.

Fig. 2

Graph showing the distribution of the analyzed videos over the years

Table 2

Basic characteristics of the analyzed videos

Video metrics
Days on YouTube	2413 (1292–3229)
Number of views	3418 (1206–9716)
Number of views per day	1.73 (0.71–4.71)
Number of likes	12 (4–43)
Number of dislikes	1 (1–4)
Number of comments	1 (1–3)
Duration (minutes)	4.21 (2–14.42)
Like ratio	100 (93.85–100)
View ratio	1.59 (0.43–5.14)
Popularity
VPI	1.92 (0.54–5.47)
Reliability and quality
DISCERN	3.0 (2.0–4.0)
GQS	3.0 (2.0–4.0)

Data presented as median (interquartile range) values

DISCERN modified DISCERN score, GQS Global Quality Scale score, VPI Video Power Index

Video content

Among the analyzed 147 videos, 37 (25.2%) contained information on MSUS overview, followed by 23 (15.6%) on wrist and hand US, and 21 (14.3%) on shoulder US. The remaining topics were about introduction to MSUS, advantages of MSUS, artifacts and pitfalls, US-guided injections, muscle US, tendon US, temporomandibular joint US, elbow US, hip US, knee US, and ankle and foot US.

Video presenters

The majority of the presenters were radiologists, followed by rheumatologists and sports physicians. There was no statistically significant difference detected between the profession of presenters in terms of quality (p = 0.055). Seventy-nine videos were presented by experts in the field, while the profession of the presenters was unknown for 68 videos. There was no significant difference between these two groups regarding the median number of views (14,743; range, 464–12,865 vs 11,612; range, 1703–9399), views per day (7.21; range, 0.61–6.82 vs 17.95; range 0.77–3.08), and VPI (6.93; range 0.54–6.12 vs 17.51; range, 0.57–2.60) (p = 0.542, p = 0.529, and p = 0.167, respectively). However, the median number of comments and the median length of video duration were significantly higher in the videos presented by experts (p = 0.007 and p < 0.001, respectively). The median scores of the modified DISCERN (3.66; range, 3.0–4.0 vs 1.91; range, 2.0–2.0) and GQS (3.89; range, 3.0–5.0 vs 2.19; range, 2.0–2.0) were also significantly higher in the videos presented by experts indicating high reliability and quality (p < 0.001 for both).

Quality of the videos

The videos were classified into three categories according to GQS, with the scores of 1–2 indicating low quality, 3 moderate quality, and 4–5 high quality. The proportion of the low- and high-quality videos was nearly the same (40.1%, 39.5%, respectively), while the percentage of moderate-quality videos was 20.4% (Fig. 3).

Fig. 3

Proportion of the low- and high-quality videos

Concerning the quality of the videos, significant differences were noted. The modified DISCERN score and the duration of videos were significantly higher in the high-quality group than the moderate- and low-quality groups (p < 0.001 for both). VPI, number of views per day, number of likes, and number of comments were significantly lower in the low-quality videos than the high-quality and moderate-quality videos (p < 0.001, p < 0.001, p < 0.001, and p = 0.007, respectively). In contrast, in the low-quality group, the number of days on YouTube was detected to be higher compared with the remaining groups (p < 0.001). Moreover, the number of views was significantly higher in the moderate-quality group than the other groups (p = 0.043). The comparison of the videos according to the quality groups is presented in Table 3.

Table 3

Comparison of the analyzed videos according to their quality stratification

	High quality	Moderate quality	Low quality	P
	58 (39.5%)	30 (20.4%)	59 (40.1%)
Video metrics
Days on YouTube	1546 (831.25–2609)	2401 (828.25–3006)	3229 (2312–3229)	< 0.001*
Number of views	3682 (880.75–18,351.50)	7420 (1376.50–16,865.75)	2744 (1468–6938)	0.043*
Number of views per day	3.63 (0.99–9.66)	3.22 (0.65–9.18)	1.08 (0.57–2.46)	< 0.001*
Number of likes	33 (13–172.25)	21 (6.25–69.50)	5 (3–10)	< 0.001*
Number of dislikes	2 (2–4.25)	1.5 (0.75–4)	1 (0–3)	= 0.219
Number of comments	1 (1–5)	1 (1–3.25)	0 (0–1)	= 0.007*
Duration (minutes)	19.22 (8.42–29.4)	4.13 (1.30–8.03)	2.17 (0.42–3.21)	< 0.001*
Popularity
VPI	3.34 (0.88–9.29)	2.66 (0.57–8.81)	0.85 (0.39–1.53)	< 0.001*
Reliability
DISCERN	4.0 (4.0–4.0)	3.0 (2.0–3.0)	2.0 (1.0–2.0)	< 0.001*

Data presented as n (%) or median (interquartile range)

Comparisons between the groups were performed using the Kruskal–Wallis test

*Values of P < 0.05 were accepted as significant and marked in bold

DISCERN, modified DISCERN score; VPI, Video Power Index

Additional search with the keyword “rheumatological musculoskeletal ultrasound”

After following the original search strategy including the keyword “musculoskeletal ultrasound,” we also conducted an additional analysis using the search term “rheumatological musculoskeletal ultrasound” to assess differences. Of the 200 videos screened with the keyword “rheumatological musculoskeletal ultrasound,” 105 met the inclusion criteria and were recorded for further analysis. Thirty-two of the analyzed videos were also present in the original research. The median number of days on YouTube (1338; range, 333–2608) was significantly lower than the original search results (p = 0.005), indicating that these videos had been added more recently. Unlike the original search results, 25.5% of the videos focused on specific rheumatic conditions, such as ultrasound imaging in gout and rheumatoid arthritis. The median duration of the videos was 7.28 (range, 2.13–29.01) minutes, indicating no significant difference compared to the original search results (p = 0.109). While the median number of the view count (3418; range, 1206–9716) was significantly lower than the original search results (p = 0.002), the median number of views per day was 0.93 (range, 0.45–5.02), with no significant difference (p = 0.710). Although the median numbers of likes (7; range, 1–37), dislikes (1; range, 1–3), and comments 0 (range, 0–2) were statistically lower than the original study results (0.039, 0.029, and 0.034, respectively), the median number of VPI scores (0.90; range, 0.11–4.82) did not significantly differ between the original and additional search results (p = 0.109). Considering reliability and quality, the median DISCERN score was 3.0 (2.0–4.0) and the GQS score was 3.0 (3.0–4.0), demonstrating no significant difference compared to the original search results (p = 0.274 and p = 0.216, respectively).

Discussion

Over the past two decades, a growing number of rheumatologists have integrated MSUS into their daily clinical practice as a valuable imaging instrument [4]. MSUS has a profound function in the diagnosis and treatment of rheumatic diseases, as well as determining disease activity and progression, especially when combined with power Doppler US [2]. In addition, MSUS has the capacity to accurately guide interventions, such as joint aspiration and intra-articular injections [4]. However, previous studies have demonstrated barriers pertaining to training opportunities related to MSUS [21]. YouTube is a steadily growing online video resource providing easy access to videos. In recent years, the increase in the number of people using YouTube for health-related research has attracted attention, and the popularity of this platform has been progressively increasing [22]. YouTube is not limited to patient education, and it also has considerable potential to educate healthcare professionals [11]. However, the current study determined that the YouTube videos on MSUS had insufficient quality to be used as an educational source.

We evaluated 147 videos with a total viewership of approximately two million and a total duration of 25.7 h, suggesting that YouTube is a trendy source of medical information on MSUS. The oldest video was uploaded on October 22, 2010, whereas the most recent was posted on December 9, 2020. There was sufficient content on almost every subject related to MSUS, with most videos including an overview of MSUS and the remaining videos covering a wide range of topics, from the ultrasound of all joints to US to ultrasound-guided injections. An additional analysis with the keyword “rheumatological musculoskeletal ultrasound” demonstrated that there were also videos available on the specific ultrasound findings of rheumatic diseases, such as gout and rheumatoid arthritis. Rheumatologists should take into account this information when searching for ultrasound videos on a specific rheumatological topic. Most of the videos were presented by radiologists as expected, followed by rheumatologists and sports physicians. As primary specialists in musculoskeletal and rheumatic diseases, rheumatologists should consider uploading more videos on MSUS.

Many studies examined the quality and accuracy of YouTube videos on a diversity of medical topics as educational tools. Previous studies on the quality of medical education materials on YouTube revealed a significant amount of poor quality [11]. Kanlioz et al. and Ferhatoglu et al. reported that the overall quality of the educational YouTube videos on laparoscopic inguinal hernia operations and sleeve gastrectomy was low [23, 24]. In a study by Nason et al., 18.4% of the YouTube videos regarding male urethral catheterization were considered as useful, 49% somewhat useful, and 32.7% not useful [25]. Fischer et al. reported 61.5% of the videos on knee arthrocentesis were categorized as educationally useful and 38.5% as unhelpful [26]. While 60% of the videos analyzed in our study with high and moderate quality were deemed acceptable for educational purposes, 40% with low quality did not meet the required standards for MSUS learning. As a result, the quality of the YouTube videos aimed at improving the practical skills of healthcare professionals on specific topics was determined to be generally unsatisfactory. However, considering that more than half of the videos are of acceptable quality for education in some studies including ours, this popular video-sharing resource can be used for medical education by selecting appropriate videos.

In our study, as expected, the modified DISCERN score indicating high reliability was significantly higher in the high-quality videos. Similarly, Garg et al. and Ng et al. stated that the useful videos had the highest DISCERN scores [12, 27]. In addition, the duration of the videos was significantly higher in the high-quality videos in our sample. In previous studies, Ovenden et al. [28] and ReFaey [29] et al. did not find any relation between video duration and quality. However, Rittberg et al. [30] reported a positive correlation between video duration and usefulness, which is in agreement with our study. Long video duration may increase the comprehensibility and adequacy of video content.

In a study evaluating YouTube videos on cardiopulmonary resuscitation, Murugiah et al. stated that 48% of the videos were presented by those with unknown credentials [31]. Similar to this study, we observed that the identity and profession of the presenters were not known for a significant portion of the videos (46%). The reliability and quality scores of the videos were significantly higher in the videos presented by experts indicating high reliability and quality. Additional care should be taken when selecting YouTube videos to be used for training purposes in MSUS, and those presented by the experts should be preferred.

A study conducted by Staunton et al. on scoliosis demonstrated that higher quality information was associated with a lower number of views [32]. In other studies, audience interaction parameters on influenza pandemic (Pandey et al.) and spondyloarthritis (Elangovan et al.) did not statistically significantly differ between the useful group and the remaining videos [33, 34]. In the current study, the low-quality videos demonstrated the lowest viewer engagement and interaction parameters, such as the number of views, likes, and comments significantly differed according to video quality. For the evaluation of video popularity in a more comprehensive way, it is recommended to use VPI, which combines the number of views, likes, and dislikes in a formula [35]. Ferhatoglu et al. and Mangan et al. revealed that as the quality of the videos increased, VPI decreased [25, 36]. Similarly, Moon et al. reported that high-quality videos were not popular [35]. In contrast, our study revealed that the videos with high and moderate quality had higher VPI scores than the low-quality videos. These differences in viewership and VPI compared to previous studies can be attributed to most YouTube viewers in our study being healthcare professionals with a higher level of knowledge than non-healthcare viewers. Therefore, an increase in online interaction and popularity may have been detected in the high-quality group. In this context, filtering YouTube videos according to the preferences of viewers can lead to the identification of reliable educational videos in MSUS.

In the current study, the number of days on YouTube was detected to be higher in the low-quality group than the remaining groups. This result indicates that the quality of old videos was lower than that of the recent ones. Thus, viewers should consider that the quality of old videos is generally poor when searching for medical information. Poor quality videos not having been removed from this platform is not consistent with YouTube’s corporate identity. In her letter posted in January 2021, YouTube CEO, Susan Wojcicki, stated that they had removed more than 500,000 videos spreading misinformation related to COVID-19 since February, 2020. Videos presenting other health-related information should also be evaluated by experts and removed from the website if they are of low quality or misleading. Wojcicki also noted that they had recently created new partnerships with the American Public Health Association, Cleveland Clinic, and Mayo Clinic to make health information more accessible and understandable and were ready to establish further cooperation for this purpose [37]. YouTube should also consider collaborating with professional organizations such as ACR and EULAR for filtering low-quality videos on rheumatology education including MSUS. In addition, professional organizations should increase the number of new videos with high quality and reliability on YouTube.

Over the last few decades, rheumatologists have shown a growing interest in MSUS. There is an increasing need for the education and training of rheumatologists through the widespread use of MSUS [38]. The integration of MSUS into rheumatology has been widely adopted in European countries, such as Italy, Germany, and the UK, followed by the USA and Canada. Professional organizations, such as EULAR and ACR, now offer training courses on MSUS [39]. Online US courses can be particularly useful in the early stages of training as they can provide basic information. In 2012, EULAR also started an online MSUS course, which is subject to a standard fee. This course consists of seven modules and aims to teach basic skills related to MSUS [1]. Specialists interested in MSUS should first consider the validated online courses of professional organizations, such as EULAR’s online course. However, since this course is not accessible to everyone due to the fee involved, YouTube can be an alternative. While YouTube content cannot replace hands-on practice, it can at least play a complementary role to the traditional courses on MSUS training.

Limitations

This study includes a few limitations. First, this study demonstrates videos available on YouTube at a fixed moment in time. However, YouTube content shifts rapidly over time. Second, we searched YouTube for videos presented only in English language. Third, a direct YouTube search was performed and YouTube videos on other websites were ignored. Lastly, we used only one keyword “musculoskeletal ultrasound.” As it represents an umbrella term, we used a single keyword similar to previous studies (12,14).

Conclusion

Our study demonstrated that the YouTube videos on MSUS had inadequate content for the web-based learning of rheumatologists and other musculoskeletal specialists. However, considering that more than half of the videos were acceptable for education purposes, YouTube presents as a potential learning source for MSUS learning. In addition to being easily accessible and free, the interruption of traditional US courses during the pandemic suggests that YouTube may become an alternative tool for online education. Therefore, we strongly recommend healthcare professionals to create high-quality videos that clinicians can access to obtain more accurate and high-quality health information regarding MSUS. In addition, YouTube should consider collaborating with professional organizations, such as EULAR and ACR, for screening and removing low-quality videos.

Key Points • Although rheumatologists are increasingly using ultrasound as an extension of their exam, accessing musculoskeletal ultrasound training is still facing some challenges. • YouTube is commonly used as an educational source by medical students and healthcare professionals. • Approximately 60% of the videos related to musculoskeletal ultrasound on YouTube revealed high and moderate quality, while the remaining 40% were of low quality and not suitable for medical education. • A filtering mechanism is required to improve the quality of YouTube content on MSUS.