Quality of English-language videos available on YouTube as a source of information on osteoporosis.

Osteoporosis awareness is essential for preventing osteoporotic fractures. Social media platforms have enormous potential to both support and deter patients' decisions on healthcare. The current study demonstrated that the majority of YouTube videos on osteoporosis provide useful information with sufficient quality.
PURPOSE: YouTube is the most popular video-sharing platform for patients seeking online information about their medical condition. However, there are concerns regarding the accuracy and quality of YouTube content. We aimed to analyze the quality of English-language YouTube videos on osteoporosis.
METHODS: A YouTube search was performed on April 21, 2021, using the keywords "osteoporosis," "osteoporosis exercise," "osteoporosis diet," and "osteoporosis treatment." The reviewers recorded the content, source, and detailed characteristics of the included videos. The reliability and quality of the videos were analyzed using the modified DISCERN score and Global Quality Scale (GQS) score by a rheumatologist and endocrinologist.
RESULTS: Of the 400 videos screened, 238 were included in the study after applying the exclusion criteria. A total of 205 (86.1%) videos revealed useful information about osteoporosis whereas 33 (13.9%) were misleading. In terms of quality, 48% of the videos were of high quality, 34% were of moderate quality, and 18% were of low quality. The videos posted by universities and professional organizations had the highest modified DISCERN and GQS scores indicating high reliability and quality.
CONCLUSION: This study demonstrated that the majority of YouTube videos on osteoporosis contained useful information with sufficient quality. However, physicians should be aware of misleading information and correct any misinformation during face-to-face meetings with patients. YouTube should consider creating partnerships with professional organizations in the field of osteoporosis to produce high-quality videos in line with their new health content policy.

Introduction

Osteoporosis is the most common systemic skeletal disorder characterized by decreased density and microarchitectural deterioration of the bone, leading to increased bone fragility and fracture risk [1]. Although the disease can involve all bones due to its systemic nature, the hip, vertebra, and wrist are most prone to be affected [2]. With the aging of population worldwide, a growing number of fractures are being caused by osteoporosis [3]. Osteoporotic fractures are linked with increased patient morbidity, mortality, and decreased quality of life [4, 5]. In addition, osteoporotic fractures result in a major economic burden due to their healthcare costs exceeding those related to stroke, myocardial infarction, and breast cancer in women over 55 years [2]. Paradoxically, there is a significant treatment gap in patients with osteoporosis [6]. In addition to pharmacological treatment, a well-balanced diet and exercise should be recommended to patients at high fracture risk [1].

More than two-thirds of adults seek medical information on the internet [7]. YouTube, a global international popular online video platform, is frequently utilized as a source of healthcare information [8]. The well-designed use of audio and visual communication providing easy access for all individuals is one of the greatest advantages of YouTube, compared with other social media platforms [7]. Thus, YouTube may serve as a prized tool in providing health-related information. However, there is not yet a sufficient control mechanism on the accuracy, reliability, and quality of medical information uploaded on YouTube [9].

Several studies have recently assessed YouTube as a resource of medical information [6, 10, 11]. Tejada-Llacsa et al. [12] also performed a study evaluating Spanish-language YouTube videos about osteoporosis. However, to our knowledge, there is no study assessing the accuracy and quality of YouTube videos on osteoporosis presented in English language. The objective of this study was to assess the accuracy, reliability, and quality of English-language videos related to osteoporosis on YouTube and investigate whether they contained misleading medical information. The secondary objective was to present recommendations to encourage healthcare professionals and professional organizations to better utilize YouTube and ensure the distribution of accurate and quality information.

Materials and methods

Data were obtained from the publicly available YouTube (http://www.youtube.com) videos on April 21, 2021. The search terms were identified using Google Trends (https://trends.google.com/trends/?geo=TR). “Osteoporosis” was entered as a search term, and the filters were selected as “worldwide,” “last five years,” “all categories,” and “YouTube search.” From the top related queries on Google Trends, “osteoporosis exercise,” “osteoporosis diet,” and “osteoporosis treatment” were selected as search terms, in addition to the primary search term “osteoporosis.” Finally, the first 100 videos (400 videos in total) on YouTube were recorded for each of these search terms. The search was performed using the incognito mode on Google Chrome after clearing the search history to prevent the search results from being affected by user activity. Similar to most viewers, we did not apply any filter other than YouTube’s standard default “relevance” setting in our search. In this manner, we tried to mimic the search model that viewers use frequently. It is known that approximately 90% of people view only the first three pages of search results on the internet [13, 14]. Nonetheless, YouTube no longer uses pages to show results; it provides a continuous list. Therefore, we identified the first 100 videos for each search term, considering that it would be sufficient for a strong statistical analysis, based on previous studies [12, 15]. For further analysis, all the selected videos were saved into a playlist in our database since search results on YouTube can alter from one day to another.

Videos with primary content related to osteoporosis were included in the study. The exclusion criteria comprised irrelevant videos, duplicate videos, videos in languages other than English, those with no audio, and music videos. Similar exclusion criteria were used in previous studies [9, 13, 16]. After applying the exclusion criteria, the sample consisted of 238 videos (Fig. 1).

Fig. 1

Flowchart showing systematic video selection

All the videos were assessed independently by a rheumatologist (M.E.O.) and endocrinologist (C.E.O.) blinded to the purpose of the study. Any disagreement between the reviewers was resolved through the evaluation of a third reviewer (rheumatologist, O.Z.). Main video characteristics, including the upload date, days on YouTube, number of views, views per day, likes, dislikes, comments, and video duration, were extracted and documented by the authors.

Video contents

The videos were classified based on the following content: osteoporosis overview, pathophysiology, diagnosis, clinical signs and symptoms, secondary osteoporosis, osteoporosis exercises, nutrition and diet for osteoporosis, preventing osteoporosis, preventing osteoporotic fractures, and medical treatment for osteoporosis (Fig. 2). As some videos addressed more than one topic, only the topic that involved the longest duration in the entire video was selected.

Fig. 2

Distribution of analyzed videos by content

Evaluating usefulness

Following data extraction, all the videos were categorized into useful and misleading videos by the independent reviewers based on consensus. This categorization strategy was used in the assessment of medical information in previous studies [16-18]. The videos were deemed misleading if they contained one or more misleading statements in accordance with the guidelines of two leading osteoporosis organizations, namely the International Osteoporosis Foundation (IOF) and the National Osteoporosis Foundation (NOF).

Target audience

Based on the overall presentation, description, and source of the video, the most likely target audience was recorded. For example, if an endocrinologist narrated osteoporosis in a medical congress, these videos were classified as “targeting healthcare professionals,” or if a video described daily exercises for osteoporosis prevention without using medical terms, it was labeled as “targeting non-healthcare professionals.”

Video sources

The sources of the videos were categorized into four headings as “universities and professional organizations,” “healthcare professionals,” (physicians and non-physician healthcare professionals), “health-related websites,” and “independent users.”

Scoring protocol

The videos were further analyzed in terms of content, accuracy, reliability, and quality. Video reliability was evaluated with the modified DISCERN tool, which was originally developed by Charnock et al. [19] to assess written health information and later adapted by Singh et al. [14] to assess video reliability. The modified DISCERN instrument has been previously used in several YouTube studies [8, 13, 17]. This evaluation tool contains five “yes”/ “no” questions. Each “yes” response is given 1 point, and the highest score is 5. Higher scores indicate better reliability of assessed information.

The quality of the videos was evaluated with the five-point Global Quality Scale (GQS), in which each of the criteria is scored 1 point, and the total score ranges from 0 to 5. Higher scores demonstrate better quality of assessed information. This scale is also used for quality stratification, with scores of 1–2 points representing low quality, 3 points moderate quality, and 4–5 points high quality. This tool has been previously used for quality assessments in similar studies [16, 20, 21].

Lastly, the Video Power Index (VPI) was used to evaluate video popularity. It is difficult to measure video popularity due to the alteration of video metrics. For example, of the two videos with the same number of views uploaded 1 year ago and 1 month ago, the second can be misinterpreted as popular. VPI was developed to solve this problem. It denotes the views, likes, and dislikes over the number of days since upload. View ratio refers to the number of views per day. Like ratio is calculated as number of likes / [number of likes + number of dislikes] × 100. VPI was calculated according to the following formula: like ratio × view ratio / 100) [16, 22].

Statistical methods

Cohen’s kappa coefficient was used to assess the degree of agreement between the two reviewers. The Kolmogorov–Smirnov test was performed to assess the normality of data. The Mann–Whitney U test was used for the comparison of two groups and the Kruskal–Wallis test for the comparison of more than two groups in terms of non-normally distributed data. The Games-Howell post hoc test was carried out for pairwise comparisons following the Kruskal–Wallis test to determine which groups were significantly different. p < 0.05 was regarded as statistically significant. Descriptive data were presented as number, percentage, and median (interquartile range) values. All data analyses were performed using SPSS, version 22 (IBM SPSS Statistics for Windows; Armonk, NY: IBM Corp).

Ethics approval

As publicly accessible YouTube videos were used and no human/animal participants were involved in the study, ethical approval was not required.

Results

Baseline characteristics of the analyzed videos

Of the total 400 videos screened, 238 that met the inclusion criteria were included in the study. The median length of the videos was 5.80 (range, 2.54–16) minutes. The median number of views was 4719 (range, 997.75–26,243.50) and the median number of views per day was 5.24 (range, 1.14–35.18). The median number of likes was 52 (range, 9.75–274.50), the median number of dislikes was 2 (range, 0–11.25), and the median number of comments was 3.50 (range, 0–24.25). The median VPI score was 4.81 (range, 1.03–34.01). Considering reliability and quality, the median modified DISCERN score was 3 (range, 2–4), and the median GQS score was 3 (range, 3–4). According to quality stratification, 48% of the videos were of high quality, 34% were of moderate quality, and 18% were of low quality (Fig. 3). Cohen’s kappa coefficient between the authors was 0.823 ± 0.19 for the modified DISCERN and 0.860 ± 0.18 for GQS. Of the total 238 videos analyzed, 145 (60.9%) were presented by healthcare professionals and 93 (39.1%) by non-healthcare professionals. No significant difference was detected between the healthcare professionals and non-healthcare professionals regarding the number of views, views per day, likes, dislikes, comments, and VPI (p < 0.05). In terms of reliability and quality, the videos presented by healthcare professionals had higher GQS and modified DISCERN scores compared to the non-healthcare professionals, as expected (p < 0.001).

Fig. 3

Quality stratification of the analyzed videos

Video content

The majority of the videos were related to osteoporosis exercises (n = 76, 31.9%), followed by medical treatment (n = 53, 22.3%) and nutrition and diet for osteoporosis (n = 51, 21.4%). The remaining topics were osteoporosis overview (n = 29, 12.2%), preventing osteoporosis (n = 9, 3.8%), secondary osteoporosis (n = 6, 2.5%), pathophysiology (n = 4, 1.7%), diagnosis (n = 4, 1.7%), symptoms and clinical findings (n = 3, 1.3%), and preventing osteoporotic fractures (n = 3, 1.3%).

Usefulness

Of the total 238 videos analyzed, 205 (86.1%) contained useful information and 33 (13.9%) contained misleading information. Cohen’s kappa coefficient between the reviewers regarding usefulness was 0.890 ± 0.06. There was no significant difference between the useful and misleading videos in terms of the median number of days on YouTube, views, views per day, likes, dislikes, comments, and video length. The median VPI score, as a measurement of popularity, did not significantly differ between the two groups. Considering reliability and quality, the useful videos achieved higher median scores compared to the misleading videos for both modified DISCERN (3, range 3–4 vs 2, range 1–2) and GQS (4, range 3–4 vs 2, range 2–2) at a significant level (p < 0.001 for both). While 33 (16.3%) videos that targeted patients contained misleading information, all videos targeting healthcare professionals contained useful information. The detailed characteristics of the useful and misleading videos are presented in Table 1.

Table 1

Characteristics of YouTube videos on osteoporosis according to their usefulness

	Useful information	Misleading information	p
	205 (86.1%)	33 (13.9%)
Video metrics
Days on YouTube	918 (363–2036.50)	1137 (443.50–1548.50)	0.991
Number of views	4644 (990.50–25,749)	4933 (856–42,645.50)	0.554
Views per day	5.40 (1.13–35.50)	4.38 (1.57–38.78)	0.561
Number of likes	50 (9–285.50)	76 (11.50–261.50)	0.456
Number of dislikes	2 (0–11.50)	4 (1–12)	0.357
Number of comments	3 (0–23)	5 (0.50–34.50)	0.465
Duration (minutes)	5.59 (2.49–15.46)	6.01 (4.50–21.11)	0.361
Like ratio	96.88 (92.47–100)	96.32 (92.42–98.49)	0.718
Popularity
VPI score	4.84 (1.02–34.54)	4.08 (1.33–37.22)	0.688
Reliability and quality
Modified DISCERN score	3 (3–4)	2 (1–2)	< 0.001*
GQS score	4 (3–4)	2 (2–2)	< 0.001*
Target audience (n (%))
Patients	170 (83.7%)	33 (16.3%)	N/A
Healthcare professionals	35 (100%)	0 (0%)	N/A
Source (n (%))
Universities/professional organizations	45 (100%)	0 (0%)	N/A
Healthcare professionals	43 (87.8%)	6 (12.2%)	N/A
Health-related websites	95 (83.3%)	19 (16.7%)	N/A
Independent users	22 (73.3%)	8 (26.7%)	N/A

Data presented as n (%) or median (interquartile range)

Pairwise comparisons between the “useful” and “misleading” videos were performed using the Mann–Whitney U test

*Values of p < 0.05 were considered statistically significant and marked in bold

VPI, Video Power Index; GQS, Global Quality Scale; N/A, not applicable

Of the analyzed 238 videos, 114 (47.9%) had been uploaded by health-related websites, 49 (20.6%) by healthcare professionals, 45 (18.9%) by universities and professional organizations, and 30 (12.6%) by independent users. Independent users were more likely to upload misleading videos (26.7%). No misleading video had been uploaded by universities and professional organizations. The rates of misleading videos posted by health-related websites and physicians were 16.7% and 12.2%, respectively.

There was no significant difference in the mean number of days on YouTube, views, views per day, likes, dislikes, comments, video duration, and VPI score according to the video source. The videos uploaded by universities and professional organizations had the highest median duration (6 min 45 s) with no significant difference (p = 0.329). In terms of popularity, the median VPI scores revealed no significant difference between the sources (p = 0.100). There was a significant difference in the modified DISCERN and GQS scores between the sources (p < 0.001 for both). The comparison of the videos according to their sources is given in Table 2.

Table 2

Comparison of the video metrics, popularity, reliability, and quality scores of the videos according to sources

	Universities and professional organizations	Healthcare professionals	Health-related websites	Independent users	p
	45 (18.9%)	49 (20.6%)	114 (47.9%)	30 (12.6%)
Video metrics
Days on YouTube	1005 (401.50–2326.50)	875 (226–1952)	1147 (443.25–2067.25)	411 (260.25–1525)	0.172
Number of views	3080 (825.50–7333)	5105 (581.50–33,800)	6147.50 (1146.75–35,328)	5018 (719.50–17,058)	0.260
Views per day	3.01 (0.74–11.51)	6.17 (1.20–57.81)	5.65 (1.12–48.53)	9.28 (1.84–36.36)	0.146
Number of likes	27 (6–59)	46 (6.5–251)	76 (12.50–504)	71 (15.50–345.25)	0.069
Number of dislikes	1 (0–3.5)	5 (1–16)	3 (0.75–12.25)	1 (0–6.25)	0.055
Number of comments	2 (0–9)	4 (0–18)	4 (0–41)	7.50 (0–31.25)	0.229
Duration (minutes)	6.45 (2.25–53.89)	5.20 (2.35–13.32)	6.17 (3.19–13.70)	5.92 (4.16–13.08)	0.329
Like ratio	95.56 (89.60–100)	95.46 (89.11–98.47)	97.32 (93.50–99.38)	97.99 (95.40–100)	0.086
Popularity
VPI score	2.80 (0.60–10.03)	5.95 (0.86–56.37)	5.36 (1.09–47.01)	9.12 (1.75–35.46)	0.094
Reliability
Modified DISCERN score	4 (4–5)	3 (3–4)	3 (2–3)	2 (2–3)	< 0.001*
Quality
GQS score	5 (4–5)	3 (3–4)	3 (3–4)	3 (2–3)	< 0.001*

Data presented as n (%) or median (interquartile range)

Comparison of continuous variables for useful videos without normal distribution was performed using the Kruskal–Wallis test

*Statistically significant at p < 0.05

VPI, video power index; GQS, Global Quality Scale

Pairwise comparisons revealed that the videos posted by universities and professional organizations had higher median scores in both the modified DISCERN (4, range 4–5) and GQS (5, range 4–5) than those uploaded by healthcare professionals, health-related websites, and independent users with a statistically significant difference (p < 0.001 for all). In contrast, the videos provided by independent users had the lowest median scores compared to universities and professional organizations, healthcare professionals, and health-related websites with a significant difference in the modified DISCERN (p < 0.001, p = 0.001, and p = 0.033, respectively) and GQS (p < 0.001, p = 0.11, and p = 0.001, respectively) scores. The videos of healthcare professionals and health-related websites did not significantly differ in terms of the modified DISCERN and GQS scores (p = 0.141 and p = 0.998, respectively).

Discussion

The current study was the first to evaluate the content of English-language osteoporosis videos on YouTube. In this study, we analyzed 238 videos uploaded between December 3, 2008, and May 10, 2021, with 12,866,799 views and a cumulative duration of 54.3 h, revealing that YouTube is a popular information source related to osteoporosis. The majority of the videos (85.2%) targeted patients. The videos provided sufficient content on almost every topic related to osteoporosis, including osteoporosis overview, osteoporosis exercises, medical treatment, nutrition, and preventing fractures.

Osteoporosis is a major public health issue. Although it is considered a silent disease, osteoporotic fractures can lead to devastating results, including morbidity, mortality, loss of quality of life, and economic burden [23]. The main goal in the management of osteoporosis is the prevention of fractures [24]. As various studies have presented a positive association between the individuals’ level of knowledge on osteoporosis and implementation of preventive measures [5], osteoporosis awareness is crucial for preventing the disease and fragility fractures. This study demonstrated that YouTube provides plentiful useful information with sufficient quality on osteoporosis.

In our study, nearly two-thirds of the videos were presented by healthcare professionals, and these videos demonstrated higher quality and reliability than those presented by non-healthcare professionals. However, we found no significant difference between the two groups regarding video metrics (number of views, views per day, likes, dislikes, and comments) in contrast to a previous study performed by Tejada-Llacsa et al. [12] on Spanish-language YouTube videos about osteoporosis. Similar to our study, Elangovan et al. [10], who evaluated spondyloarthritis on YouTube, reported no significant difference between the videos presented by healthcare professionals and non-healthcare professionals in relation to video metrics (number of views, views per day, likes, and dislikes), but noted a difference in the number of comments. In light of these findings, we suggest that healthcare professionals should consider making more attractive videos.

As osteoporosis is related to many disciplines, video presenters were distributed in a wide range including physicians, such as endocrinologists, rheumatologists, orthopedists, physiatrists, and geriatrists, non-physician healthcare professionals, such as physiotherapists, dietitians, and nurses, and non-healthcare professionals, such as plates teachers, yoga practitioners, fitness coaches, and wellness consultants. Among all these groups, the highest percentage of video presenters was endocrinologists, as expected. In about one-third of the videos, the identity of the presenters was unknown. This proportion is lower than reported by Murugiah et al. (48%) for videos on cardiopulmonary resuscitation [25]. However, in our study, the quality and reliability of the videos with unidentified presenters were significantly lower than those presented by experts. Therefore, users should avoid watching videos posted by unidentified presenters when seeking medical information about osteoporosis on YouTube.

Of the reviewed 238 videos, 86.1% were considered as useful, similar to previous studies performed by Elangovan et al. [10] on spondyloarthritis (86%) and Ng et al. [11] on systemic lupus erythematosus (83.6%). However, the rate of useful videos in the literature was generally lower than our results. For example, the rate of useful videos was found to be 73.7% for Ebola virus disease [26], 69.9% for COVID-19 [27], 58.3% for dialysis [28], 54.9% for Sjogren’s syndrome [14], and only 35% for colostomy and ileostomy [29]. In contrast to our study, Tejada-Llacsa et al. [12] reported a low number of appropriate messages in Spanish-language osteoporosis videos on YouTube. However, the authors applied a checklist that contained five domains (definition, diagnosis, recommendations, treatment, and follow-up) to assess the presence of appropriate messages, which may have resulted in low scores, as very few of the videos contained all these domains. According to the new 2021 policies, YouTube decided to remove content including information on COVID-19 and vaccination that contradicts consensus from healthcare authorities [30]. Similarly, YouTube should consider collaborating with reputable organizations to remove videos containing misinformation on other major public health issues, such as osteoporosis.

In the current study, the useful videos had the highest reliability and quality, as expected. However, it was concerning to find that the useful and misleading videos did not significantly differ in relation to viewer engagement parameters, such as likes, dislikes, and comments, suggesting that viewers cannot distinguish between videos of different quality. This discrepancy between the quality of videos and viewer interaction parameters has also been shown in previous studies [10, 27]. In addition, we found no difference between the useful and misleading videos in terms of video popularity, similar to previous studies performed by Delli et al. [15] and Moon et al. [16]. Physicians should be aware of this issue and possible exposure of patients to misinformation about osteoporosis and direct them to accurate online resources in face-to-face meetings.

Considering video sources, we found that universities and professional organizations showed the highest reliability and quality, which is in agreement with previous studies [11, 16]. In addition, no misleading video was uploaded by universities and professional organizations. These videos demonstrate the potential of YouTube to be used as a supplementary platform for healthcare education. The independent user-produced videos constituted the group with the highest percentage of misleading videos, similar to previous studies [16]. As 75% of patients do not consistently verify the source of online information, they are more likely to be exposed to misinformation [11]. YouTube should consider presenting videos from validated sources on the first page of the search results and filtering those uploaded by independent users in collaboration with experts. In addition, universities and professional organizations should increase the number of videos they produce.

YouTube is a popular video-shared platform increasingly used for healthcare information since 2005. However, health-related content was not adequately moderated until the end of 2020. YouTube decided to implement new health policies in early 2021 and established new partnerships with professional organizations, such as the Cleveland Clinic and the Mayo Clinic, to make health information more accessible and understandable. The Director and Global Head of YouTube’s Healthcare and Public Health Partnerships, Dr. Garth Graham, announced that they were willing to create more partnerships with reputable health organizations [31]. In this regard, professional organizations, such as IOF and NOF, should consider collaborating with YouTube to promote credible health information on osteoporosis, which is a serious public health issue affecting millions of people.

Limitations

The current study has some limitations. Firstly, it was a cross-sectional study that captured YouTube videos at a certain moment in time. Due to the nature of YouTube, new videos are uploaded constantly, and video interaction parameters rapidly change. Secondly, we only analyzed the English-language videos available on YouTube, but English is the prevailing language of international discourse in today’s world [32]. Thirdly, as the lack of standardized tools, assessment of quality videos is challenging, and GQS was first developed for the evaluation of websites, not videos, as described by Bernard et al. [33]. Finally, we only searched for videos on YouTube’s official platform, and we did not consider those presented on other websites.

Conclusion

This study demonstrated that the majority of YouTube videos on osteoporosis present useful information for patients. In addition, nearly half of the videos were of high quality and one-third were of moderate quality, which is acceptable, while only a small proportion of the videos had poor quality. Osteoporosis videos in English language available on YouTube appear to be sufficient to raise awareness about the different aspects of the disease and can provide supplementary information to patient-physician meetings. However, patients should be warned about the possibility of misinformation on osteoporosis, misrepresented video metrics, and non-professional video sources by their physicians. YouTube should consider creating partnerships with the leading professional organizations, such as NOF and IOF, to produce more high-quality content related to osteoporosis.