YouTube Is a Poor-Quality Source for Patient Information on Rehabilitation and Return to Sports After Hip Arthroscopy.

PURPOSE: To investigate the information quality on YouTube regarding rehabilitation and return to sport (RTS) after hip arthroscopy.
METHODS: By use of private browsing and predefined search terms, 217 unique videos regarding RTS and rehabilitation after hip arthroscopy were included and systematically reviewed. A total of 164 videos were included in the final analysis. Videos were scored using 4 scoring systems: (1) Journal of the American Medical Association benchmark criteria, (2) the Global Quality Score, (3) a score for RTS after hip arthroscopy, and (4) a score for rehabilitation after hip arthroscopy.
RESULTS: A large majority of the included videos provided substandard information quality, dependability, and precision. RTS videos that were uploaded by physicians had a significantly higher Journal of the American Medical Association score, Global Quality Score, and RTS score compared with commercial and personal testimony videos (P = .0003, P = .0021, and P = .0005, respectively). Physician videos pertaining to RTS were also significantly longer than videos in other categories (P = .0397).
CONCLUSIONS: The quality and reliability of video content on YouTube pertaining to rehabilitation and RTS after hip arthroscopy are generally poor. The educational content of YouTube videos produced by physicians is of significantly higher quality as compared with non-physicians, patient testimonials, and commercials. CLINICAL RELEVANCE: The quality of the information patients receive on rehabilitation and RTS after hip arthroscopy is important for successful outcomes.

Rehabilitation after hip arthroscopy is critical, and clinical outcomes are dependent on the implementation of high-quality rehabilitation protocols. There have been considerable advances in rehabilitation protocols that have allowed patients to return to sport (RTS) quickly and safely., Despite a technically well-performed surgical procedure, patients who are inadequately rehabilitated are not immune to recurrent and/or further injury. A recent study evaluating 350 patients undergoing rehabilitation after hip arthroscopy found that those who did not achieve appropriate rehabilitation thresholds by 3 months postoperatively were nearly 3 times more likely to require reoperation.

Patients have been turning to the Internet as a source of medical and surgical information., More than 50% of patients in North America have reported using the Internet to search for health information, and sports medicine patients are more likely to use social media as a resource for health information than patients treated by other orthopaedic subspecialties.6, 7, 8, 9 One of the most accessed and widely available resources of information on the Internet is YouTube (Alphabet, Mountain View, CA), which is available in over 100 countries and in 80 different languages and boasts greater than 1 billion guests per month.,, Young adults are the primary demographic consuming content on YouTube, with approximately 90% of them worldwide using the website regularly. Despite the ubiquity of health information on YouTube, there is considerable skepticism about the quality of the information, and recent studies have shown considerably low information quality.,,,

The purpose of this study was to investigate the information quality on YouTube regarding rehabilitation and RTS after hip arthroscopy. The hypothesis was that the informational quality, reliability, and accuracy of educational, commercial, or personal testimony videos on YouTube would be poor and that resources distributed by physicians would be of higher quality than information produced by non-physicians.

Methods

On August 3, 2020, a search of YouTube videos pertaining to rehabilitation and RTS after hip arthroscopy was conducted. For RTS, the following search terms were used: Hip arthroscopy return to sport, Hip arthroscopy return to running, Hip arthroscopy return to work. For rehabilitation, the following search terms were used: Hip scope rehab protocol, Hip scope physical therapy protocol, Hip scope physical therapy, Hip arthroscopy rehab, Hip arthroscopy physical therapy, Hip arthroscopy PT, Hip arthroscopy recovery tips. To account for the variability in video content presentation, a second search was performed on August 18, 2020, using the same search terms. The inclusion criterion was any video that discussed rehabilitation or RTS after hip arthroscopy. Videos that were not in English, did not have sound support, or did not explicitly discuss rehabilitation or RTS even though it was included in the title; videos that did not involve human patients; and duplicate videos were excluded from the study.

Similarly to previous studies in the literature, the search terms were initially derived using the autofill tool in the YouTube video search bar.,14, 15, 16 The autofill function is based on a complex algorithm and ultimately stratifies terms based on overall popularity. To account for previous browsing behavior and avoid biased search results, the anonymous browsing function of Google Chrome (Alphabet) was used for this study. Google Chrome is free software that enables anonymous Web browsing through the incognito function; by concealing the user’s identity and browsing history, standard YouTube algorithms cannot introduce bias, and thus, the results were not influenced.

The review of videos was performed by 1 orthopaedic surgery resident (T.R.J.) and 2 medical students (L.A. and K.J.W.), and any discrepancies between the observers were clarified by mutual agreement. As the default settings, YouTube organizes search results based on descending order of relevance to search term used. Users seldom access videos beyond the third page, and in accordance with this, the first 60 videos under each search term were included in the study.,

The following information was recorded for each video: Universal Resource Locator (URL), video title, number of total views, video category, duration of video in minutes, date of publication, days since upload, number of likes, number of dislikes, like ratio, video power index (VPI), and view ratio (views per day). The like ratio was determined by the number of likes divided by the number of dislikes. The VPI is an index used to quantify the popularity of a video based on the views and likes and has been validated in previous literature.,18, 19, 20 The VPI is calculated as (Like ratio × View ratio)/100. Because YouTube lacks a modality for indexing videos based on popularity, the VPI has been used as a surrogate measure. An increased VPI signifies increased video popularity when compared with videos with a lower VPI. All included videos were categorized into 1 of 4 groups: educational–physician, educational–non-physician (health professionals other than licensed medical doctors), personal testimony, and commercial.

Video Scoring

Four scoring methods were implemented to assess the quality of included videos: Journal of the American Medical Association (JAMA) benchmark criteria, the Global Quality Score (GQS), a rehabilitation score, and an RTS score. The JAMA benchmark criteria are an objective assessment aid that consists of 4 independent guidelines. One point is given for the presence of each guideline. A maximum of 4 points can be achieved, and a higher score designates better accuracy and reliability for the source. The caliber of educational content was evaluated by the use of the GQS. The GQS contains 5 criteria that gauge the educational value of online resources. One point was assigned for the presence of each criterion. The higher the score, the higher the educational quality. The highest attainable score is 5 points, which indicates excellent quality and flow of information. To assess the educational content regarding rehabilitation and RTS, 2 nonvalidated grading systems were used: an RTS–after–hip arthroscopy score and a rehabilitation–after–hip arthroscopy score. The use of nonvalidated assessment tools to assess the educational quality of orthopaedic videos online has been accepted in the literature.,19, 20, 21 Because of the lack of a gold standard in rehabilitation and RTS, these scoring systems were developed based on recent literature evaluating the current state of rehabilitation and RTS after hip arthroscopy, as well as expert opinion., The RTS score had a scale from 0 to 15 points. Likewise, the rehabilitation score had a scale from 0 to 15 points. For both scoring systems, a score of 0 to 4 points is indicative of videos of poor quality; 5 to 10, moderate quality; and 11 to 15, excellent quality (Appendix Tables 1-4, available at www.arthroscopyjournal.org).

Statistical Analysis

Continuous data were reported as means and standard deviations. Categorical data were reported as absolute and relative frequencies. Descriptive analysis was used to describe video characteristics. Variables with a non-normal distribution were evaluated using the Mann-Whitney U test. JAMA benchmark, GQS, RTS, and rehabilitation criteria were all evaluated using 2-tailed independent t tests. Results with P < .05 were considered statistically significant. Multivariate analysis was performed to assess the relation between video characteristics and categories. All calculations were performed with SPSS software (version 21; IBM, Armonk, NY).

Results

Initially, 217 YouTube videos (135 rehabilitation and 82 RTS videos) were identified. Of these videos, 53 were excluded because they did not meet the criteria necessary to be included. A total of 164 videos (82 rehabilitation and 82 RTS videos) remained in the final analysis (Fig 1). Of the 164 videos present in the analysis, 95 (57.9%) reported on rehabilitation after femoroacetabular impingement, 22 (13.4%) reported on rehabilitation for other pathologies, and 47 (28.7%) did not disclose an underlying pathology. Videos that reported on rehabilitation for other pathologies included but were not limited to gluteus medius repair, instability, and labral tears. The sum of all video views among the 164 videos was 6,038,762. Figure 2 and Figure 3 show the distribution of JAMA scores and GQS values, respectively, for all videos. Figure 4 shows the distribution of rehabilitation and RTS scores for all videos.

Fig 1

Flowchart of search criteria used for inclusion of YouTube videos regarding rehabilitation and return to sport (RTS) after hip arthroscopy.

Fig 2

Distribution of video scoring using Journal of the American Medical Association (JAMA) benchmark criteria. (Rehab, rehabilitation; RTS, return to sport.)

Fig 3

Distribution of video scoring using Global Quality Score (GQS). (Rehab, rehabilitation; RTS, return to sport.)

Fig 4

Distribution of video scoring using rehabilitation score and return-to-sport (RTS) score.

Rehabilitation After Hip Arthroscopy

The mean duration was 5.9 ± 5.0 minutes, the mean number of views was 34,115.00 ± 81,321.90, the mean time since upload of the video (i.e., video age) was 1,525.60 ± 918.20 days, and the mean VPI was 4.7 ± 9.7. Of the videos, 26 were classified as educational–physician; 34, educational–non-physician; 4, commercial; and 18, personal testimony. The mean JAMA benchmark criteria score was 2.2 ± 0.9 points, the mean GQS was 2.0 ± 1.1 points, and the mean rehabilitation score was 2.3 ± 2.0 points (Table 1). Only 11.0% of videos regarding rehabilitation (9 of 82) had a GQS of 4 points or greater.

Table 1

Characteristics of All Included Videos

Variable	Rehabilitation	RTS	P Value
Views	34,115.5 ± 81,321.9	39,527.9 ± 151,365.5	.776
Duration, min	5.9 ± 5.0	6.4 ± 5.8	.614
Video age, d	1,525.6 ± 918.2	1,524.9 ± 1,029.9	.996
Likes	229.1 ± 557.3	345.7 ± 1,592.0	.533
Dislikes	12.8 ± 35.2	17.7 ± 79.0	.606
Like ratio	18.7 ± 22.1	9.8 ± 13.0	.002∗
View ratio	20.4 ± 40.7	27.4 ± 113.5	.602
VPI	4.7 ± 9.7	5.5 ± 24.7	.772
JAMA benchmark criteria score	2.2 ± 0.9	2.5 ± 1.3	.054
GQS	2.0 ± 1.1	2.4 ± 1.2	.046∗
RTS score		2.7 ± 2.6
Rehabilitation score	2.3 ± 2.0

NOTE. Continuous variables are presented as mean ± standard deviation.

GQS, Global Quality Score; JAMA, Journal of the American Medical Association; RTS, return to sport; VPI, video power index.

Statistically significant (P < .05).

Commercial videos had a significantly higher like-dislike ratio compared with all other categories (P = .004). Videos classified as educational–physician had a significantly older video age (P = .001). There were no statistically significant differences in the number of views, video duration, likes, dislikes, VPI, JAMA score, GQS, and rehabilitation score (Table 2). A multivariate analysis to control for confounders confirmed that educational–physician videos had a significantly longer video age (P = .0289).

Table 2

Characteristics of Rehabilitation Videos by Category

Variable	Educational–Physician	Educational–Non-physician	Personal Testimony	Commercial	P Value
Duration, min	5.2 ± 5.64	6.4 ± 4.59	6.9 ± 5.12	2.3 ± 1.71	.8759
Video age, d	1,982 ± 827.3	1,495 ± 887.1	1,006 ± 828.3	1,158 ± 950.5	.0005∗
Likes	118 ± 271.0	424 ± 798.1	61.5 ± 49.37	43.8 ± 46.62	.7094
Dislikes	7.3 ± 15.15	22.2 ± 51.52	5.5 ± 9.39	0.8 ± 0.96	.7571
Like ratio	10.4 ± 8.42	19.3 ± 23.16	25.6 ± 23.98	36.8 ± 46.27	.004∗
View ratio	12.0 ± 22.64	34.3 ± 57.35	9.5 ± 8.06	6.3 ± 4.96	.7771
VPI	2.9 ± 6.83	7.5 ± 13.16	2.2 ± 2.96	3.9 ± 5.84	.9501
JAMA benchmark criteria score	2.5 ± 0.86	2.1 ± 1.03	2.0 ± 0.49	1.5 ± 1.29	.0105
GQS	2.0 ± 1.11	2.5 ± 1.11	1.4 ± 0.61	1.8 ± 0.96	.1535
Rehabilitation score	2.1 ± 2.36	3.1 ± 1.91	1.4 ± 1.14	1.3 ± 1.50	.1821

NOTE. Continuous variables are presented as mean ± standard deviation.

GQS, Global Quality Score; JAMA, Journal of the American Medical Association; VPI, video power index.

Statistically significant (P < .05).

RTS After Hip Arthroscopy

The mean duration was 6.4 ± 5.8 minutes, the mean number of views was 39,527.90 ± 151,365.50, the mean time since upload of the video was 1,524.90 ± 1,029.9 days, and the mean VPI was 5.5 ± 24.7. The mean JAMA benchmark criteria score was 2.5 ± 1.3 points, the mean GQS was 2.4 ± 1.2 points, and the mean RTS score was 2.7 ± 2.6 points. Only 17.1% of videos regarding RTS (14 of 82) had a GQS of 4 points or greater. Of the videos, 50 were classified as educational–physician; 14, educational–non-physician; 10, commercial; and 8, personal testimony. Educational–physician videos had a significantly higher JAMA score, GQS, and RTS score compared with all other groups (P = .0003, P = .0021, and P = .0005, respectively) (Table 3). Educational–physician videos were also significantly longer than videos in other categories (P = .0397). On multivariate analysis, video categories were not found to have a significant impact on video characteristics (P > .05).

Table 3

Characteristics of RTS Videos by Category

Variable	Educational–Physician	Educational–Non-physician	Personal Testimony	Commercial	P Value
Duration, min	7.2 ± 5.81	6.7 ± 6.40	6.0 ± 4.69	2.0 ± 1.79	.0397∗
Video age, d	1,551 ± 1,055	1,426 ± 968.2	1,625 ± 1,489	1,263 ± 1,223	.8435
Likes	259 ± 936.0	805 ± 2,844	9.6 ± 15.47	22.5 ± 23.53	.6424
Dislikes	12.7 ± 46.31	42.5 ± 140.7	1.0 ± 1.73	0.2 ± 0.41	.647
Like ratio	9.7 ± 13.84	8.0 ± 7.80	4.1 ± 3.62	22.5 ± 23.53	.4121
View ratio	21.3 ± 73.23	59.2 ± 198.1	4.4 ± 5.32	3.1 ± 3.06	.6604
VPI	4.9 ± 19.68	11.0 ± 40.08	0.3 ± 0.49	1.2 ± 1.37	.5879
JAMA benchmark criteria score	3.0 ± 1.26	2.5 ± 1.12	0.8 ± 0.45	1.8 ± 0.98	.0003∗
GQS	2.7 ± 1.24	2.2 ± 1.04	1.6 ± 0.89	2.0 ± 0.89	.0021∗
RTS score	3.8 ± 2.85	2.0 ± 1.80	1.0 ± 1.41	1.3 ± 1.03	.0005∗

NOTE. Continuous variables are presented as mean ± standard deviation.

GQS, Global Quality Score; JAMA, Journal of the American Medical Association; RTS, return to sport; VPI, video power index.

Statistically significant (P < .05).

Discussion

This investigation showed that, in general, YouTube videos provided poor-quality educational information regarding rehabilitation and RTS after hip arthroscopy. Recent studies have shown that nearly two-thirds of patients use the Internet prior to their medical appointment to acquire knowledge about their condition, and YouTube is a frequently used platform for patient information. The content on YouTube, however, is not subject to a rigorous peer-review process. Patients should exercise caution when using this resource as a source of information for postoperative treatment after hip arthroscopy, and physicians and physical therapists alike should be mindful of the poor quality of information regarding rehabilitation and RTS.

When assessing the quality of educational health content on YouTube, previous studies have shown the poor quality and reliability of information available. Springer et al. examined the quality of the educational content on YouTube regarding RTS and rehabilitation after anterior cruciate ligament reconstruction. Their evaluation of 140 videos found consistently poor quality with reported scores for rehabilitation and RTS, with GQS values of 1.95 ± 0.64 points and 1.62 ± 0.82 points, respectively; JAMA scores of 1.32 ± 0.64 points and 1.26 ± 0.7 points, respectively; a rehabilitation score of 5.0 ± 3.42 points; and an RTS score 3.05 ± 3.35 points. In an assessment of the educational content on YouTube regarding facial plastic surgery procedures, Ward et al. evaluated 240 videos using the DISCERN score. The DISCERN score is a validated information quality assessment instrument. The average DISCERN score was reported as 2.21, indicating that videos had important and potentially serious shortcomings in quality. Delli et al. assessed the quality of 70 educational YouTube videos discussing Sjögren syndrome and found a mean GQS of 2.5 ± 1.2 points. This indicates poor educational quality and further confirms the consistent finding of poor educational information quality among YouTube videos. In accordance with previous investigations, our study found YouTube videos reporting rehabilitation and RTS after hip arthroscopy to be of poor quality. This is likely due to the fact that only 46.9% of all videos included in the final analysis were produced and maintained by physicians and these are not likely to be subject to vigorous review. Patients looking for rehabilitation and RTS information after hip arthroscopy on YouTube should be aware of the poor quality of information and the fact that the information provided on YouTube is not peer reviewed for accuracy. It is advisable that clinicians provide patients with educational resources that have been reviewed for accuracy regarding hip arthroscopy.

On the evaluation of educational health content on YouTube, several prior investigations have shown that content produced by physicians showed higher quality scores across multiple different specialties. Ferhatoglu et al. examined the quality of information in 100 videos on YouTube regarding sleeve gastrectomy. They found that videos produced by university-affiliated physicians had significantly higher JAMA scores and GQS values compared with all other cohorts examined. In an assessment of the quality of information pertaining to rotator cuff repairs on YouTube, Celik et al. reviewed 67 videos and showed that content uploaded by physicians had significantly higher DISCERN and JAMA scores (P < .001) when compared with all other author types. Our investigation corroborates these findings. Educational videos made by physicians comprised 61% of all the RTS videos and had a significantly higher JAMA score, GQS, and RTS score compared with all other video types. This phenomenon was not found among rehabilitation videos. This phenomenon is likely reflected by the degree of variability in rehabilitation composition and duration ranges found in the literature at which a patient can progress through his or her rehabilitation goals for each hip arthroscopic procedure. These findings suggest that patients should be aware that content posted by physicians is more likely to be accurate and of value when compared with posts by other entities.

Previous studies have sought out to evaluate YouTube videos based on their VPI. In a study evaluating the quality of 52 YouTube videos on developmental dysplasia of the hip (DDH), it was found that the VPI did not vary significantly based on the source of the videos, even though videos produced in an academic setting had a significantly higher GQS and DDH score when compared with commercially produced videos (P = .01 and P = .037, respectively). The DDH score is a nonvalidated scoring system that was developed by Oztermeli and Karahan to assess the educational quality of videos regarding DDH. In a study on the reliability of 100 YouTube videos on sleeve gastrectomy, Ferhatoglu et al. found that videos produced by surgeons had a significantly lower VPI, even though this category had a higher JAMA score, GQS, DISCERN score, and sleeve gastrectomy scoring system score when compared with patient testimonials and advertisement-based videos. These findings are supported by the results of our study, which found that RTS videos produced by physicians had a significantly higher JAMA score, GQS, and RTS score compared with commercial and personal testimony videos. There was no statistically significant difference in the VPI between groups, and physician videos had an equivalent VPI to videos of lower quality (e.g., commercial videos). These findings suggest that patients and physicians alike must be rigorous in evaluating educational material on YouTube and that the VPI may not be an appropriate surrogate measurement for high-quality information.

Limitations

This study is not without limitations. Although the JAMA score and GQS are validated assessment tools, the scoring systems used in this study for the assessment of rehabilitation and RTS are not currently validated; however, these metrics show a similar low degree of precision to validated metrics. Whereas the interobserver reliability of the JAMA score and GQS has been shown to be good to excellent in the literature, the correlation coefficient between observers using the rehabilitation and RTS scoring systems is yet to be formally evaluated. Furthermore, assessment of video content on YouTube was performed at 2 different, albeit close, time points, which is subject to bias owing to the dynamic nature of the platform because new content is constantly being produced and uploaded on the platform. It must also be noted that the information quality of video content was assessed against a scoring criteria checklist. This process involved allocation of points if specific information was presented; it did not take into consideration the presence of false or inaccurate information. Finally, it must be acknowledged that currently, there is no gold standard in rehabilitation after hip arthroscopy. The lack of a standardized protocol can result in significant variation in the recommendations and practices of orthopaedic sport surgeons and, as such, can influence the scoring of videos on these topics.

Conclusions

The quality and reliability of video content on YouTube pertaining to rehabilitation and RTS after hip arthroscopy are generally poor. The educational content of YouTube videos produced by physicians is of significantly higher quality as compared with non-physicians, patient testimonials, and commercials.

Appendix Table 1. JAMA Criteria

Criterion	Description
Authorship	Authors and contributors, their affiliations, and relevant credentials should be provided.
Attribution	References and sources for all content should be listed clearly, and all relevant copyright information should be noted.
Disclosure	Website "ownership" should be prominently and fully disclosed, as should any sponsorship, advertising, underwriting, commercial funding arrangements or support, or potential conflicts of interest. This includes arrangements in which links to other sites are posted as a result of financial considerations. Similar standards should hold in discussion forums.
Currency	Dates on which content was posted and updated should be indicated.

NOTE. One point is assigned for each criterion, with a maximum score of 4 points.

JAMA, Journal of the American Medical Association.

Appendix Table 2. Global Quality Score

Description	Score, Points
Poor quality and poor flow of site; most information missing; not at all useful for patients	1
Generally poor quality and poor flow; some information listed but many important topics missing; of very limited use to patients	2
Moderate quality and suboptimal flow; some important information adequately discussed but other important information poorly discussed; somewhat useful for patients	3
Good quality and generally good flow; most of relevant information listed but some topics not covered; useful for patients	4
Excellent quality and flow; very useful for patients	5

NOTE. The maximum score is 5 points.

Appendix Table 3. RTS Score

Description	Score, Points
General information	3
Aim: patient should return to preinjury level of activity	1
Takes ≥4 mo to RTS	1
Each individual should be evaluated individually	1
Impact of age, sex, and BMI	3
Age, sex, and BMI affect RTS	1 each
Procedure type	2
Healing time depends on presence of FAI and/or intracapsular pathology	2
RTS depends on which sport	2
Time since surgery	1
No earlier than 4 mo	1
Muscle strength	1
Maximal recovery can take 1 yr	1
Delay RTS	2
Secondary injury or poor control will delay progression	1 each
Function tests and balancing, proprioceptive tests, and/or sport-specific tests	1
Patient should undergo standardized functional tests before cleared for RTS	1

NOTE. The maximum score is 15 points.

BMI, body mass index; FAI, femoroacetabular impingement; RTS, return to sport.

Appendix Table 4. Rehabilitation Score

Description	Score, Points
Pain, swelling, and inflammation	4
Control pain, swelling, and inflammation	2
Interventions: crutches, cryotherapy, resting, and medication	0.5 for each (2 total)
ROM	3
Recover and improve ROM gradually as tolerated; allow full passive ROM by 21 d postoperatively, restrictions prior	1
Symmetrical ROM by 6-8 wk	1
Perform PROM (circumduction, abduction, and log rolls) starting on POD 1	1
Crutches and/or brace	3
Brace application or use	1
Discontinue use of crutches at 2 wk	1
Patient fully off using crutches and brace once gait is pain free and noncompensatory	1
Neuromuscular training and proprioceptive training	4
Phases: (1) protection of joint while avoiding irritation, (2) noncompensatory gait and progression, (3) return to preinjury level, and (4) RTS	0.25 each
Gradually increase intensity of exercises	1
Lists useful exercises: squat, planks, leg bridges, prone hip extensions, and so on	0.25 each (maximum, 2)
Sport-specific exercises	1
Gradual implementation of sport-specific exercises (after postoperative week 20)	1

NOTE. The maximum score is 15 points.

POD, postoperative day; PROM, passive range of motion; ROM, range of motion; RTS, return to sport.