YouTube Videos Lack Efficacy as a Patient Education Tool for Rehabilitation and Return to Play Following Medial Patellofemoral Ligament Reconstruction.

Purpose: To investigate the efficacy of YouTube videos as a patient education resource related to rehabilitation and return to play following medial patellofemoral ligament (MPFL) reconstruction.
Methods: YouTube was queried using 6 predetermined search terms. Videos were included if they met the following criteria: (1) written in the English language; and (2) within the first 100 videos for each search term. Videos were excluded if they met any of the following criteria: (1) not written in the English language; (2) did not include medial patellofemoral ligament/MPFL in the title; (3) duplicate videos; (4) part of a multivideo series such a vlogs; (5) advertisements; and (6) videos <1 minute. The remaining videos were evaluated by 2 independent viewers and scored using 4 distinct scoring systems: Global Quality Scale, The Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT), MPFL Rehabilitation and Return to Play Score, and the Journal of the American Medical Association benchmark criteria. The data was analyzed with IBM SPSS Statistics, version 27). The Kruskal-Wallis test was used to compare quality scores and video analytics to their assigned categories. Results where P < .05 were considered statistically significant and pairwise comparison analysis was completed to determine the video categories with statistically significant differences. Correlation of categorical variables with video analytics (views, video power index, duration, and days since publication) and quality scores was determined using the Pearson Correlation coefficient.
Results: Of the initial 600 videos, 58 met inclusion criteria, which were subsequently reviewed and scored. Most videos scored on the low-end of the possible scoring ranges with a mean Global Quality Scale score of 1.61 (standard deviation [SD] 0.81), PEMAT Understandability score of 59.40 (SD 17.54), PEMAT Actionability score of 18.20 (SD 29.92), MPFL Rehabilitation and Return to Play Score of 1.64 (SD 2.13), and Journal of the American Medical Association benchmark score of 2.08 (SD 0.75). Conclusions: The videos in YouTube's library received low scores in quality, reliability, understandability and actionability. Therefore, YouTube is currently a poor source of information for patients regarding postoperative rehabilitation and return to play following MPFL reconstruction. Clinical Relevance: Patients increasingly view medical information online. YouTube is second only to Google as the most used search engine. It is important to understand the quality of information patients receive on YouTube following MPFL reconstruction so orthopaedic surgeons know to guide patients to higher-quality alternatives.

Medial patellofemoral ligament (MPFL) reconstruction is a common surgical treatment option for patients with patellar instability. In addition, numerous studies suggest that surgical management leads to reduced rates of recurrent patellar instability for first-time patella dislocators when compared with nonoperative management.1, 2, 3 Acute traumatic patellar dislocations account for nearly 3% of all traumatic knee injuries and almost always (94%-100%) result in damage to the MPFL., Patellar dislocations have an incidence of 6 in 100,000 in the general population. Fithian et al. found the greatest incidence of acute patellar dislocations is in patients between the ages of 10 and 17 years, with 29 in 100,000 affected and 61% of all first-time acute patellar dislocations having occurred during sporting activities. Patients, such as those with MPFL injuries, commonly turn to the Internet to obtain information about surgical intervention.

YouTube is second only to Google as the most commonly used search engine for general Internet searches and is increasingly being used by patients to access health care–related information., More than a billion hours of video are watched per day on YouTube, with more than 2 billion users logging onto the website each month. Seventy-seven percent of Gen Z, which currently comprises people between the ages of 6 and 24 years, watch YouTube daily, a considerable overlap with the age group at greatest risk for patellar instability and MPFL injury. Recently, the quality of information found on YouTube pertaining to a variety of specific orthopaedic conditions was assessed.12, 13, 14, 15, 16 These studies demonstrated that most information regarding various orthopaedic injuries and procedures on YouTube is of poor quality. The purpose of this study was to investigate the efficacy of YouTube videos as a patient education resource related to rehabilitation and return to play following MPFL reconstruction. Our hypothesis was that the information found on YouTube regarding rehabilitation and return to play following MPFL reconstruction would be of low quality.

Methods

YouTube (www.youtube.com) was searched on March 28, 2021, for videos pertaining to rehabilitation and return to play following MPFL reconstruction. YouTube’s search algorithm uses age, gender, geolocation, and watch history to personalize search results. To prevent these potential confounders from affecting our search, we used Incognito mode on the Google Chrome web browser to anonymize our online identity. Incognito mode prevents YouTube from using personal browsing data and influencing search results. To conduct a comprehensive search, the first 100 videos recorded in the English language for each of the following 6 search terms were analyzed: (1) rehab and return to play medial patellofemoral ligament reconstruction; (2) rehab and return to play MPFL reconstruction; (3) rehabilitation MPFL reconstruction; (4) rehab MPFL reconstruction; (5) return to play MPFL reconstruction; and (6) recovery MPFL reconstruction. Videos were excluded if they met any of the following criteria: (1) not in the English language; (2) did not include medial patellofemoral ligament/MPFL in the title; (3) duplicate videos; (4) part of a multivideo series such as vlogs; (5) advertisements; and (6) videos <1 minute. The remaining videos were categorized into the following 5 groups based on the background of the publishing individual: nonphysician (videos not created by a physician), academic (videos created by physicians with current university, research, or medical education affiliations), independent physician (videos created by independent physicians without academic affiliations), other medical (videos created by medical sources other than physicians), and nonmedical (videos created by nonmedical sources). Two independent viewers evaluated the efficacy of the videos for patient education by using 4 different scoring systems: Global Quality Scale (GQS),13, 14, 15, 16 The Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT),, MPFL Rehabilitation and Return to Play Score (MPFL RRPS), and the Journal of the American Medical Association (JAMA) benchmark criteria. In addition, video analytics such as views, duration, days since publication, like, and dislikes were recorded. Video power index (VPI) was then calculated using the following formula to gauge user approval of the video:

Assessment of Nonspecific Video Content Quality

The quality of nonspecific video content was assessed using the GQS. The GQS is a 5-point rating system that evaluates educational content based on its quality, flow, topics covered, and usefulness (Table 1). Although unvalidated, other studies have used the GQS rating system to evaluate the nonspecific educational quality of YouTube videos for patients.13, 14, 15, 16

Table 1

Global Quality Score

Score	Global Score Description
1	Poor quality, poor flow of the site, most information missing, not at all useful for patients
2	Generally poor quality and poor flow, some information listed but many important topics missing, of very limited use to patients
3	Moderate quality, suboptimal flow, some important information is adequately discussed but others poorly discussed, somewhat useful for patients
4	Good quality and generally good flow, most of the relevant information is listed, but some topics not covered, useful for patients
5	Excellent quality and excellent flow, very useful for patients

Assessment of Specific MPFL Rehabilitation and Return to Play Video Content Quality

To assess the quality of video content specific to the rehabilitation and return to play for athletes following MPFL reconstruction, we developed the MPFL RRPS. Currently, there are no published rehabilitation or return to play guidelines following MPFL reconstruction that have been widely adopted, and so the MPFL RRPS was based on the current literature.22, 23, 24, 25 The MPFL RRPS is composed of a checklist of 8 topics and is scored in a binary fashion; 1 point is given for any mention of each checklist item and 0 points are given for no mention of each checklist item, with a maximum score of 8 (Tables 2 and 3). Novel scoring checklists have been used in similar studies to assess the educational content of YouTube videos for specific orthopaedic topics.13, 14, 15, 16

Table 2

MPFL Rehabilitation and Return to Play Score Topics

Pain management

Immobilization/bracing

Range of motion exercises

Weight-bearing guidelines

Strengthening exercises

Proprioception exercises

Functional testing prior to return to play

Timeline for rehabilitation/return to play

MPFL, medial patellofemoral ligament.

Table 3

MPFL Rehabilitation and Return to Play Score Content Ratings

Score	Quality of MPFL Rehabilitation and Return to Play Content
1-2	Poor
3-4	Moderate
5-6	Good
7-8	Excellent

MPFL, medial patellofemoral ligament.

Assessment of the Understandability and Actionability of Video Content

To determine the usefulness of the video content for a broad range of patients, we used the PEMAT scoring system. This tool consists of 13 items measuring understandability and 4 items measuring actionability scored in a binary fashion, resulting in a percentage score. Understandability is defined as the ability of patients from a diverse background with varying levels of health literacy to process and explain the main video topics. Actionability is defined as the ability of patients from diverse backgrounds and varying levels of health literacy to act based on the information presented in the videos., Although other studies did not use the PEMAT scoring system, this study deemed the ability of patients to comprehend and act on video content to be an important characteristic to assess.12, 13, 14, 15, 16

Assessment of the Reliability and Quality of Video Source Information

The reliability and quality of the information source for each video was assessed using the JAMA benchmark criteria. This scoring system rates the reliability and quality of source information by determining whether the information meets 4 criteria: clear authorship, attribution, disclosure, and currency (Table 4). Each of the 4 criteria is scored in a binary fashion, with a maximum score of 4 denoting excellent reliability and a quality source of information. As with the GQS, other similar studies have used this method of scoring the JAMA benchmark criteria to assess the reliability and quality of YouTube videos.12, 13, 14, 15, 16

Table 4

The Journal of the American Medical Association (JAMA) Benchmark Criteria

Authorship	Authors and contributors, their affiliations, and relevant credentials should be provided.
Attribution	References and sources for all content should be listed clearly, and all relevant copyright information noted.
Disclosure	Web site “ownership” should be prominently and fully disclosed, as should any sponsorship, advertising, underwriting, commercial funding arrangements or support, or potential conflicts of interest. This includes arrangements in which links to other sites are posted as a result of financial considerations. Similar standards should hold in discussion forums.
Currency	Dates that content was posted and updated should be indicated.

Statistical Analysis

The collected data were analyzed with IBM SPSS Statistics, version 27 (IBM Corp., Armonk, NY). To analyze the 2 reviewers’ agreement for each scoring system, the interclass correlation coefficient (ICC) (model: 2-way fixed, type: consistency), along with their 95% confidence intervals (CIs), were found. The closer the ICC value is to 1.0, the greater the agreement between the video reviewer’s scores. The scores of the 2 reviewers were then averaged for further analysis.

Using the Shapiro–Wilk test, we determined whether the assessed variables resembled a normal distribution. Since not all variables showed significant normality, the Kruskal–Wallis test was used to compare quality scores and video analytics with their assigned categories. Results where P < .05 were considered statistically significant and pairwise comparison analysis was performed to determine the video categories with statistically significant differences.

Correlation of categorical variables with video analytics (views, VPI, duration, and days since publication) and quality scores was determined using the Pearson Correlation coefficient. The significance of each correlation was also found and correlations with P < .05 were considered statistically significant.

Results

Of the initial 600 videos, 542 met 1 or more of the exclusion criteria. Seven videos were advertisements, 247 were duplicates, 92 were vlogs or multipart series, 24 were less than 1 minute in length, and 213 did not contain information about MPFL in their title. This left 58 videos to be scored. The mean duration of the videos was 9:19 (standard deviation [SD] 11:20), mean number of views was 13,864 (SD 30,273), and mean VPI was 95.73 (SD 8.60). The number of videos assigned to each category is outlined in Figure 1 and Table 5. The frequency of scores by both reviewers are outlined in Fig 2, Fig 3, Fig 4, Fig 5, Fig 6 and Table 6, Table 7, Table 8, Table 9, Table 10. The averaged results of the scoring systems for each video category are summarized in Table 11.

Fig 1

Number of videos per video category. Non-physician: Videos not created by a physician. Academic: Videos created by physicians with current university, research, or medical education affiliations. Other Medical: Videos created by medical sources other than physicians. Non-medical: Videos created by non-medical sources. Ind. Physician: Videos created by independent physicians without academic affiliations.

Table 5

Number of Videos Per Video Category

Video Category	Number of Videos
Nonphysician	3
Academic	29
Other medical	7
Nonmedical	3
Independent physician	16

NOTE. Nonphysician: Videos not created by a physician. Academic: Videos created by physicians with current university, research, or medical education affiliations. Other medical: Videos created by medical sources other than physicians. Nonmedical: Videos created by non-medical sources. Independent physician: Videos created by independent physicians without academic affiliations.

Fig 2

Number of videos per GQS. (GQS, Global Quality Score.)

Fig 3

Number of videos per PEMAT Understandability Score. (PEMAT, Patient Education Materials Assessment Tool for Audiovisual Materials.)

Fig 4

Number of videos per MPFL RRPS score. (MPFL, medial patellofemoral ligament; RRPS, Rehabilitation and Return to Play Score.)

Fig 5

Number of videos per PEMAT Actionability Score. (PEMAT, Patient Education Materials Assessment Tool for Audiovisual Materials.)

Fig 6

Number of videos per JAMA Benchmark Score. (JAMA, Journal of the American Medical Association.)

Table 6

Number of Videos per GQS Score

Global Quality Score (GQS)	Number of Videos
1	65
2	36
3	11
4	3
5	1

GQS, Global Quality Score.

Table 7

Number of Videos per PEMAT Understandability Score

Score (%)	Number of Videos
0-20	0
21-25	5
26-30	0
31-35	6
36-40	1
41-45	13
46-50	6
51-55	15
56-60	5
61-65	19
66-70	6
71-75	13
76-80	9
81-85	14
86-90	0
91-95	4
96-100	0

PEMAT, Patient Education Materials Assessment Tool for Audiovisual Materials.

Table 8

Number of Video per MPFL RRPS Score

MPFL RRPS	Number of Videos
0	63
1	10
2	5
3	11
4	8
5	12
6	4
7	3
8	0

MPFL RRPS, Medial Patellofemoral Ligament Rehabilitation and Return to Play Score.

Table 9

Number of Videos per PEMAT Actionability Score

Score (%)	Number of Videos
0-10	77
11-30	0
31-40	22
41-50	0
51-60	1
61-70	8
71-100	0

PEMAT, Patient Education Materials Assessment Tool for Audiovisual Materials.

Table 10

Number of Videos per JAMA Benchmark Score

JAMA Benchmark Score	Number of Videos
0	0
1	28
2	58
3	23
4	7

JAMA, Journal of the American Medical Association.

Table 11

Summary of Video Data

Variable	All Videos (n = 58)	Nonphysician (n = 3)	Academic (n = 29)	Other Medical (n = 7)	Nonmedical (n = 3)	Independent Physician (n = 16)
Global Quality Score (GQS)	1.61 (.81)	2.33 (0.58)	1.36 (0.53)	1.64 (1.11)	1.67 (0.58)	1.91 (1.02)
PEMAT Understandability	59.40 (17.54)	65.00 (18.52)	58.52 (14.73)	58.86 (20.54)	54.00 (28.05)	62.06 (20.49)
PEMAT Actionability	18.20 (29.92)	55.67 (50.95)	6.55 (13.50)	23.79 (31.73)	22.00 (19.05)	29.13 (39.70)
MPFL RRPS	1.64 (2.13)	2.00 (2.65)	1.22 (1.85)	1.64 (2.69)	1.67 (1.53)	2.31 (2.40)
JAMA Benchmark	2.08 (0.75)	1.50 (0.50)	2.48 (0.77)	1.86 (0.48)	1.17 (0.29)	1.72 (0.45)
Days Since Publication	1,500.26 (1,044.98)	1,426.67 (899.44)	1,433.62 (1,112.14)	1,231.86 (1,508.63)	2,011.33 (1,157.38)	1,656.44 (722.70)
Number of Views	13,864.36 (30,273.83)	2,751.00 (1,771.18)	11,872.24 (29,655.84)	16,729.71 (40,851.78)	39,291.33 (66,159.81)	13,537.69 (20,587.29)
Duration (sec)	559.12 (679.54)	454.33 (450.07)	550.72 (395.27)	1,109.14 (1694.17)	123.67 (21.73)	435.00 (300.12)
Video Power Index (VPI)	95.73 (8.60)	96.30 (6.42)	93.39 (11.70)	98.66 (2.61)	98.40 (2.77)	97.74 (2.39)

NOTE. Nonphysician: Videos not created by a physician. Academic: Videos created by physicians with current university, research, or medical education affiliations. Other medical: Videos created by medical sources other than physicians. Nonmedical: Videos created by non-medical sources. Independent physician: Videos created by independent physicians without academic affiliations.

GQS, Global Quality Score; JAMA, Journal of the American Medical Association; MPFL RRPS, Medial Patellofemoral Ligament Rehabilitation and Return to Play Score; PEMAT, Patient Education Materials Assessment Tool for Audiovisual Materials.

The ICC demonstrated a good agreement of 0.82 (CI 0.69-0.89) between the 2 reviewers for the JAMA benchmark, while showing excellent agreement for the GQS, PEMAT Understandability, PEMAT Actionability, and MPFL RRPS scoring systems with values of 0.94 (CI 0.89-0.96), 0.98 (CI 0.96-0.99), 0.98 (CI 0.97-0.99), and 0.96 (CI 0.94-0.98), respectively. The mean GQS was 1.61 (SD 0.81) (CI 1.40-1.82), with 56% being characterized as poor and 31% as generally poor quality by the GQS (Fig 2 and Table 6). The novel MPFL RRPS also showed that, on average, the videos were of poor quality, with a mean of 1.64 (SD 2.13) (CI 1.08-2.20) (Fig 4 and Table 8). In fact, the scores from both reviewers indicated that the majority of videos (54%) did not mention a single included criterion (Fig 4 and Table 8). The ability for YouTube users to both understand and take away actionable information from these videos was also less than satisfactory, with an average PEMAT Understandability score of 59.4 (SD 17.54) (CI 54.79-64.01) and Actionability score of 18.2% (SD 29.92) (CI 10.33-26.06) (Table 1) (Figs 3, 5, and Tables 7, 9).

Through the Kruskal–Wallis comparison of average score results to the videos’ assigned category, only JAMA benchmark scores were found to significantly differ across categories (P < .001). Further pairwise comparisons indicated that a difference existed between academic and non-medical sources (P = .02) as well as academic and independent physician sources (P = .02). This indicates that video category does not signify a more useful video in any scoring system other than the JAMA benchmark.

The Pearson coefficients indicated that average GQS (r = 0.30, P = .02) and MPFL RRPS (r = 0.31, P = .02) were significantly correlated with video duration. In addition, the JAMA benchmark was found to be significantly and inversely correlated to the days since publication (r = –0.32, P = .01), as newer videos were found to have greater scores. Both views and VPI were not significantly correlated with any scoring criteria.

Discussion

The study demonstrates that YouTube provides poor educational resources to patients looking for information about rehabilitation following MPFL reconstruction. With a mean GQS score of 1.61, PEMAT Understandability score of 59.40, PEMAT Actionability score of 29.92, MPFL RRPS of 1.64, and JAMA benchmark score of 2.08, the videos in YouTube’s library fall on the low end of the possible scoring ranges, demonstrating significant room for improvement. Therefore, the videos found do not provide patients with the high-quality information necessary to understand the recovery process following MPFL reconstruction. However, the data also show that videos published more recently have greater JAMA scores. This may be due, in part, to similar studies12, 13, 14, 15, 16 having found YouTube to be a poor source of information regarding various orthopaedic topics, thereby prompting the development of more reliable and higher-quality videos on YouTube.

In addition to most videos receiving poor marks for each scoring system, the lack of correlation between video views and reviewers’ scores highlights that many YouTube users are not accessing the highest-quality videos offered on YouTube. This may be due, in part, to YouTube’s search algorithm. By using user age, gender, geolocation, and watch history to personalize search results, YouTube users may be accessing more popular videos that the algorithm shows them, not necessarily they highest-quality videos. Significant correlations were found between video duration and average GQS scores and MPFL rehabilitation scores. Although longer videos received greater marks in these scoring categories, it is important to note that videos experience a significant fall-off in user engagement if they are longer than 2 minutes and another decrease if they are longer than 12 minutes. Keeping videos short, while still containing relevant information, is an important goal to maximize user engagement.

The majority of similar orthopaedic studies have found comparable results. Springer et al. reviewed YouTube videos concerning rehabilitation following anterior cruciate ligament reconstruction and concluded that YouTube provided poor information quality, reliability, and accuracy. In 2 separate studies, Kunze et al., concluded that YouTube was a poor source of information for patient education on the meniscus and posterior cruciate ligament. Kocyigit et al., however, found YouTube to be a high-quality source of information regarding ankylosing spondylitis exercises, which may be because this topic is nonoperative and therefore requires less technical information for a comprehensive video. This discrepancy shows that while YouTube does not currently provide high-quality information on many orthopaedic topics, it has the potential to provide the information necessary to make patients feel more comfortable about what to expect as they begin postoperative rehabilitation.

All of the aforementioned studies used similar scoring systems and statistical analysis to reach their conclusions. While the GQS, PEMAT, and JAMA systems were used in this study to measure the overall efficacy of the videos as a patient education resource, the novel MPFL RRPS was developed to specifically review each video's usefulness in regards to MPFL rehabilitation. The MPFL RRPS revealed that the most infrequently discussed topics in the videos were pain management (8 videos, 13.79%), functional testing guidelines prior to return to play (5 videos, 8.62%), and proprioception exercises (3 videos, 5.17%). These topics are important aspects of MPFL rehabilitation and the infrequency with which they were discussed further highlights the overall poor efficacy of educational resources available to YouTube users seeking information on MPFL rehabilitation. As previously discussed, the greatest incidence of acute patellar dislocation occurs between the ages of 10 and 17 years. As younger generations become more reliant on the internet as a source of information, it will be increasingly important for the websites they use to provide them with high-quality information.

Limitations

There are several limitations to this study. First, this is a cross-sectional study. With the popularity of YouTube, it is possible that more videos related to rehabilitation following MPFL reconstruction have since been added. YouTube may become a more reliable source of MPFL rehabilitation information as the video library expands. Second, there is no peer-review process in place for videos published on YouTube; therefore, there is no system to oversee the quality or efficacy of information shared. In addition, since YouTube was the only search engine queried, there is a potential for selection bias. Another limitation is the use of Google Chrome’s Incognito mode to prevent YouTube from using our browsing data to personalize our search results and adding potential confounders. However, each patient who uses YouTube to learn about rehabilitation and return to play following MPFL reconstruction has their own unique personalized search results. Furthermore, our search terms may not be reflective of those that the patient population would use. Both of which, may affect the generalizability of our results. Another limitation of this study is that the GQS and MPFL RRPS scoring systems used are unvalidated. In addition, all of the scoring systems used are subjective in nature. The use of the ICC to highlight excellent agreement for the majority of scoring systems somewhat diminishes this limitation. However, further analysis with more reviewers could lead to more concrete agreement over the quality and reliability of the videos studied.

Conclusions

The videos in YouTube’s library received low scores in quality, reliability, understandability and actionability. Therefore, YouTube is currently a poor source of information for patients regarding postoperative rehabilitation and return to play following MPFL reconstruction.