Analysis of the Educational Value of YouTube Laparoscopic Appendectomy Videos.

Purpose: To evaluate the educational value of laparoscopic appendectomy (LA) videos on YouTube for surgical trainees.
Methods: The search term "Laparoscopic appendectomy" was used on YouTube. The top 100 videos sorted by the number of views were evaluated. Each YouTube account was analyzed, and only videos uploaded by medical physicians were included in this study. Video quality was evaluated using an arbitrary appendectomy scoring system. Video characteristics and Global Operative Assessment of Laparoscopic Skills (GOALS) scores were analyzed regarding video quality and upload source.
Results: The video quality of 14 (25.0%) videos was graded as good, 36 (64.3%) moderate, and 6 (10.7%) of poor quality. Video characteristic analysis showed no differences in video quality according to the upload source (p=0.573). Video quality and upload source were not related to video length, total views, days online, number of likes, number of dislikes, number of comments, or GOALS score. Among the factors analyzed, only appendicitis severity was found to be associated with video grade (p=0.049).
Conclusion: The quality of LA YouTube videos varied. Categories considered as viewer feedback were not associated with video grade or upload source. Responsible video uploading by academic institutions, and appropriate censorship by YouTube seems necessary. Further research with objective data on actual application to surgical trainees is necessary.

INTRODUCTION

Internet access has become easier with the development of mobile devices. This has led to significant changes in the field of surgical education.1 One of these changes includes the use of medical information through video material, which is a form of multimedia learning. Education using multimedia has the advantage of converting cognitive input into long-term memory.2 According to a study by Rapp et al.3 95% of surgical trainees have used video to prepare for surgery. In particular, 95% reported obtaining information from YouTube. YouTube is world’s largest free video sharing website, allowing users to freely upload, view, share, and comment on videos.4 Because of these features, YouTube is an important source for sharing medical information. However, the quality of medical information contents is not guaranteed because it is not a medical website.

Laparoscopic surgery has video-based features, and a large number of laparoscopic surgery videos have been uploaded on YouTube. Laparoscopic appendectomy (LA) is a surgical procedure primarily performed when surgical trainees begin to learn laparoscopic surgical techniques.5 There are many LA videos uploaded on YouTube, so it can be inferred that many, if not most, surgical trainees will access YouTube to view LA videos.

The authors of this study have previously analyzed YouTube laparoscopic cholecystectomy (LC) videos, and concluded that videos uploaded by tertiary centers were of better quality than those uploaded by secondary hospitals or private institutions.6 However, this quality discrepancy was not recognized by viewers. We sought to evaluate the quality of LA video content on YouTube, perform an analysis by video grade and upload source, and analyze viewer feedback.

MATERIALS AND METHODS

Study design

The term “laparoscopic appendectomy” was searched on YouTube, on May 1, 2018. The top 100 videos sorted by the number of views were evaluated. The “incognito mode” web browser was used so that search results are not affected by viewing history or cookies. Of the 100 videos, those uploaded by non-medical individuals via YouTube ID analysis were excluded. Animation or slide-based presentations, videos demonstrating other procedures such as an incisional hernia were excluded. Video characteristics including video length, total number of views, days online, views per day, likes, dislikes, number of comments, upload source, and appendicitis severity were reviewed. A Google search was performed for each YouTube ID to find the upload source. General hospitals and medical websites dedicated to medical education were classified as academic centers. Hospitals that were not general hospitals were classified as private centers. Videos with no hospital information were classified as independent surgeons. Appendicitis severity was classified as non-complicated appendicitis for normal looking or mildly inflamed appendix and complicated appendicitis for more severe cases, according to the appendicitis grading system proposed by Gomes et al.7

Institutional Review Board approval was unnecessary for this study, since only public access data was used.

Evaluation of video quality

Since there is no standardized system for evaluating the quality of the LA procedure, an arbitrary appendectomy procedure scoring system was devised by referring to textbooks and journals (Table 1).8–10 Each category was scored based on whether the entire procedure was illustrated in the video. A total score of 6~7 points was classified as “good,” 4~5 points as “moderate,” and 3 points or less as “poor.” The overall laparoscopic surgical technique was evaluated through Global Operative Assessment of Laparoscopic Skills (GOALS) score.11 The GOALS score is originally composed of 5 categories (depth perception, bimanual dexterity, efficiency, tissue handling, autonomy), and is graded on a 5-point Likert scale. The Autonomy category was excluded because it could not be determined through video analysis. Therefore, the maximum GOALS score was 20 points.

Table 1

Appendectomy procedure score

Category	Assessment	Score
Inspection	Not demonstrated	0
	Inspection of entire abdomen	1
Mesoappendix ligation	Not demonstrated	0
	Demonstrated with proper hemostasis	1
Appendix ligation	Stump length >3 mm	0
	Stump length <3 mm	1
Contamination material spillage prevention	None	0
	Stump manipulation (lumen electro-cauterization or endo-stapler)	1
	Specimen manipulation (grasper holding or endo-loop or endo-stapler)	1
Appendix extraction	Inappropriate	0
	Use of the endo-bag	1
Peritoneal lavage	Not demonstrated	0
	Irrigation (Douglas pouch and RLQ, at least)	1
Total score[*]		0~7

Total score: 0~3 = poor; 4~5 = moderate; 6~7 = good.

RLQ = Right Lower Quadrant.

All researchers, who had more than 50 cases of LA experiences, assessed the quality of the video independently. Disagreements between the researchers about the scoring of a particular video were resolved by discussing the issue until a consensus was reached.

Statistical analysis

Data were analyzed using IBM SPSS statistics version 21.0 software (IBM Co., Armonk, NY, USA). Continuous variables were analyzed using Kruskal-Wallis analysis, with the Mann-Whitney test for Post-hoc comparison. The Bonferroni method was used to set the significance method. Categorical data were analyzed using Fisher’s exact test. p<0.05 was considered significant, p<0.017 was considered significant in post-hoc comparison.

RESULTS

Among the 100 videos identified, 44 were excluded. The excluded videos consisted of 3 videos uploaded by non-medical individuals, 16 non-surgical videos, 13 videos with poor quality for analysis, and 12 videos not demonstrating LA. After exclusion, a total of 56 videos were included in the study.

The quality of the videos was analyzed using the appendectomy procedure score. Overall, 14 (25%) were graded as good, 36 (64.3%) as moderate, and 6 (10.7%) as poor. Table 2 shows the demographics of the videos by quality. The only characteristic that correlated with video grade was appendicitis severity (p=0.049). No difference was found in total views, days online, likes, dislikes, number of comments, upload source, or GOALS score between the different video quality groups. Although the mean length differed based on the video quality (p=0.028), the post-hoc test showed that the p-value was higher than 0.017, indicating that there was no significant difference.

Table 2

Analysis by video quality

Video demographics	Video quality			Total	p value
	Good	Moderate	Poor
Videos	14 (25.0)	36 (64.3)	6 (10.7)	56	-
Mean score	6.36±0.50	4.47±0.51	2.33±1.03	4.71±1.29
Inspection	0.64±0.50	0.19±0.40	0.00±0.00	0.29±0.46	0.001
Mesoappendix ligation	1.00±0.00	0.94±0.23	0.83±0.41	0.95±0.23	0.148
Stump ligation	1.00±0.00	0.92±0.28	0.83±0.41	0.93±0.26	0.164
Specimen manipulation	1.00±0.00	0.97±0.17	0.33±0.52	0.91±0.29	<0.001
Stump manipulation	0.93±0.27	0.83±0.38	0.00±0.00	0.77±0.43	<0.001
Appendix extraction	0.71±0.47	0.39±0.49	0.17±0.41	0.45±0.50	0.013
Peritoneal lavage	0.79±0.43	0.22±0.42	0.17±0.41	0.36±0.48	0.001
Mean length	11:16±09:39	07:39±06:06	05:00±04:40	08:16±07:10	0.028[*]
Mean views	36,423±83,387	28,995±52,962	37,404±57,401	31,753±61,172	0.565
Days online	1,784±884	2,122±763	2,171±1,074	2,043±827	0.490
Views per day	34.1±87.7	16.1±32.9	35.3±74.9	22.7±55.7	0.966
Mean likes	69.6±155.6	81.6±309.7	87.5±191.5	79.2±264.8	0.713
Mean dislikes	6.9±15.4	5.2±11.3	6.8±13.3	5.8±12.4	0.872
Mean comments	12.3±34.5	20.8±77.7	7.8±9.6	16.0±62.7	0.761
Upload source					0.675
Independent surgeon	6 (42.9)	17 (47.2)	2 (33.3)	25
Private center	0	6 (16.7)	2 (33.3)	8
Academic center	8 (57.1)	13 (36.1)	2 (33.3)	23
Appendicitis severity					0.049
Non-complicated	9 (64.3)	30 (83.3)	6 (100.0)	45
Complicated	5 (35.7)	6 (16.7)	0	11
GOALS	18.93±1.27	18.53±2.02	17.83±2.40	18.55±1.90	0.625

Values are presented as number (%) or mean±standard deviation.

Good vs. Moderate = 0.053; Good vs. Poor = 0.033; Moderate vs. Poor = 0.088.

Table 3 shows the demographics of videos by upload source. Overall, 25 (44.6%) were independent surgeons, 8 (14.3%) were private centers, and 23 (41.1%) were academic centers. No statistically significant differences were seen in video quality, mean length, mean views, days online, views per day, mean likes, mean dislikes, mean number of comments, appendectomy severity, or GOALS score between the different sources of video uploaded.

Table 3

Analysis by upload source

Video demographics	Video source			Total	p value
	Independent Surgeon	Private center	Academic center
Videos	25 (44.6)	8 (14.3)	23 (41.1)	56	-
Mean score	4.88±1.13	3.88±1.36	4.83±1.37	4.71±1.29	0.216
Mean length	09:41±08:28	08:04±06:59	06:49±05:31	08:16±07:10	0.198
Mean views	19,911±29,493	13,416±17,922	51,002±87,379	31,753±61,172	0.351
Days online	2,109±914	2,150±695	1,933±788	2,043±827	0.768
Views per day	11.2±17.5	5.8±7.5	41.0±82.4	22.7±55.7	0.374
Mean likes	27.8±44.1	9.9±6.0	159.1±402.1	79.2±264.8	0.143
Mean dislikes	3.2±4.7	1.9±1.8	10.0±18.1	5.8±12.4	0.322
Mean comments	8.3±17.1	4.3±4.9	28.3±96.0	16.0±62.7	0.661
Video quality					0.573
Good	6 (24.0)	0	8 (34.8)	14
Moderate	17 (68.0)	6 (75.0)	13 (56.5)	36
Poor	2 (8.0)	2 (25.0)	2 (8.7)	6
Appendicitis severity					0.222
Non-complicated	22 (88.0)	6 (75.0)	17 (73.9)	45
Complicated	3 (12.0)	2 (25.0)	6 (26.1)	11
GOALS	18.56±1.61	18.50±2.33	18.57±2.11	18.55±1.90	0.863

Values are presented as number (%) or mean±standard deviation.

DISCUSSION

Basic laparoscopic techniques are important in all fields of laparoscopic surgery. Inadequate techniques may cause potentially fatal complications. Therefore, it is essential for surgical trainees to learn these techniques properly at the beginning of their training. Surgical trainees most often begin their surgical training with LA, and YouTube is one of the most often sought training platforms. To our knowledge, this is the first study to examine the quality of YouTube videos of LA.

An important basic principle of surgery is to minimize postoperative complications. Although postoperative intra-abdominal abscess (IAA), such as stump appendicitis, has a low incidence, it is one of the most serious complications.12 The incidence of IAA depends on how well the stump and specimens are manipulated. The stump should be ligated to less than 3 mm to avoid stump appendicitis.8 In our analysis, this process had a high average score of 0.93, suggesting that most surgeons in the videos included in this study were aware of the importance of this process. Visible mucosa must be electro-cauterized when the endo-loop is used in the stump ligation process.10 This is because a contaminated stump mucosa is likely to extrovert and there is risk of postoperative leak, which can lead to IAA.13 In this study, we found 13 videos not illustrating this process, including 10 without electro-cauterization; three were edited so it was unclear whether or not the procedure was performed. Since this procedure is a mandatory step that trainees must know, video content creators should not skip this process when editing. To prevent contaminated material dissemination of the specimen at the time of appendix division, the specimen opening must be held closed using a grasper or endo-loop. Videos received no points if the grasper or endo-loop was located too far from the opening or if the appendicolith or contaminated mucosa spread into the peritoneal cavity. However, most surgeons seemed to perform this procedure well, and the average score was 0.91.

It is better to use an endo-bag when extracting the specimen from the abdominal cavity.9 Previous studies have suggested that directly extracting the specimen through the trocar is more cost-effective than specimen extraction using an endo-bag.14 However, from the videos observed in our study, during direct specimen extraction, even when the size of the specimen was very small, contaminated material was often observed in the trocar tip, which could potentially cause IAA or surgical site infection. Textbooks and the Society of American Gastroenterology Surgeons (SAGES) guideline recommend peritoneal lavage after specimen extraction, to reduce the incidence of IAA by washing contaminated materials with saline solution.9,10,15 However, some surgeons object to this guideline. St. Peter et al.16 reported that irrigation can increase the risk of IAA because it cannot reduce the microorganism load of bacteria attached to peritoneum mesothelial cells, is likely to spread contamination and can dilute the mediator of phagocytosis. In a recently published systemic review, the authors concluded that peritoneal lavage had no additional benefit and only lengthened operation time.17 In light of this controversy, we expect that trainees should first refer to textbooks and guidelines when learning LA theoretically, and include this procedure in the scoring table.

In principle, surgical videos for educational purpose should include a description of the patient position and trocar insertion site. However, most of the videos included in our study began with the insertion of the laparoscope. The reason for not including both steps is that it is not recorded with the laparoscope, and the process of recording this step before surgery or making animation is troublesome. We found one (4.3%) video containing both steps from an academic center group; it seems that educators pay attention to these processes. Another procedure that requires more attention is inspection. As the video grade improved, the number of videos with inspection procedure increased. However, the average was 0.64, even in the videos of good grade. Inspection is not a major procedure in the LA process but is an important opportunity to identify pathological findings not otherwise found in an imaging study.

In a recent systematic review, most studies frequently focused on whether the characteristics of YouTube videos were associated with the parameters video quality, upload source, and the number of likes.18 The authors of this study have previously analyzed LC YouTube videos and reported a quality difference based on the upload source.6 Giovanni et al.19 also reported that there might be a quality difference based on the upload source in laparoscopic fundoplication YouTube videos. In our study, videos uploaded by academic centers had the highest percentage of good grade videos (57.1%), but there were no statistically significant differences between the upload sources. This is likely because LA is a relatively easy procedure compared to previously analyzed procedures such as laparoscopic fundoplication or LC. Therefore, good quality videos can be recorded by centers of all sizes. The average of number of likes, although not statistically significant, was the highest in videos of poor grade. This is presumably because, as well as surgical trainees, many medical students and general public also watch a lot of videos to get information about LA. This suggests that the number of likes can be affected by the visual quality level of the video rather than demonstrating medical knowledge such as surgical procedure or technique in the video.

We classified appendicitis severity into two groups. The rate of complicated appendicitis was high in good grade videos. In complicated appendicitis cases, appendix inflammation is severe, and pericolic fluid due to perforation is likely to be present. The use of endo-staplers, endo-bags, and irrigation are often required in these cases, which may have led to higher scores. However, appendicitis severity is difficult to ascertain before actually viewing the video. Therefore, it is inappropriate to use it as a filtering condition for a good quality LA video.

YouTube allows the dissemination of health care information including information about surgery, but some videos are incorrect or misleading, according to a systemic review published in 2015.20 In addition, most studies analyzing YouTube medical information seek to determine conditions that can filter good quality videos. It is not easy to identify a suitable condition to filter videos, because of the frequency with which they are uploaded. Video uploaders should always remember that numerous trainees will potentially watch their videos and thus, the authors should seek to improve the quality of video content through self-review before uploading, if peer review is difficult. Trainees watching LA YouTube videos should be aware that the quality of these videos is highly variable and that a high number of views or likes does not translate into an accurate or informative video.

We acknowledge that the study has some limitations. The most important limitation is the lack of actual application to surgical trainees. Further research including objective data after application, and trainees’ subjective responses to these videos would contribute to this topic. The list of YouTube videos changes frequently, thus, the results of our study may differ if our study methods are used at a later date. Moreover, there is a possibility that when identifying the upload source, the YouTube ID may not correctly represent the uploader, since it can be created without any verification. Since there is no standardized scoring system to evaluate a LA procedure, an arbitrary scoring system was created and used. However, we expect that the observer-related bias is low because our evaluation method is not quantitative as in the 5-point Likert scale, but only determines whether each procedure is demonstrated in the videos.

In conclusion, the quality of LA YouTube videos varies. Categories that can be considered as viewer feedback were not associated with video grade or upload source. Multimedia learning using video content of questionable quality can be potentially dangerous. Responsible video uploading by academic institutions and appropriate censorship by YouTube are necessary. Further research with objective data on actual application to surgical trainees is necessary.