YouTube: A good source for retrograde intrarenal surgery?

PURPOSE: To evaluate the quality of videos for retrograde intrarenal surgery (RIRS) on YouTube (Google, LLC) from the perspective of both patients and physicians.
MATERIALS AND METHODS: All videos longer than 2 minutes returned by the YouTube search engine in response to the keyword search "retrograde intrarenal surgery" were included in this study. The quality of content was analyzed by using the validated Journal of the American Medical Association Benchmark Score (JAMAS) and the Global Quality Score (GQS). Two surgeons developed the RIRS Scoring System (RIRSSS) to evaluate the technical quality of the videos. A video power index (VPI) was used to score the popularity of the videos.
RESULTS: A total of 63 videos with a median of 389 views were included in the present study. Forty-three videos (68.3%) were provided by health care professionals and 53 videos (84.1%) included technical aspects about RIRS. The median (interquartile range) GQS, JAMAS, RIRSSS, and VPI scores were 2 (1-3), 1 (1-2), 2 (1-5), and 0.41 (0.08-1.29), respectively. Videos with audio had significantly higher GQS and RIRSSS scores than did with videos with no audio (p<0.001, p=0.039, respectively). The GQS of videos providing general information about RIRS was higher, whereas RIRSSS scores were higher for videos detailing technical aspects (p=0.027, p=0.038, respectively).
CONCLUSIONS: The quality of YouTube videos containing information about RIRS evaluated in this study was very low. It is necessary for health care organizations to prepare online materials and upload these materials to popular social media platforms to convey accurate information to patients. © The Korean Urological Association, 2021.

INTRODUCTION

The incidence of kidney stones has increased dramatically in recent years [12]. In the pediatric age group, the average annual increase in incidence is reported to be about 4% [2]. In the adult age group, the relative increase in the incidence rate was shown to be 1.29 in women and 1.14 in men, and the incidence of symptomatic stones has increased more than that of asymptomatic stones (1.58 vs. 1.11) [1]. Geographic environmental factors, dietary characteristics, and metabolic disorders are thought to play a role in this situation [3]. With the advances in endoscopic and laser technology [4], the role of retrograde intrarenal surgery (RIRS) in the treatment of kidney stone disease has gradually increased [56]. According to the European Association of Urology (EAU) guidelines, RIRS is the first-line recommended treatment method for kidney stones <2 cm [7].

The internet is increasingly used for health purposes [89]. Approximately two-thirds of the population has started to use the internet for health purposes and these rates are expected to gradually increase [89]. There are many different social media platforms providing health information on the internet [10]. One of these social media sources is YouTube (Google, LLC). YouTube is the world's largest video-sharing platform, watched by over 2 billion users every day, and providing over 1 billion hours of videos [11]. However, some of the data about health information on YouTube is known to be deceptive and inaccurate [12].

Other studies in the literature have evaluated video platforms for different urologic and nonurologic diseases [1314151617181920]. To the best of our knowledge, however, videos for RIRS on YouTube have not been previously evaluated. Therefore, in the present study, we assessed the quality of available videos on RIRS on YouTube using validated questionnaires and a scoring system developed to evaluate the preoperative, intraoperative, and postoperative features of RIRS.

MATERIALS AND METHODS

We conducted a search on YouTube using the keyword “retrograde intrarenal surgery” on January 15, 2020. Popularity-based ranking was used to sort the related videos. All videos longer than 2 minutes were taken into consideration. Videos containing any information about RIRS; videos recorded by medical professionals, patients, or industry; and educational videos were included in the evaluation. In case of duplicate videos, only one was considered. Nonrelevant videos were excluded from the study. Seventy-five videos were reviewed. After applying the exclusion criteria, 63 videos were included in the study. After saving the search results in a playlist, two independent researchers (urologists, H.B.H., and S.T.) performing RIRS in routine daily practice viewed and analyzed the videos. All scorings were done separately by two surgeons. Discrepancies between researchers were discussed and clarified.

The videos were classified into groups according to source of upload (academic center, urologist, commercial, and other/unknown), video content (general information and technical aspects), country of origin (Europe, Asia, and Africa) and video language (English, other, and no audio). For each video, the number of views, likes, dislikes, upload date, video length (second), duration on YouTube (days), like ratio (like/like +dislike), and view ratio (the number of views/duration on YouTube) were collected.

Popularity of videos was evaluated by the video power index (VPI) described by Erdem and Karaca [16]. VPI was calculated by the formula: VPI=like ratio×view ratio/100. Every video was assessed by using the Global Quality Score (GQS) [21] and the Journal of the American Medical Association Benchmark Score (JAMAS) as described in previous studies [22]. The GQS is a five-point Likert scale and has functions to determine whether the publication would be useful to patients or not. The JAMAS has four different criteria (authorship, association, disclosure, and validity) with 1 possible point for each criterion and is used to determine the validity, effectiveness, and reliability of the publication. The RIRS Scoring System (RIRSSS) was developed by two surgeons to evaluate the demographics, diagnosis, intraoperative features, complications, and postoperative follow-up period according to the EAU guidelines (Table 1) [7]. The RIRSSS consisted of 19 criteria, and 1 point was given for each criterion written or orally presented in the video.

Table 1

Retrograde Intrarenal Surgery Scoring System (RIRSSS)a

A. Preoperative evaluation
	1. Was tde age of tde patient specified on video?
	2. Was tde gender of tde patient specified on video?
	3. Was tde body mass index value of tde patient specified on video?
	4. Was tde patient's comorbid diseases stated on video?
	5. Were preoperative abdominal imaging findings specified on video?
	6. Was information about tde patient's previous surgery history stated?
B. During surgery
	1. Was tde patient's position specified on video?
	2. Was tde type of flexible ureterorenoscope specified on video?
	3. Was tde diameter of ureteral access sheatd specified on video?
	4. Was tde type of guide wire specified on video?
	5. Was tde diameter of guide wire specified on video?
	6. Was tde pressure of irrigation fluid or hand-pump use specified on video?
	7. Was tde type of laser energy utilized for stone fragmentation specified on video?
	8. Was tde type of stone fragmentation (dusting vs. active removal) specified on video?
	9. Were tde settings (power, frequency) of tde laser specified on video?
	10. Was tde usage or not usage of double-J stent specified on video?
C. After surgery
	1. Was tde hospitalization period or discharge time specified on video?
	2. Was tde postoperative course and possible postoperative complications specified on video?
	3. Was any radiological examination performed to assess postoperative stone-free status specified on video?

a:Yes=1 point and no=0 point.

All procedures were conducted in accordance with the Helsinki Declarations of 2004.

1. Statistical analysis

All statistical analyses were performed by use of SPSS 24.0 (IBM Corp., Armonk, NY, USA) software for Windows. Mann–Whitney U-test and Kruskal–Wallis test were used in the analysis. The median and interquartile range were used to define the parameters. Spearman correlation test was used to evaluate the correlation between variables. A p-value <0.05 was considered as the statistical significance level.

RESULTS

Of 63 videos evaluated, the majority of videos (n=43, 68.3%) were provided by health care professionals, followed by academic centers (n=16, 25.3%) and industry (n=2, 3.2%). Fifty-three videos (84.1%) included technical aspects, while 10 (15.9%) provided general information about RIRS. Several of the videos (n=8, 12.7%) were prepared for patients. The shortest video was 120 seconds and the longest was 4,800 seconds. The median number of views was 389 and the median time since the video was uploaded was 1,167 days. The median like and view ratios were 100 (interquartile range, 75–100) and 0.48 (interquartile range, 0.21–1.48), respectively. The median (interquartile range) GQS, JAMAS, RIRSSS, and VPI scores were 2 (1–3), 1 (1–2), 2 (1–5), and 0.41 (0.08–1.29), respectively. The basic data for the included videos are shown in Table 2.

Table 2

Basic characteristics of the included videos

Parameter		Value
Video language
	English	13 (20.6)
	Other	5 (7.9)
	No audio	45 (71.5)
Video source
	Academic center	16 (25.3)
	Urologist	43 (68.3)
	Commercial	2 (3.2)
	Other	2 (3.2)
Country of origin
	Europe	14 (22.2)
	Asia	44 (69.8)
	Africa	5 (8.0)
Video content
	General information	10 (15.9)
	Technical aspect	53 (84.1)
Target audience
	Physicians	55 (87.3)
	Patients	8 (12.7)
GQS		2 (1–3)
JAMAS		1 (1–2)
RIRSSS		2 (1–5)
Time since upload (d)		1,167 (285–2,103)
Thumbs up		3 (1–6)
Thumbs down		0 (0–1)
Length of video (s)		256 (184–458)
Number of views		389 (145–1,310)
Like ratio		100 (75–100)
View ratio		0.48 (0.21–1.48)
VPI		0.41 (0.08–1.29)

Valuesa are presented as number (%) or median (interquartile range).

GQS, Global Quality Score; JAMAS, Journal of the American Medical Association Benchmark Score; RIRSSS, Retrograde Intrarenal Surgery Scoring System; VPI, video power index.

The RIRSSS score and VPI were significantly higher in newer videos, whereas the JAMAS was higher in older videos (p=0.017, p=0.049, and p=0.007, respectively). Videos with audio had significantly higher GQS and RIRSSS scores than did videos with no audio (p<0.001, p=0.039, respectively). Videos in English had significantly higher GQS and RIRSSS scores than did other videos (p<0.001, p=0.027, respectively). The GQS of general information about RIRS videos was higher, while RIRSSS scores were higher for the videos about technical aspects (p=0.027, p=0.038, respectively). The JAMAS, GQS, RIRSSS, and VPI scores were similar for the videos uploaded from Europe, Asia, and Africa (p=0.096, p=0.864, p=0.696, and p=0.091, respectively) (Table 3).

Table 3

Comparison of scores according to the characteristics of videos

Parameter		JAMAS	p-value	GQS	p-value	RIRSSS	p-value	VPI	p-value
Upload year			0.007*		0.188		0.017*		0.049*
	2010–2015	2 (1–2)		2 (1–2)		1 (0.5–3)		0.19 (0.03–0.79)
	2016–2020	1 (0–1.25)		2 (1–3)		3 (1–7.25)		0.55 (0.20–1.63)
Video language			0.114		<0.001*		0.039*		0.831
	Yes	1 (0–2)		3 (1–2)		3.5 (1–4)		0.45 (0.07–1.33)
	No	1 (1–2)		1 (2.75–4)		2 (1–9.25)		0.39 (0.12–1.21)
English language			0.353		<0.001*		0.027*		0.812
	Yes	1 (1–2)		4 (2.5–4)		4 (1.5–9.5)		0.48 (0.08–1.24)
	No	1 (0–2)		1 (1–3)		2 (1–4)		0.40 (0.07–1.31)
Video source			0.346		0.240		0.174		0.799
	Academic	1 (1–1)		2.5 (1–3.75)		3.5 (1–7)		0.29 (0.01–1.19)
	Urologist	1 (0–2)		2 (1–3)		2 (1–4)		0.42 (0.15–1.29)
	Commercial	2 (2–2)		3 (2.5–3.5)		1 (0.5–1.5)		0.59 (0.29–0.89)
	Other	1 (1–1)		1 (1–1)		0.5 (0.25–0.75)		1.16 (0.45–1.88)
Country of origin			0.096		0.864		0.696		0.091
	Europe	2 (0.75–2)		2 (1–3)		2.5 (0.75–4.25)		0.21 (0–0.72)
	Asia	1 (0.25–2)		2 (1–3)		2 (1–5.5)		0.51 (0.16–1.49)
	Africa	1 (1–2)		2 (1–2.5)		3 (1.5–7)		0.16 (0–0.47)
Video content			0.629		0.027*		0.038*		0.534
	Technical aspects	1 (0.5–2)		2 (1–3)		2 (1–6.5)		0.41 (0.07–1.20)
	General information	1 (1–2)		3 (1.75–4)		1.5 (0–2.25)		0.73 (0.14–1.79)

Values are presented as median (interquartile range).

GQS, Global Quality Score; JAMAS, Journal of the American Medical Association Benchmark Score; RIRSSS, Retrograde Intrarenal Surgery Scoring System; VPI, video power index.

*p<0.05.

A positive correlation was found between VPI and GQS and JAMAS (r=0.284, p=0.024, and r=0.277, p=0.028, respectively). No statistically significant correlation was found between VPI and the time since the video was uploaded and video length (p=0.070, p=0.575, respectively).

No correlation could be detected between like and like ratio, and JAMA, GQS, and RIRSSS (p=0.116, p=0.185, p=0.973, p=0.851, p=0.674, and p=0.478, respectively). Similarly, when videos were evaluated in terms of source, content, audio, and language, the like and like ratio were similar between groups (p=0.481, p=0.106, p=0.531, p=0.683, p=0.842, p=0.875, p=0.543, and p=0.639, respectively).

DISCUSSION

One recent use of the internet is in the context of health care [23]. Various social media platforms are used for this purpose, one of which is YouTube. YouTube has been evaluated in the past as a source of information on immunization, neurosurgery, prostate cancer, kidney stones, and the H1N1 and coronavirus pandemics [242526272829]. To the best of our knowledge, however, no previous study has examined the quality and content of YouTube videos on the subject of RIRS. For this reason, we aimed to evaluate the quality of videos on YouTube, one of the most popular social media platforms, for RIRS.

In our study, we determined that the majority of videos were uploaded by health care professionals and included technical aspects of RIRS; the median VPI, GQS, JAMAS, and RIRSSS scores of the RIRS videos were 0.41, 2, 1, and 2, respectively. Only 12.7% of the videos were directed toward patients. In this era, social media is frequently being used by patients to gather information about a disease, diagnostic methods, and treatment options; the fact that so few videos were directed toward patients was considered as a major deficiency. These low scores indicate that the informative level of the RIRS videos available on YouTube was low and did not contain enough information. Similarly, studies evaluating videos related to other urologic diseases stated similar problems [1315]. Given that all those who are reaching this information are not health care professionals, these results suggest that viewers can obtain misleading information. This situation indicates that health care organizations should prepare and upload videos based on evidence-based information to enlighten society with accurate and sufficient information.

We found that videos with audio and videos in English had high GQS and RIRSSS. This finding revealed that videos with audio and videos in English were more efficacious and educational for both patients and physicians planning to perform this procedure. This situation revealed the educational effect of the auditory stimulus along with the visual stimulus. For this reason, we think that it is important to use English audio, which is the most widely spoken language in the world, to make higher-quality and VPI videos.

The GQS is a scale used to determine whether a publication is useful to patients or not [21]. Considering that patients can benefit from general information about an operation more than information about the technical aspects, the GQS was higher in videos containing general information about RIRS, as expected. Consistent with this, the RIRSSS score, which includes the stages of RIRS, was found to be higher in videos including technical aspects of the surgery.

In our study, the VPI and RIRSSS scores were determined to be higher for newly uploaded videos. Given the increased instrumentation capability in RIRS, it may be necessary to provide more technical information in the videos. This situation may explain the high RIRSSS score in newly uploaded videos. Similarly, people may have adjusted their video preferences for newly uploaded videos owing to the technological advances, and therefore the VPI score of these videos may have been higher.

The GQS and JAMAS for the three most-watched videos were higher than the median scores of the study. This suggests that video-makers should pay attention to these scoring systems during the video preparation phase to achieve a large number of views.

There were some potential limitations to this study. First, our sample size was quite small. However, given the strategies people use to search on the internet [30], we thought that 63 videos was a sufficient number for interpretation. Also, YouTube has a dynamic process and these video sequences may change over time. Second, different keywords can be used to search for the same subject. However, we thought that using different keywords, like “ureterorenoscopy,” would result in videos entering the playlist that did not match the purpose of this study. Therefore, the specific keyword “retrograde intrarenal surgery” was preferred, but creating a playlist by using a single keyword may be seen as another limitation of the present study. Last, video analysis made by two urologists can be considered as another limitation. However, we suggest that the extensive experience of these urologists in RIRS (>25 cases per year) eliminates this limitation. Besides these limitations, we believe that the results of our study will make a significant contribution to the current literature, as this is the first study evaluating the quality and accuracy of videos for RIRS on YouTube by use of validated scoring systems.

CONCLUSIONS

In the present study, we found that the videos for RIRS on YouTube are not suitable for patients to obtain proper information. Many videos contained low-quality content and provided misinformation. This reveals the importance of patient-physician communication. Videos with audio and videos in English were determined to be more effective. Today the use of social media is becoming more and more common; therefore, health care providers should prepare patient education materials that will provide correct information to patients about RIRS and upload these materials to the popular social media platforms.