YouTube as a platform to better understand the treatment of lymphoma using ibrutinib: a cross-sectional study.

Background: Ibrutinib, one Food and Drug Administration-approved, orally available, small-molecule Bruton tyrosine kinase (BTK) inhibitor, is an effective targeted therapy for patients with chronic lymphocytic leukemia (CLL) and mantle cell lymphoma (MCL). YouTube is increasingly used for health purposes. However, videos for ibrutinib on YouTube have not been previously evaluated. This study assessed the accuracy and quality of YouTube videos on ibrutinib, to better understand the information shown on a dominant media platform.
Methods: The first 150 video results returned by the YouTube search engine in response to the keyword "ibrutinib" were included (up to June 27, 2022). Typically used predefined inclusion and exclusion criteria were applied to screen the videos based on our needs. A 5-point Global Quality Scale (GQS) determined whether the videos would be useful to patients or not, and the quality of content was analyzed by five content-specific items. The quality of the included videos was classified as "low", "moderate", or "excellent" according to GQS and content score. The median and interquartile range were used to describe the values and Kruskal-Wallis test were used in the analysis.
Results: A total of 99 videos with a median of 237 views met the inclusion criteria. The videos were categorized into educational videos (n=6, 6.07%), personal experience and blog (n=3, 3.03%) and interviews videos (n=90, 90.9%). Almost half of the videos were classified as moderate (n=51, 51.51%), followed by excellent (n=25, 25.26%) and low (n=23, 23.23%). Between the groups, no statistically significant differences were observed in the numbers of dislikes, comments, posted days, percentage positivity and viewing rate (P>0.05). There were marked differences in the length, likes, views, viewers' interaction and likeability (P<0.05). Conclusions: YouTube could be an effective source for different groups of people to obtain helpful information about ibrutinib. The physicians, pharmacists, nurses and healthcare organizations should prepare and upload more comprehensible and reliable videos with evidence-based information. 2022 Annals of Translational Medicine. All rights reserved.

Introduction

Over the past decade, Ibrutinib has emerged as an encouraging therapy for patients with certain types of B cell malignancies. It has been shown to be particularly effective in the treatment of chronic lymphocytic leukemia (CLL) and mantle cell lymphoma (MCL) during early-stage clinical trials, which provides a strong rationale for subsequent phase 3 trials (1,2). In the phase 3 RESONATE-2 study, which involved a median follow-up of 5 years, more than half of the CLL/small lymphocytic lymphoma (SLL) patients could receive long-term first-line treatment with single-agent ibrutinib. This demonstrated sustained progression-free survival (PFS) and overall survival (OS) benefits versus chlorambucil as well as a growing depth of response over time (3). An international, randomized, open-label, phase 3 study reported a significant improvement in PFS and better tolerability versus temsirolimus in patients with relapsed or refractory mantle-cell lymphoma (4), lending further support to ibrutinib treatment.

Oral chemotherapy has been increasingly prescribed for cancer treatment owing to patient preferences, lower costs, proven efficacy, and the lack of infusion-related inconveniences (5). Notably, oral chemotherapy enables patients to reduce the number and duration of hospital admissions, so that their quality of life can be improved (6,7). Similar to the results observed in other fields, patient education about oral chemotherapy has demonstrated proven efficacy and effectiveness in supporting cancer patients (8,9).

With the development of the Internet, an increasing amount of health-related information is available to the public simply through an online search. For example, Facebook, Twitter, Blogs and YouTube can create a space to share, comment, and discuss health information on a diverse range of health issues. To determine the relative effectiveness of different social media applications for health communication, some methodology, like reviewing the current published literature or assessing the quality of available videos, has been used (10). YouTube, which was created in 2005, is the largest video-sharing website worldwide and is watched by over 2 billion users each day. At present, YouTube contains over 1 billion hours of video (11,12). However, as is well known, many YouTube videos are not subjected to peer-reviewing and lack quality control. Some data about health information has been deemed to be deceptive and inaccurate, which may result in the dissemination of inaccurate details and lead to severe health problems.

With the emergence of YouTube as a promising source of searching for and sharing health information with the public, the application of YouTube for health purposes has become a hot topic. Other studies have assessed videos for different diseases or drugs to empower patients to live their optimal lives by improving the management of drugs, better offsetting side effects, and ultimately reducing non-scheduled hospitalizations (13,14). Ibrutinib is the first Bruton tyrosine kinase (BTK) inhibitor on the market and a promising new drug that can be used orally long-term for the treatment of lymphoma, which has been a focus for our team. So far, ibrutinib has been approved in 100 countries and regions, benefiting more than 200,000 patients globally. Yet, a better understanding of ibrutinib can improve medication adherence and maximize therapeutic effectiveness. However, to our knowledge, YouTube videos on ibrutinib have not been previously evaluated. Hence, the present study aims to assess the content, reliability, and quality of the available videos on ibrutinib using validated questionnaires and a proper scoring system, to identify whether YouTube is a useful resource for people.

Methods

Search strategy

We performed a search on YouTube using the keyword “ibrutinib” on June 27, 2022, based on previous study (15). Relevance-based ranking was used to screen the related videos, which may be the most common option for searches (16). From the search results, the first 150 videos were considered, and all advertisements were neglected.

Criteria

The inclusion criteria were as follows: (I) videos available on June 27, 2022; (II) videos with ibrutinib-related content; and (III) English videos.

The exclusion criteria were as follows: (I) advertisements; (II) duplicate content; (III) videos without audio; (IV) non-relevant videos; (V) non-English videos.

Furthermore, videos containing multiple sub-parts were merged into one.

Video evaluation

Based on the inclusion/exclusion criteria, relevant videos were included in this study and saved in a playlist. For each video, the relevant data was recorded and calculated (17). All the videos were classified according to the following categories: educational, personal experience and blogs, and interviews. Moreover, based on the characteristics as well as the protagonist(s) in each video, we categorized the videos as academic conferences, physician remarks, popular science, or others.

Scoring system

Similar to other studies, each video was assessed using the Global Quality Scale (GQS), which was used to assess the educational value to confirm whether the videos would be useful to patients based on 5 criteria (i.e., poor quality, generally poor quality, moderate quality, good quality and excellent quality). For each of the 5 identifiable criteria present in a video, 1–5 point could be assigned (18,19). Five specific contents of the videos were systematically evaluated: mechanism, indications, effectiveness, adverse reactions, and prospects. For each of these elements, the videos were scored 1 point (briefly introduced) or 2 points (introduced in detail), while 0 points were given if the videos did not contain any of the abovementioned elements (19). The sum of the GQS score and content score could be 1–15, and relevant videos were divided into three qualitative groups based on their sum score: “low” [1-6], “moderate” [7-11], or “excellent” [12-15].

The included videos were analyzed and scored in routine practice by two independent researchers (Qiuling Zhao and Xiuliang Qiu), who are clinical pharmacists with a working knowledge of ibrutinib. If the GQS or content scores afforded by the two researchers differed by three or more points, the final scores were resolved by an arbitrator (Lin Yang) (20). In addition, the average scores from the two independent researchers were used for the final statistical analysis.

Assessment of reliability

The DISCERN tool was used to analyze the reliability of the videos as mentioned previously (21). This 5-point scale, which is based on five questions, is recognized as a reliable and valid health indicator. Each question is answered as yes (1 point) or no (0 points), for a total possible score of 5 points.

Statistical analysis

Descriptive analysis was conducted using IBM SPSS® statistical software (version 25.0, USA). The interclass correlation coefficient (ICC) and Cohen’s kappa coefficient (κ) were applied to assess the inter-observer agreement. Variables with skewed distribution were represented by the median and interquartile range (50th and 75th percentiles) (22). To test whether differences existed in the video characteristics between three groups (“low”, “moderate”, and “excellent” group), the Kruskal-Wallis test was used, and a two-tailed P value <0.05 was deemed statistically significant.

Results

As shown in , the first 150 video search results from YouTube were screened, and a total of 99 unique videos met the inclusion criteria. A more detailed overview of the videos is presented in , including the total number of views (88,931 times) and total duration (430.66 min). The median duration of the videos was 1.67 min. As for the sources of the videos, nearly all were provided by healthcare institutions (data not shown). The majority of videos were interviews (n=90, 90.9%), while the others were educational videos (n=6, 6.07%) and personal experiences and blogs (n=3, 3.03%). Based on the characteristics of videos, most videos were classified as academic conference (n=61, 61.62%), followed by physician remarks (n=30, 30.3%), popular science (n=4, 4.04%) and others (n=4, 4.04%). The interquartile ranges of the GQS, content, total, and DISCERN scores of the included videos were 3 [3-5], 6 [4-7], 9 [7-12], and 3 [2-3], respectively.

Figure 1

Details of videos included in the study.

Table 1

Summary of the included YouTube videos about ibrutinib

Values	Value (n=99)
Variables, median [interquartile range]
Length (min)	1.67 [1.13–3.33]
Likes	2 [1–5]
Dislikes	0
Comments	0
Views	237 [112–657]
Days posted	1,147 [517–1,815]
Likeability	0.0015 [0.0004–0.0058]
Percentage positivity	1 [0.5–1]
Viewing rate	0
Viewer’s interactions	0
GQS score	3 [3–5]
Content score	6 [4–7]
Total score	9 [7–12]
DISCERN score	3 [2–3]
Total duration (min)	430.66
Total views	88,931
Categories, n (%)
Educational videos	6 (6.07)
Personal experiences and blogs	3 (3.03)
Interviews	90 (90.9)
Characteristics, n (%)
Academic conferences	61 (61.62)
Physician remarks	30 (30.30)
Popular science	4 (4.04)
Others	4 (4.04)

GQS, Global Quality Scale.

The basic data for inter-rater reliability is shown in . The ICC and κ were used to evaluate inter-observer agreement, including the GQS, content, total, and DISCERN scores. There was no statistically significant correlation between different scores given by the two reviewers before the arbitrator intervened. The mean DISCERN value of the included videos was 2.81 [standard deviation (SD) =0.56], and the DISCERN score was positively related to three kinds of videos in (P<0.001).

Table 2

Inter-rater reliability for customized usefulness scoring scheme

Reliability parameters	GQS score	Content score	Total score	DISCERN
ICC	0.972	0.991	0.992	0.939
κ	0.929	0.929	0.875	0.920

ICC, interclass correlation coefficient; κ, Cohen’s kappa coefficient; GQS, Global Quality Scale.

Table 3

Detailed analysis of the video characteristics based on usefulness

Characteristics	Low (n=23, 23.23%)	Moderate (n=51, 51.51%)	Excellent (n=25, 25.26)	H	P value
Length (min)	1.67 [1.13–2.93]	3.00 [1.90–4.12]	4.68 [3.13–9.57]	27.532	<0.001*
Like	1 [0–2]	1 [1–5]	4 [1–9.5]	10.705	0.005*
Dislike	0	0	0	0.688	0.716
Comments	0	0	0 [0–0.5]	3.844	0.146
Views	166 [79–314]	235 [107–653]	818 [247.5–2,001.5]	11.601	0.003*
Days posted	1,179 [581–1,811]	1,108 [385–1,784]	1,147 [470–2,108]	0.284	0.867
Likeability	0 [0–0.0026]	0 [0.00046–0.0062]	0 [0.0013–0.009]	7.380	0.025*
Percentage positivity	1 [0–1]	1.00 [0.5–1.00]	1.00 [0.95–1.00]	1.652	0.440
Viewing rate	0	0	0	3.554	0.169
Viewer’s interactions	0	0	0 [0–1.05]	6.184	0.045*
DISCERN	3 [2–3]	3 [2–3]	3 [3–4]	21.401	<0.001*
Total views/day	0.17	0.82	1.06	–	–
Total duration (%) (min)	48.15 (11.18)	200.45 (46.56)	181.96 (42.26)	–	–
Total views (%)	4,846 (5.45)	49,227 (55.35)	34,858 (39.20)	–	–

Data are presented as median [interquartile range]. *, P<0.05. H, Kruskal-Wallis test.

The videos were further categorized as “low”, “moderate”, and “excellent” according to their total scores. The video statistics are shown in . Almost half of the videos were classified as moderate (n=51, 51.51%), followed by low (n=23, 23.23%) and excellent (n=25, 25.26%). There were no significant differences among videos regarding the number of dislikes, comments, posted days, percentage positivity and viewing rate. A statistically significant difference was detected in respect of the length, likes, views, viewers’ interaction and likeability among videos (P<0.05).

Details among videos according to the categories and characteristics are shown in . In terms of the categories, the interview videos comprised the majority of the excellent videos (19/25, 76%). In terms of the characteristics, the academic conferences and physician remarks videos made up most of the excellent videos (20/25, 80%).

Table 4

Detailed relations of videos according to categories and characteristics

Values	Low (n=23)	Moderate (n=51)	Excellent (n=25)	Total
Category, n (%)
Education	0	1 (16.67)	5 (83.33)	6
Personal experience and blog	0	2 (66.67)	1 (33.33)	3
Interview	23 (25.56)	48 (53.33)	19 (21.11)	90
Characteristics, n (%)
Academic conferences	15 (24.59)	36 (59.02)	10 (16.39)	61
Physician remarks	7 (23.33)	13 (43.34)	10 (33.33)	30
Popular science	0	1 (25.00)	3 (75.00)	4
Others	1 (25.00)	1 (25.00)	2 (50.00)	4

Discussion

Given its popularity, YouTube has been previously assessed as a source of health information on drug poisoning, immunization, smoking cessation, first aid for burns, medication use in pregnancy, and the corona virus disease 2019 (COVID-19) pandemic (22-27). Like other video-sharing platforms, YouTube allows every registered user to upload and share health-related videos freely and openly. However, to the best of our knowledge, the content and quality of YouTube videos on ibrutinib have not been examined by previous studies. Targeting B-cell receptor (BCR) signaling with a BTK inhibitor (ibrutinib) is considered a highly active therapeutic strategy for patients with selected B cell malignancies, with the majority of promising data currently in patients with CLL and MCL (28,29).

This study aimed to better understand which information regarding ibrutinib is available to people. We evaluated 99 videos with a cumulative duration of 430.66 min and 88,931 views, which indicates there are many people who are interested in ibrutinib on YouTube. We also found that nearly all of the videos were uploaded and endorsed by global and national professional healthcare organizations (e.g., American Society of Hematology, American Society of Clinical Oncology, MD Anderson Cancer Center, etc.) and included aspects of ibrutinib clinical trials. Physicians have played a vital role in expounding views on ibrutinib; not surprisingly, the indications, effectiveness, and adverse reactions of ibrutinib were mostly elaborated, while the mechanism and prospects were less discussed.

Of the 99 videos evaluated in this study, nearly half of the videos were of moderate quality, suggesting that YouTube can be considered an ideal and attainable source of helpful ibrutinib information. The lower percentage of excellent quality videos in this study is similar to the findings of previous studies (14,30). In our study, about a quarter of videos obtained a lower score due to having few content details. Interestingly, the excellent videos had significantly higher likes, likeability, views and viewer’s interactions than those of the low and moderate videos, possibly because these videos provided considerably more scientific and valid information that met the viewers’ criteria, leading to a greater number of people seeking to view them. The higher views and likes of the excellent videos, in turn, confirmed their high quality, as was reported by other studies (15,31). Also, videos in the excellent group had significantly higher durations than those in the other groups. This may be because the excellent videos usually contain more content or longer discussions than others. We found that audiences largely did not represent their dislike or comment after watching the videos. Furthermore, the median time since the videos were uploaded was 1,147 days, suggesting that most of the relevant videos were not up to date. Hence, no statistically significant correlation was identified in the number of dislikes, comments, and days posted between the different kinds of videos, suggesting that the users could not easily judge the quality of videos. Put simply, the number of dislikes, comments, and days posted are not useful factors for evaluating the quality of videos. The overall quality of each video was assessed by GQS, which is a scale applied to assess the educational value, to confirm whether the videos would be useful to patients. Considering that patients can obtain greater benefit from excellent videos about ibrutinib than from low and moderate videos, the GQS score was higher in excellent videos about ibrutinib, as expected. Consistent with this, the DISCERN score in excellent videos was found to be higher than those in the low and moderate videos.

Moreover, 90.9% of the videos included in our study were interviews, which typically involved discussions regarding the indications, effectiveness, adverse reactions, and prospects of ibrutinib. Personal experience and blog videos mainly involved discussion about the mechanism, indications, effectiveness, and/or treatment-related personal experiences of ibrutinib, implying that individuals can use the Internet to seek medication information and propagate opinions. Educational videos mainly utilized animations to report on the mechanism, indications, effectiveness, and prospects of ibrutinib, which attracted a greater number of potential viewers. Furthermore, similar to the findings of previous studies evaluating total knee arthroplasty or nursing professionals during the COVID-19 pandemic on YouTube, educational videos had longer durations and higher scores than other videos (32,33). Notably, none of the personal experience/blog and educational videos included in this study were considered low quality.

In terms of characteristics, academic conferences and physician remarks videos made up most of the excellent videos (20/25, 80%). Meanwhile, our study revealed that these videos also comprised the majority of low-quality videos. In this era, social media websites are frequently being used by patients to obtain information about diseases, drugs, and treatment options. However, the fact that only a few high-quality videos were patient-oriented is regarded as a major deficiency. Similarly, studies evaluating videos related to patient education for treatments on psoriasis or lower limb amputations showed similar problems (34,35). Given that not all viewers are medical professionals, these low-quality videos may impart misleading information. Also, it is easier for health-literate individuals to understand some professional videos, whereas educational videos would be more suitable for people without such knowledge. Unfortunately, the percentage of educational videos on ibrutinib was very low (6.07%).

According to previous studies (23,30,36), at least two aspects need to be taken into account when analyzing YouTube videos: first, video-related data, such as the length, as well as the numbers of likes, dislikes, views, and comments, are factors that may be related to video quality; second, multiple-word searches can generate more meaningful results and larger variability over time.

There were some potential limitations in this study that should be noted. Firstly, our sample size was relatively small. However, given the strategies used by people to search on YouTube (37), we thought that a sample of 99 videos was sufficient for interpretation. Moreover, since some videos had zero dislikes and comments, the likeability, viewing rates, and viewer interactions were not calculated. Also, the search results on YouTube are dynamic and may change over time. So, our study only demonstrates the results for ibrutinib at the time that the study was conducted. Secondly, different keywords can be used to search for ibrutinib. However, we thought that using different keywords, such as “Imbruvica”, which was known by fewer people, might result in a greater number of videos that do not satisfy the study objective being included in the playlist. Therefore, the specific keyword search using “ibrutinib” was preferred. Lastly, nearly all of the included videos were published by healthcare institutions, which resulted in a deficiency of diversity in the video sources and may have led to differences when evaluating the videos.

Yet, despite these limitations, we believe that our study findings offer a significant contribution to the existing literature as this is the first study evaluating the quality and accuracy of ibrutinib videos on YouTube. Furthermore, oncological pharmacists can upload high-quality videos, especially educational videos, providing information about oral chemotherapy (such as efficacy, adverse reaction, patient self-monitoring, etc.), and paying greater attention to the target audiences and the scoring systems during the video preparation phase, so a larger number of viewers can benefit from videos in the future.

Conclusions

Undoubtedly, there is an increasing trend toward the use of YouTube for health purposes. In addition, we have noticed the large-scale entry of health care institutions into the field of healthcare. However, some videos on YouTube contain low-quality content and are not up-to-date, which reveals the importance of patient-physician communication and suggests that healthcare organizations should prepare and upload comprehensive and reliable videos with evidence-based information. In this study, we found that the ibrutinib videos on YouTube are suitable for different groups of people to obtain helpful information, including its mechanism, indications, effectiveness, adverse reactions, and prospects. Put simply, YouTube has immense potential as an effective source of available information about ibrutinib.

The article’s supplementary files as