Are YouTube videos enough to learn anakinra self-injection?

Anakinra is a drug that can be administered subcutaneously as a self-injection in both children and adults. We aimed to evaluate the content, reliability, and quality of the videos most viewed on "anakinra self-injection" on YouTube, which is an easily accessible source of information. We performed a YouTube search using the keywords "anakinra", "anakinra injection", and "anakinra self-injection" in addition to the generic and commercial names of the biologic agent in September 2021. The quality and reliability of the videos were assessed according to the Global Quality Score (GQS) and DISCERN score. Video power index was used to assess both the view and the like ratio of the videos. A total of 51 videos were analyzed, a majority of which were uploaded by physicians (56.9%). The median GQS was 3 and total DISCERN score was 49. According to the GQS, 21.6% of the videos were of low quality, 35.3% were of fair quality, and 43.1% were of high quality. High-quality videos had higher DISCERN scores and longer duration of videos (p < 0.05). 41 (80.4%) videos were categorized as useful information, and 8 (15.7%) as useful as per patients' opinion. Further, GQS and DISCERN scores of videos that had useful information were significantly higher. There are numerous YouTube videos with helpful information that can be a source of knowledge on the safe and correct technique of daily anakinra self-injection for both adults and children. There was no significant difference in patient interaction between useful and misleading videos. This indicates that patients do not differentiate between high- and low-quality videos.

Introduction

Anakinra is a human recombinant interleukin (IL)-1 receptor antagonist and the scope of its usage is increasing gradually. It is routinely used in patients with colchicine-resistant idiopathic recurrent pericarditis [1]; undefined autoinflammatory diseases [2]; gout flares, especially in patients who are unresponsive to conventional therapy and/or have contraindications with nonsteroidal anti-inflammatory drugs, glucocorticoids, and colchicine [3]; and colchicine-resistant familial Mediterranean fever [4]. It has also been approved by the United States Food and Drug Administration for moderate to severe rheumatoid arthritis (RA) and cryopyrin-associated periodic fever syndrome (5). The European Medicines Agency has approved the use of anakinra for these indications and Still’s disease [both adult-onset Still’s disease (AOSD) and systemic juvenile idiopathic arthritis (sJIA)] [5]. Recently, anakinra has also been used in cases of severe coronavirus disease 2019 (COVID-19) and acute respiratory distress syndrome caused by COVID-19 [6, 7].

Anakinra is administered daily as a subcutaneous injection and can be self-administered by patients. The advantages of subcutaneous administration are that the patients can decide where and when to take their medication and don't have to go to a health center daily. However, before self-administration, patients should receive appropriate self-injection training from a healthcare professional to learn the method of administration.

With the use of the internet in the last 20 years, it has become easier for patients to access information about their diseases, and the internet has become a popular research tool [8]. In recent years, with its rich video content, YouTube is generally preferred by everyone as an educational tool and information resource. YouTube is one of the most used online resources to obtain health information [9], and nearly three-quarters of the patients are reported to be influenced by information available from online searches on treatment modalities [10]. However, patients also worry about the reliability of this information .

In previous studies, the quality of anti-tumor necrosis factor (anti-TNF) and methotrexate (MTX) subcutaneous self-injection videos, which required practical application, were evaluated [11, 12]. Anakinra is a drug that can be administered subcutaneously as a self-injection in both children and adults, and there is no study evaluating anakinra self-injection videos on YouTube in the literature to the best of our knowladge. . Hence, we aimed to evaluate the content, reliability, and quality of the most viewed videos   of “anakinra self-injection” on YouTube, which is an easily accessible source of information.

Materials and methods

Videos on YouTube (http://www.youtube.com) were searched using the keyword “anakinra”, “anakinra injection”, and “anakinra self-injection” on September 8, 2021. The browser search history was deleted before the study to minimize the effect of past internet use on search results. Video listings were made based on view counts which enables the most viewed videos to be listed on the first page. A total of 85 videos were listed according to “most viewed”. We assessed all videos, and the 85 videos were recorded in a file for future analysis because search results on YouTube constantly change. A total of 51 videos were included in the study after removing duplicate (14 videos), non-English (13 videos), and irrelevant (7 videos) videos found during the YouTube search.

Video features, quality, reliability, and usability analysis

The videos were viewed and analyzed independently by a rheumatologist (MP) and a rheumatology fellow (TID). The properties of the YouTube videos were recorded: titles of videos, length of each video, number of views, number of comments, number of likes and dislikes, duration, upload date, and number of days since upload. Video popularity was evaluated by calculating the like ratio (likes / [likes + dislikes] × 100), view ratio (views/day), and video power index (VPI; like ratio × view ratio/100) [13]. The profiles of the uploaded resources were recorded and divided into five categories: physician, non-physician health personnel, health-related websites, medical company, and patient.

Evaluation of usefulness

Useful information (Group 1): These are videos that contain useful and accurate information. The video showed how to use an anakinra syringe and taught self-injection.

Misleading information (Group 2): These are videos that contain false information or do not contain information on how to use and self-administer an anakinra injection.

Useful patient opinion (Group 3): These videos contain a patient's current or past personal experiences and/or feelings regarding the use of an anakinra syringe. The video showed how to use an anakinra syringe and taught self-injection.

Misleading patient opinion (Group 4): Videos that contain false information from a patient or do not contain information on how to use and self-administer an anakinra injection.

Global Quality Scale (GQS) was used for the quality analysis. It is a five-point (1–5) scale that measures the quality, flow, and usefulness of a video. Accordingly, four or five points indicate high quality, three points indicate medium quality, and one or two points indicate low quality [14]. GQS is shown in Supplementary Table S1.

The DISCERN instrument was developed to enable patients and information providers to judge the quality of information. It consists of 15 questions plus an overall quality rating: section 1 : It is composed of three sections evaluating reliability and has eight questions, section 2 : gives quality of information about treatment options and has seven questions, and section 3 : gives general quality of the information and includes an overall rating. Each question was scored on a 5-point (1–5) scale. If the quality criterion was completely fulfilled, a score of 5 was assigned, and if it was not fulfilled at all, a score of 1 was assigned [15]. The total DISCERN score was calculated by summing up the scores for the 15 questions [16]. Accordingly, it can be categorized as very poor (< 27), poor (27–38), fair (39–50), good (51–62), and excellent (63–75) [17, 18].

Statistical analysis

The Statistical Package for the Social Sciences 22 was used for the analysis of the data. Shapiro–Wilk test was performed to test the normality of data. Frequency, median, minimum, and maximum were used as descriptive methods. Kruskal–Wallis test was used to determine statistically significant differences between more than two groups of an independent variable. The Spearman test was performed for correlation analysis. The inter-rater agreement was assessed with the kappa coefficient. The results were evaluated at a 95% confidence interval and a significance level of p < 0.05.

Results

A total of 51 videos were analyzed. Video characteristics were summarized in Table 1. Thirteen of the videos' titles are 'how to use', 9 of RA, 8 of COVID-19, 7 of AOSD, 4 of periodic fever syndromes, 3 of hidradenitis suppurativa, 3 of gout, 2 of biological agents, 1 of sJIA, and 1 of uveitis.

Table 1

Characteristics and quality assessments of YouTube videos

Source of upload	n	%
Physician	29	56.9
Patient	7	13.7
Medical company	6	11.8
Non-physician health personnel	5	9.8
Health-related websites	4	7.8

GQS Global Quality Scale, VPI video power index

A majority of the videos (56.9%) were uploaded by physicians. The median GQS score was 3.0 and DISCERN total score was 49.0. “According to DISCERN classification 9.8% were “very poor”, 13.7% were “poor”, 35.3% were “fair”, 37.3% were “good” and 3.9% were “excellent”. According to GQS score, 21.6% were low quality, 35.3% were fair quality, and 43.1% were high quality. The Cohen kappa score was calculated at 0.897 for the GQS score and 0.836 for the DISCERN total.

Among the total 51 videos, 41 (80.4%) were categorized as useful information, 1 (2.0%) as misleading information, 8 (15.7%) as useful patient opinion, and 1 (2.0%) as misleading patient opinion. Of the videos, 100% (n = 5) produced by non-physician health personnel, 96.6% (n = 28) produced by physicians and 66.7 (n = 4) produced by medical company were of useful information. GQS scores and DISCERN scores were significantly higher in videos which had useful information (Table 2).

Table 2

Video quality assessments according to usefulness category

	Useful information	Misleading İnformation	Useful patient opinion	Misleading patient opinion	p*
Video source
Physician, n (%)	28 (96.6)	0 (0)	0 (0)	1 (3.4)	0.005
Patient, n (%)	2 (28.6)	1 (14.3)	4 (57.1)	0 (0)
Medical Company, n (%)	4 (66.7)	0 (0)	2 (33.3)	0 (0)
Non-physician health personnel, n (%)	5 (100.0)	0 (0)	0 (0)	0 (0)
Health-related websites, n (%)	2 (50.0)	0 (0)	2 (50.0)	0 (0)
Total, n (%)	41 (100.0)	1 (100.0)	8 (100.0)	1 (100.0)
Video features
Duration (sec)	966 (84–6680)	718 (718–718)	467 (178–2904)	78 (78–78)	0.207
Time since upload (day)	935 (136–3588)	2100 (2100–2100)	828 (177–2175)	1236 (1236–1236)	0.675
Number of views	1571 (216–264394)	2318 (2318–2318)	685 (228–56963)	733 (733–733)	0.836
View ratio	2.4 (0.1–188.6)	1.1 (1.1–1.1)	1.8 (0.1–28.4)	0.6 (0.6–0.6)	0.652
Number of likes	15 (0–1600)	33 (33–33)	9.5 (1–438)	8 (8–8)	0.768
Number of dislikes	0 (0–176)	8 (8–8)	0.5 (0–47)	1 (1–1)	0.568
Like ratio	100 (80–100)	80 (80–80)	99.4 (88.9–100)	88.9 (88.9–88.9)	0.185
VPI	2.4 (0.1–169.9)	0.9 (0.9–0.9)	1.7 (0.1–25.7)	0.5 (0.5–0.5)	0.631
GQS score	4 (2–5)	2 (2–2)	2.5 (1–3)	2 (2–2)	0.004
DISCERN reliability	28 (11–38)	11 (11–11)	19 (10–27)	13 (13–13)	0.012
DISCERN treatment	24 (10–34)	13 (13–13)	16.5 (12–23)	16 (16–16)	0.040
DISCERN quality	3 (1–5)	2 (2–2)	2 (1–3)	1 (1–1)	< 0.001
DISCERN total	54 (24–76)	26 (26–26)	38.5 (23–49)	30 (30–30)	0.008

Results are presented as median (min–max)

GQS Global Quality Scale, VPI Video Power Index

*Kruskal–Wallis test

GQS and DISCERN quality scores were found to be significantly higher in the physicians’ group than in the patient group when the video features were compared according to the video source (4 (2–5) vs. 2 (1–3) for GOS score and 4 (1–5) vs. 2 (1–4) for DISCERN quality score, respectively). No statistical significance was found in terms of other features of the videos (p > 0.05).

Significant differences were detected between the quality groups in terms of DISCERN scores and duration (sec) (p < 0.05 and p = 0.031, respectively). The highest scores were found in the high-quality group (Table 3).

Table 3

Video features according to the video quality

	Low quality	Fair quality	High quality	p*
Duration (sec)	492 (78–1836)	935 (84–3852)	1327 (207–6680)	0.031
Time since upload (day)	1236 (308–3588)	712.5 (136–3222)	1070.5 (248–3520)	0.246
Number of views	964 (277–6978)	1179 (216–198679)	1604 (271–264394)	0.930
View ratio	1.2 (0.1–17.2)	3 (0.2–123.2)	1.9 (0.1–188.6)	0.402
Number of likes	13 (1–155)	25 (2–1400)	14 (0–1600)	0.448
Number of dislikes	1 (0–8)	0.5 (0–79)	0 (0–176)	0.632
Like ratio	94.3 (80–100)	98.6 (88.6–100)	100 (83–100)	0.405
VPI	1.2 (0.1–16.3)	2.8 (0.2–116.7)	1.8 (0.1–169.9)	0.353
DISCERN reliability	17 (10–27)	26 (013–33)	33 (11–38)	< 0.001
DISCERN treatment	13 (12–26)	18.5 (12–28)	29 (10–34)	0.001
DISCERN quality	2 (1–4)	3 (2–4)	4 (2–5)	< 0.001
DISCERN total	33 (23–51)	48 (30–65)	65 (24–76)	< 0.001

Results are presented as median (min–max)

GQS Global Quality Scale, VPI Video Power Index

*Kruskal–Wallis test

We detected a positive correlation between GQS score and DISCERN scores. Correlation analyses for GQS score and DISCERN scores were presented in Table 4.

Table 4

Correlation analyses for DISCERN scores and GQS score

	GQS		DISCERN reliability		DISCERN treatment		DISCERN quality		DISCERN total
	r*	p	r*	p	r*	p	r*	p	r*	p
GQS	–	–	0.622	< 0.001	0.575	< 0.001	0.652	< 0.001	0.633	< 0.001
DISCERN reliability	0.622	< 0.001	–	–	0.798	< 0.001	0.690	< 0.001	0.946	< 0.001
DISCERN treatment	0.575	< 0.001	0.798	< 0.001	–	–	0.740	< 0.001	0.940	< 0.001
DISCERN quality	0.652	< 0.001	0.690	< 0.001	0.740	< 0.001	–	–	0.775	< 0.001
DISCERN total	0.633	< 0.001	0.946	< 0.001	0.940	< 0.001	0.775	< 0.001	–	–
Duration (s)	0.389	0.005	0.015	0.918	0.114	0.427	0.259	0.066	0.064	0.655
Number of views	0.010	0.944	0.209	0.141	0.383	0.006	0.266	0.059	0.323	0.021
View ratio	0.097	0.497	0.294	0.036	0.466	0.001	0.355	0.011	0.399	0.004
Number of likes	0.003	0.982	0.156	0.274	0.373	0.007	0.330	0.018	0.293	0.037
Number of comments	0.091	0.525	0.114	0.424	0.302	0.031	0.324	0.020	0.242	0.087
VPI	0.084	0.567	0.283	0.049	0.455	0.001	0.370	0.009	0.387	0.006

GQS Global quality scale, VPI Video Power Index

*Spearman p correlation coefficient

Discussion

The internet contributes significantly to the way people communicate and obtain information. It has been reported that people turn to the internet as the first source of health information, even before obtaining a doctor’s recommendation, regarding medical issues. [19]. With open access and a user-friendly interface, YouTube, while providing free content, may present false information, owing to the lack of an evaluation process in terms of accuracy and timeliness. In addition to various medical branches, the quality and content of YouTube videos for various rheumatological diseases have been examined, and apart from the useful information about rheumatological diseases [20, 21], it was emphasized that misinformation was also present on YouTube [12, 13]. To the best of our knowledge, this study is the first to evaluate YouTube videos on “anakinra self-injection” using validated tools. The videos containing information about “anakinra self-injection” on YouTube were mostly uploaded by physicians. The vast majority of the videos had fair to high quality and reliability. High-quality videos had higher DISCERN scores and longer duration of videos. And also, while the majority of the videos contained useful information, the GQS and DISCERN scores of the videos containing useful information were significantly higher.

In several studies, video sources on YouTube have been classified in different ways. In a study evaluating Botox videos, it was observed that the videos uploaded by health professionals were at the rate of 43% [22], while in another study evaluating fibromyalgia videos, it was reported that the videos were mostly uploaded by physicians (28%) [13]. Similarly, in a study evaluating the quality and validity of spondyloarthritis videos, 62% of the videos were uploaded by healthcare professionals [20]. In another study, YouTube videos about the side effects of biological drugs were evaluated, and it was emphasized that 33.8% of the videos were uploaded by professional organizations and physician groups [23]. Similarly, in this study, 56.9% of the anakinra self-injection videos were uploaded by physicians. Unlike our findings, in a study evaluating Sjögren’s syndrome videos, it was stated that 51.4% of the videos were from independent users [24]. In addition, in studies,MTX and anti-TNF self-injection methods were evaluated, 52.9% and 79.6% of the videos, respectively, were uploaded by patients [11, 12]. It can be thought that this difference may be due to the differences in the usage areas of the drugs, subjective evaluations, and publications from different geographical regions that are conducted at different times.

Previously, many studies have been conducted to evaluate  the quality of open-access videos about health information in different branches. In the study of Zengin et al. in which they evaluated the videos on the side effects of biological agents, it was reported that 40.3% of the videos were high quality, 23.4% of intermediate quality and 36.4% of low quality [23]. In another study, 34% of the videos about secukinumab were reported to be of high quality, 32% of intermediate quality, and 34% of low quality [26]. Similarly, in this study, 43.1% of the videos were high quality and 35.3% of fair quality. The median total DISCERN scores and GQS were 49 (23–76) and 3 (1–5), respectively. In contrast, a study conducted by Duran et al. in 2021, videos on testicular cancer were evaluated and 63.2% of the videos were reported to be poor, 26.9% fair, and 9.9% good quality [25]. It was thought that these differences between studies could be due to various video sort . These findings show that the reliability and quality of YouTube videos about anakinra self-injection are satisfactory.

Videos on health information can be uploaded by a wide variety of sources, and in addition to videos containing useful information, videos containing misleading information can also be found. In this study, useful videos made up 80.4% of the total videos regardless of the source. This rate was similar to that reported in previous studies exploring information on RA and anti-TNF self-injection [11, 27]. On the contrary, in a study in which MTX self-injection videos were evaluated, only 19.6% of the videos were found to be useful [12]. These results show that most of YouTube videos about anakinra self-injections are considered useful and of good quality, containing accurate and unbiased information, and can therefore be used to inform patients about the safe and correct technique of anakinra self-injection. While GQS and DISCERN scores are higher in useful videos, there was no significant difference in viewer interaction parameters (number of views and likes per day) for useful videos and misleading videos. Similarly, 51% of the videos about Sjögren’s syndrome and 54.9% of the RA videos were found to be beneficial, albeit with no significant difference in audience interaction between the useful and misleading groups in both studies [24, 27]. This may suggest that the viewers did not have a preference for watching particular types of videos and had difficulty in distinguishing between useful and useless information. This result shows that patients can potentially watch lower-quality videos with false information. As an alternative solution to limit the dissemination of misleading information, physicians or healthcare organizations should upload more medical material containing reliable and accurate information.

For patients to have access to accurate information, it is necessary to ensure that they have access to useful and quality videos. In this study, useful videos were found to have high reliability, comprehensiveness, and quality. In a study in which anti-TNF self-injection training videos were examined, it was stated that 62.5% of the useful videos were from universities/professional organizations/non-profit physician/physician groups and their GQS was 5 (5–5) [11]. Further, 80% of the videos containing useful information about MTX self-injection were uploaded by universities/ professional organizations/non-profit physician/physician groups and their mean (SD) GQS was 4.2 (1.0) [12]. Similar to the literature, physicians were found to be the main source of useful videos in this study, and the GQS and total DISCERN score of useful videos were 4 (2–5) and 54 (24–76), respectively In addition, 2% of the videos in this study were misleading, which was slightly less than the educational videos about other diseases on YouTube. In the literature, 30% of the RA information-containing videos, 27.5% of the MTX self-injection training videos, and 4% of the Botox videos for the treatment of facial wrinkles were misleading [12, 22, 27]. It is thought that these differences between the studies may be due to subjective evaluation of the studies and the fact that these studies were on different subjects and sources.

This study has some limitations such as evaluating only English videos on YouTube. Some other limitations were as follows: subjective evaluation despite the use of validated tools, lack of information about the characteristics of the audience/viewers, and the fact that the study was conducted according to the YouTube settings, which may vary depending on the geographic region and time period.

In conclusion, there are numerous YouTube videos with helpful information that can be a source of information on the safe and correct technique of daily anakinra self-injection used by both adults and children. There was no significant difference in patient interaction between useful and misleading videos. This indicates that patients do not differentiate between high- and low-quality videos. Although internet is a supplementary educational tool for patients, to help ensure the best treatment outcomes, patients should be guided by doctors to sources that provide reliable and accurate information.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 12 kb)