Evaluation of YouTube videos as sources of information about complex regional pain syndrome.

Background: As the internet usage becomes easily accessible, the patients are more frequently searching about diseases and medical/non-medical treatments. Considering that complex regional pain syndrome (CRPS) is a debilitating disease, it is important to check the information that patients are accessing. Therefore, this study aimed to investigate the reliability, sufficiency, and accuracy of the YouTube videos about CRPS.
Methods: This study is a descriptive research which is derived by searching videos using the keyword 'complex regional pain syndrome' on YouTube. Relevance-based sequencing was used to sort the videos. Sources and video parameters were documented. To evaluate the accuracy, reliability and content quality of the videos, Global Quality Score, Journal of American Medical Association Benchmark Criteria and Modified DISCERN Questionnaire scales were used.
Results: A total of 167 videos were included in this study. The majority of the videos originated from USA (80.2%, n = 134). The median number of views was 639 and the viewing rate was 73.3. Most of the videos had partially sufficient data and the interaction index viewing rate parameters for videos with high content quality were greater than videos with low content quality (P = 0.010, P = 0.014). Conclusions: Our results showed that videos about CRPS on YouTube mostly had partially sufficient data and include intermediate-high quality contents. Moreover, high-content quality videos had higher viewing rates, interaction indexes, number of likes, longer durations, as well as better reliability and accuracy scores. Videos with high quality and reliable content are needed to reduce misinformation about CRPS.

INTRODUCTION

Complex regional pain syndrome (CRPS) is a disorder characterized by chronic, progressive regional pain that is usually caused by a traumatic injury like fractures, surgeries, sprains etc. [1]. In addition to pain, symptoms like swelling, skin color and temperature change, joint tenderness or stiffness, changes in nails and hair, reduced strength and altered motor functions, and hypersensitivity to even light touch can be seen at the affected limb [2]. Although the research on CRPS is increasing, the exact etiology and pathophysiology of the disease is not clearly defined yet [1]. Its pathophysiology is accepted as a combined pattern of various factors that arise after a trauma, consisting of autonomic dysregulation, nervous system alteration and sensitization, and inflammatory reactions [3,4]. The diagnosis depends on the patient’s history and physical examination [5].

CRPS is a quite a rare disease and it is divided into 2 types, namely CRPS Type 1 (previously called as reflex sympathetic dystrophy) which occurs after trauma but is characterized by absence of nerve damage, and CRPS Type 2 (previously called as causalgia) which is caused by a nerve injury [6-8]. It more commonly affects females, and the peak age of onset is found to be 45–55 years [9]. Despite the disease’s rarity, due to its debilitating course, it negatively affects patients’ quality of life [10]. According to a study conducted in CRPS patients, the relationship between pain intensity and negative mood states in patients with suicidal thoughts was found to be significant [11]. Considering the difficulty of treating CRPS, management should be multidisciplinary, including physical and occupational therapy; psychological therapies; medical treatment such as anti-inflammatory drugs, ketamine infusion, neuropathic pain medications; and, as an invasive approach, spinal cord stimulation [12,13]. Due to its complexity CRPS is found to have more reducing effects on quality of life than other chronic pain conditions [10].

Contemporarily as internet usage increases and becomes easily accessible, the patients are more frequently searching for information about diseases and medical or non-medical treatments. Since CRPS is not commonly seen in medical practices and is difficult to treat, the patients with CRPS can try to reach information on the internet. However, due to excessive amount of information on the internet, the patients may not access accurate knowledge about any kind of health issue. This can increase patients’ anxiety and expectations, and can lead them to pursue false medical or non-medical services.

Currently, YouTube is the second most commonly accessed website in the world [14]. The website involves plenty of videos about wide range of subjects. It is one of the most popular websites which is used to access medical information. Consequently, since there is no mechanism for controlling the videos’ contents and determining their accuracy, patients may access false information. Therefore, to be able to estimate the reliability of the videos about medicine, it is important to investigate their accuracy. However, in medical literature there is no study analyzing YouTube videos as a resource for information about CRPS. Considering that CRPS is a debilitating and very painful disease, it is important to check the information that patients are accessing.

Therefore, this study aimed to examine the reliability, sufficiency, and accuracy of the YouTube videos related to CRPS in order to have an idea about the accuracy of the information obtained by the patients, since it is easy to access information at any content quality level through videos on the internet.

MATERIALS AND METHODS

1. Study design and ethics

This study is descriptive research accomplished by searching on the internet without any participation of humans or experimental animals. Therefore, this study does not require ethical committee approval since the investigated data are publicly accessible.

2. Selection of videos

On May 5th, 2021, videos were searched using the keyword ‘complex regional pain syndrome’ on YouTube. Relevance-based sequencing was used to sort the videos. Since people tend to mostly click on videos on the first pages, within the total of 672 results, the first 300 videos were examined. Each video was evaluated by the first author for inclusion or exclusion, and included videos were added to a list on YouTube to be shared with the other two researchers who were examining patients with chronic pain in physical medicine and rehabilitation department. Videos were excluded if they were (1) in a language other than English, (2) videos without audio, (3) irrelevant videos, (4) duplicate videos, (5) advertisements, or were (6) videos without any information about CRPS which were only about a patient’s progression. Clearly understandable videos in English, lectures and conference videos, and videos including patients’ own experiences and knowledge were included.

3. Video parameters

Videos were evaluated by checking the following parameters: (1) country of origin, (2) number of views, (3) image quality: poor-good-high definition (which is determined by the videos features shown on website itself), (4) number of likes (which is an option shown by a thumbs up sign on the website for those who like the videos), (5) number of dislikes (which is an option shown by a thumbs down sign on the website for those who did not like the videos), (6) number of comments (which is the part of YouTube where you can share your opinions about the videos), (7) number of days online (the number of days from the date the videos were uploaded on YouTube to the date the evaluation was made for this study), (8) video duration (seconds), (9) viewing rate, calculated as the number of views/number of days since upload × 100%, and (10) interaction index calculated as the number of likes and dislikes/total number of views × 100%. The sources uploading the videos are grouped in 5 categories as: professional health organization, physician/therapist, health related website/channel, patient, and YouTube channel/TV channel.

4. Video evaluation

The total of 167 videos were evaluated by the first researcher. After sharing the link for the YouTube list of videos with the other two researchers, they rated the videos with the same ranking design. In order to have an idea of whether the information in the videos can help viewers, and to determine whether this information is reliable, accurate, and provides a better education to patients on the subject, the following scales were used: the Global Quality Score (GQS) and Journal of the American Medical Association (JAMA) Benchmark Criteria, and the Modified DISCERN Questionnaire. In cases where there was a great difference in scores between researchers, a final decision was made by consensus. The sufficiency of CRPS-related information in the videos were determined by the scores of the videos on these scales.

1) GQS

To determine the content quality of the videos including the videos’ flow and the correctness of their contents, the GQS scale, arranged between 1 and 5 points from low quality to high quality, was used. In the GQS, the information is classified as follows: 1) poor quality, poor flow, and most information is missing, so that it is not helpful for patients; 2) generally poor, with some information given but of limited use to patients; 3) moderate quality and some important information is adequately discussed; 4) good quality, good flow, and most relevant information is covered, making it useful for patients; and 5) excellent quality and excellent flow, making it very useful for patients. A higher GQS score means greater quality of information. This scale is developed by Bernard et al. [15] to interpret the accessibility and flow of the information on websites.

2) JAMA Benchmark Criteria

The JAMA Benchmark Criteria is a 4-point scale to determine the videos and resources’ reliability and accuracy. The four criteria are: (1) authorship, (2) attribution, (3) disclosure, and (4) currency. The examiner evaluates each video and gives one point for each criterion. The scores of the videos range between 0 and 4 points [16].

3) Modified DISCERN Questionnaire

The Modified DISCERN Questionnaire is a scoring scale that involves five yes/no questions. These questions examine the resources’ quality by checking the features they have. The total score was calculated by summing the ‘yes’ scores, which equal 1 point for each, with 0 points indicating the lowest resource quality and 5 points indicating highest [17,18].

5. Statistical analysis

The statistical analysis was performed using SPSS Statistics 16.0 (SPSS Inc., Chicago, IL) software. Distributions of continuous variables were evaluated using the Shapiro–Wilk test. As the descriptive statistics, the number of units (n), percent (%), and median (minimum-maximum) values were given. Comparison of independent groups for nonparametric data was performed with Kruskal–Wallis test and Bonferroni correction was applied to adjust the P values. Spearman correlation analysis was used to evaluate the associations between quantitative variables. The concordance coefficients were used to determine the agreement between three researchers. P values less than 0.05 were considered significant.

RESULTS

The YouTube search revealed a total of 672 videos about CRPS. The first 15 pages with 300 videos were chosen to be evaluated. One hundred thirty-three videos were excluded for various reasons, while the remaining 167 videos were included. A flow diagram of the selection of the videos is shown in Fig. 1.

Fig. 1

Flow diagram of the selection of the videos. CRPS: complex regional pain syndrome.

The majority of the videos were uploaded from the United States (80.2%, n = 134) and the sources of the videos were mostly (40.1%, n = 67) professional health organizations. The sources of the videos are shown in Fig. 2.

Fig. 2

Complex regional pain syndrome videos upload sources.

The videos had mostly good image quality (37.7%, n = 63), and were high definition (41.9%, n = 70). The median viewing rate was 73.3 (range: 1.29–64,445.7). Descriptive data of the videos is shown in Table 1.

Table 1

Descriptive data of the videos

Data	Value (n = 167)
Country
USA	134 (80.2)
UK	10 (6.0)
India	10 (6.0)
Other: Australia, Canada, New Zealand, Saudi Arabia	13 (7.8)
Image quality
Poor	33 (19.8)
Moderate	1 (0.6)
Good	63 (37.7)
High-definition	70 (41.9)
Number of views	639 (4–654,769)
Days online	1,186 (33–5,122)
Video duration (sec)	413 (33–4,603)
Number of comments	1 (0–1,555)
Number of likes	11 (0–10,985)
Number of dislikes	0 (0–240)
Interaction index	1.28 (0.0–33.3)
Viewing rate	73.3 (1.29–64,445.7)
GQS	3 (1–5)
JAMA	2 (1–4)
Modified DISCERN	2 (0–5)

Values are presented as number (%) or median (minimum-maximum).

GQS: Global Quality Scale, JAMA: Journal of American Medical Association benchmark criteria.

Because concordance coefficients were not high enough (0.227 for GQS, 0.206 for JAMA, and 0.037 for DISCERN), consensus was achieved among the 3 researchers for items with differences. According to this consensus, the median value for GQS was 3, while it was 2 for JAMA and DISCERN. Most of the videos had partially sufficient data (2 or 3 points on the JAMA scale) (69.5%, n = 116). The assessment parameters are shown in Table 2.

Table 2

Assessment parameters

Parameter	Value (n = 167)
GQS (1–5 points)
Low content quality (1 or 2 points)	42 (25.2)
Intermediate content quality (3 points)	63 (37.7)
High content quality (4 or 5 points)	62 (37.1)
JAMA score (0–4 points)
Insufficient data (1 point)	49 (29.3)
Partially sufficient data (2 or 3 points)	116 (69.5)
Completely sufficient data (4 points)	2 (1.2)
Modified DISCERN score (0–5 points)
0	3 (1.8)
1	30 (18.0)
2	71 (42.5)
3	41 (24.5)
4	21 (12.6)
5	1 (0.6)

Values are presented as number (%).

GQS: Global Quality Score, JAMA: Journal of American Medical Association benchmark criteria.

When comparing the video parameters with the GQS, which determines the content quality of the videos was made, it was significantly found that the interaction index and viewing rate parameters for high content quality videos were higher than videos with low content quality (P = 0.010, P = 0.014). High content quality videos were more recently added to the feed than both low and intermediate content quality videos (P = 0.010) and the video duration for high content quality videos were longer than intermediate and low-content quality videos (P < 0.001). Comparisons of assessment scores according to the GQS are given in Table 3.

Table 3

Comparison of assessment scores according to Global Quality Score

Variable	Low content quality (n = 42)	Intermediate content quality (n = 63)	High content quality (n = 62)	P valuea
Number of views	431 (10–192,094)	890 (17–654,769)	730.5 (4–104,335)	0.137
Days online	1,429 (45–5,122)	1,562 (33–4,024)	885 (35–3,656)b,c	0.010
Video duration (sec)	250.5 (33–2,204)	333 (74–2,251)	807 (112–4,603)b,c	< 0.001
Number of comments	1 (0–415)	2.0 (0–1,555)	1 (0–105)	0.331
Number of likes	4 (0–6,033)	12 (0–10,985)d	27 (0–919)b	0.001
Number of dislikes	0 (0–210)	0 (0–240)	0 (0–46)	0.607
Interaction index	0.65 (0–33.3)	1.29 (0–7.5)	1.48 (0–33.3)b	0.010
Viewing rate	32.5 (1.95–25,819)	831 (1.45–64,445.7)	112.3 (1.29–10,016)b	0.014
JAMA	1 (1–3)	2 (1–3)d	2 (1–4)b	< 0.001
Modified DISCERN	1 (0–2)	2 (1–4)d	3 (2–5)b,c	< 0.001

Values are presented as median (minimum-maximum).

JAMA: Journal of American Medical Association benchmark criteria.

aKruskal–Wallis test was perfomed. bAdjusted P value for pairwise comparison of the high content quality and low content quality groups (adjusted P < 0.05). cAdjusted P value for pairwise comparison of the high content quality and intermediate content quality groups (adjusted P < 0.05). dAdjusted P value for pairwise comparison of the intermediate content quality and low content quality groups (adjusted P < 0.05).

When we compared the assessment scores based on sources, the number of dislikes for videos uploaded by a patient and by a professional health organization were significantly different from videos uploaded by a YouTube or TV channel (P = 0.027). The GQS, JAMA, and DISCERN scores for videos uploaded by a patient were found to be lower than both those of health-related websites and professional health organizations (P < 0.001) (Table 4).

Table 4

Comparison of assessment scores based on source

Variable	Patient	Physician	Health-related website	Professional health organization	YouTube/TV channel	P valuea
Number of views	224 (10–11,150)	599 (6–74,305)	351.5 (10–104,335)	674 (4–137,038)	1,166 (17–654,769)	0.038
Number of likes	2 (0–312)	12 (0–597)	7.5 (0–919)	12.0 (0–928)	14.5 (0–10,985)	0.169
Number of dislikes	0 (0–7)c	0 (0–57)	0 (0–46)	0 (0–32)d	1.5 (0–240)	0.027
Interaction index	2.73 (0–10.2)	1 (0–33.3)	1.69 (0–14.9)	1.19 (0–11.1)	1.48 (0–6.49)	0.425
Viewing rate	13.2 (5.1–878.6)	76.7 (1.45–3,328)	34.6 (3.8–10,016.6)	90.7 (1.29–6,614.2)	87.0 (2.6–64,445.7)	0.036
GQS	2 (1 to 3)e,f	3 (2 to 4)	4 (1 to 5)	3 (2 to 5)	3 (2 to 4)	< 0.001
JAMA	1 (1–2)e,f	2 (1–2)h	2 (1–4)b	2 (1–4)d	1 (1–2)	< 0.001
Modified DISCERN	1 (0–2)e,f,g	2 (1–4)h	2 (0–5)	3 (0–4)d	2 (1–3)	< 0.001

Values are presented as median (minimum-maximum).

Viewing rate = (number of views / number of days since upload × 100%), Interaction index = ([number of likes–number of dislikes] / total number of views × 100%).

GQS: Global Quality Score, JAMA: Journal of American Medical Association benchmark criteria.

aKruskal–Wallis test was perfomed. bAdjusted P value for pairwise comparison of the health-related website vs. YouTube/TV channel (adjusted P < 0.05). cAdjusted P value for pairwise comparison of the patient vs. YouTube/TV channel (adjusted P < 0.05). dAdjusted P value for pairwise comparison of the professional health organization vs. YouTube/TV channel (adjusted P < 0.05). eAdjusted P value for pairwise comparison of the patient vs. health-related website (adjusted P < 0.05). fAdjusted P value for pairwise comparison of the patient vs. professional health organization (adjusted P < 0.05). gAdjusted P value for pairwise comparison of the patient vs. physician (adjusted P < 0.05). hAdjusted P value for pairwise comparison of the physician vs. professional health organization (adjusted P < 0.05).

There was a significant positive correlation between GQS and video duration (r = 0.533, P < 0.001), number of likes (r = 0.280, P < 0.001), interaction index (r = 0.205, P = 0.008), viewing rate (r = 0.234, P = 0.002), JAMA scores (r = 0.373, P < 0.001) and modified DISCERN scores (r = 0.757, P < 0.001). A negative correlation was found between GQS and days online (r = –0.205, P = 0.008). The correlations between variables and scores are shown in Table 5.

Table 5

Correlations between quantitative variables and scores

Variable	GQS	JAMA	Modified DISCERN
Number of views	r = 0.112	r = 0.246; P < 0.001	r = 0.101
Days online	r = –0.205; P = 0.008	r = –0.012	r = –0.185; P = 0.017
Video duration (sec)	r = 0.533; P < 0.001	r = 0.104	r = 0.390; P < 0.001
Number of comments	r = 0.071	r = 0.041	r = 0.037
Number of likes	r = 0.280; P < 0.001	r = 0.339; P < 0.001	r = 0.272; P < 0.001
Number of dislikes	r = 0.084	r = 0.088	r = 0.042
Interaction index	r = 0.205; P = 0.008	r = 0.148	r = 0.217; P = 0.005
Viewing rate	r = 0.234; P = 0.002	r = 0.327; P < 0.001	r = 0.232; P = 0.003
GQS	-	r = 0.373; P < 0.001	r = 0.757; P < 0.001
JAMA	r = 0.373; P < 0.001	-	r = 0.474; P < 0.001
Modified DISCERN	r = 0.757; P < 0.001	r = 0.474; P < 0.001	-

Nonparametric Spearman’s rank correlation coefficients (rs) are displayed.

GQS: Global Quality Score, JAMA: Journal of American Medical Association benchmark criteria.

DISCUSSION

To the authors’ knowledge, this is the first study evaluating the accuracy and educational quality of the YouTube videos about CRPS. The results of this study revealed that most of the videos had partially sufficient data, and that the interaction index and viewing rate parameters for videos with high content quality were better than videos with low content quality. Additionally, there was a significant positive correlation between GQS and video duration, number of likes, interaction index, viewing rate, JAMA scores, and modified DISCERN scores. Videos with high quality and reliable content are needed to reduce misinformation about CRPS thus contributing to the knowledge and training of patients.

Internet and social media have a very important role in daily activities in today’s world. YouTube is the most popular video sharing platform and the second most visited website worldwide [14]. Since YouTube does not have any screening mechanism on videos, anyone can upload videos about any topic. This can result in spreading misinformation. One study showed that the content of videos about idiopathic pulmonary fibrosis on YouTube were mostly not consistent with recommendations in medical guidelines, and the videos consisting of specific non-medical therapies uploaded by independent users had higher number of views [19]. Furthermore, another study investigating the YouTube videos about gallstone disease revealed that videos added by commercial websites were misleading and even more concerningly two videos uploaded by physicians contained fatal recommendations for patients [20]. In a study on vaccination, which is a very important and current issue for public health, it was revealed that almost half of YouTube videos do not openly support vaccination [21]. In another study examining videos about rheumatoid arthritis, 30.4% of the videos were found to be misleading, including misinformation on the general pathogenesis of the disease and promoting unscientific therapies. In 19% of those misleading videos, the given information was found to be lethal and pathogenic [17]. A study by Stamelou et al. [22] on movement disorders found that some of the videos had inaccurate information about movement disorders and treatments. Kumar et al. [23], investigating the usefulness of videos on hypertension, reported that the viewership was lower for useful videos than misleading videos. This can show us that the content of videos about health can be dangerous for people since accessing this information is too easy and not regulated by any controlling mechanism.

CRPS is a disturbing disease which affects patient’s quality of life, and despite the severity of pain, the lack of understanding of the pathophysiology of the disease can result in inadequate management which, is discouraging for both physicians and patients [24]. On the other hand, the awareness of the disease in pediatric patients is increasing, and there are limited evidence-based treatment options for children [25]. Thus, people with CRPS look for treatment options, and using the internet is the easiest way. For this reason, we aimed to investigate the videos about CRPS on YouTube. We grouped the included videos in to 3 groups, according to their content quality level, as high, intermediate, and low. According to the GQS 25.2% (n = 42) were of low content quality, 37.7% (n = 63) were of intermediate content quality, and 37.1% (n = 62) were of high content quality. Ranade et al. [26] investigating the content quality of YouTube videos on clubfoot found outcomes similar to our data. However, since there are different disease groups and examinations, in contrast to our study results, there are various studies, such as Gonzalez-Estrada et al. [27], Koller et al. [28], Lee et al. [29], which reported that YouTube had low videos content quality about their topic. Ultimately, according to our study data, to contain better information about CRPS, the internet needs more videos with high content quality.

In this study, the authors found that videos uploaded by patients had lower content quality, accuracy, and reliability than videos uploaded by health-related websites, physicians, and professional healthcare organizations. Also, videos posted by professional health organizations were found to be more accurate and reliable than videos uploaded by physicians. In line with our study, Şahin et al. [30], reported that the efficacy of videos uploaded by independent non-medical users was lower than those of healthcare professional uploaders. In another study it was found that the source of the videos correlated with the content quality and reliability. Videos uploaded by professional organizations and universities had the highest comprehensiveness and content quality [31]. Sood et al. [32] also reported that the most relevant videos were posted by university channels and all misleading videos were either medical advertisements or uploaded by independent users. However, in that study, it was found that the viewership of videos posted by universities constituted only 0.6% of total view numbers of all useful videos. According to these results, considering the videos’ source about CRPS, before watching it, can increase the quality of knowledge.

There was a significant positive correlation between the content quality level and the accuracy of the videos. Thus, high content quality videos were more accurate. On the other hand, it was found that videos with longer duration had higher content quality. However, there was a negative correlation between the accuracy and content quality of the videos with their number of days online. Considering this, it can be said that the older a video is, the less accurate the information it contains, and before deciding to watch a video about the CRPS, checking its source may be useful in determining how beneficial the knowledge about the disease will be. In addition, high content quality videos had higher interaction indexes and higher viewing rates. We found a positive correlation between the number of likes and GQS, JAMA, and DISCERN scores. So, it is clear that the videos with higher content quality, accuracy, and reliability had higher numbers of likes, but the sources of the videos were not correlated with the number of likes or viewers.

There are some limitations of this study. First is we included only videos in English. Secondly, despite using the GQS, JAMA, and DISCERN scales, these are specific tools, and there is no concurrence regarding the way to evaluate videos related to health care issues, and the overall evaluation of the videos is subjective, and depends on the researcher. This may have resulted in the low concordance coefficients among the researchers in this study, and a consensus had to be reached for the final decision, as was done in Koller et al. [28]. Since this study did not involve actual patients, it is not clear what viewers understood from the videos. Although the evaluation has been done in one day, since YouTube is a dynamic platform, the number of videos, likes, dislikes and views can change every second. And finally, some reliable or non-reliable contents may have been missed due to our search title but we assumed that the uploaders and searchers might have preferred to use the phrase “complex regional pain syndrome” while searching for videos. So, this limitation may have a minimal effect on this study. Despite these limitations, this study can provide instantaneous knowledge about current YouTube videos about CRPS.

Our results showed that videos about CRPS on YouTube mostly had partially sufficient data and high content quality videos had higher viewing rates, interaction indexes, numbers of likes, and longer durations. Also, according to our results higher content quality videos are more recently uploaded. Since information on health can be easily accessed on the internet and physicians do not have any chance to control misinformation, the goal of health care organizations and physicians should be to upload more reliable and accurate videos on YouTube.