Evaluation of pericardiocentesis videos on YouTube as a reliable source for on-line education in the COVID-19 period.

INTRODUCTION: Pericardiocentesis is an invasive procedure performed to drain fluid from the pericardial cavity. AIM: We investigated the quality of videos about pericardiocentesis on YouTube and analysed their reliability and teaching properties, in the covid 19 period when online education has come to the fore.
MATERIAL AND METHODS: We searched YouTube using the terms "pericardiocentesis, cardiac tamponade, pericardial effusion, pericardial effusion drainage, pericardial tamponade" for uploaded videos. We scored every video according to the questions we prepared using the guidelines about pericardiocentesis. We used the HONcode score, GQS score, and RELIABILITY score to assess the quality of videos. Two physicians independently and blindly classified videos as useful or misleading and rated them.
RESULTS: A total of 168 videos were examined. After the application of exclusion criteria, 38 videos were evaluated. The pericardiocentesis checklist average score was 10.45 ±2.56. According to sources of videos, the average score for university or research hospital videos was 13.1 ±1.5, and videos whose source could not be identified had an average score of 7.5 ±2.0. According to the level of HONcode, 17 (44.7%) videos were low quality; according to GQS score, 8 (21.1%) videos were poor quality. The quality of university hospital uploads (β-coefficient 3.960, p-value 0.004) were higher and statistically significant than other upload centres.
CONCLUSIONS: The educational value of pericardiocentesis videos on YouTube are low. It is recommended that doctors and patients be aware of and adopt the developing technology, and they should prefer videos uploaded from university hospitals and educational hospitals. Copyright:

Introduction

Accumulation of fluid in the pericardial cavity is a life-threatening condition. Fluid collection may be due to cardiac or non-cardiac causes, but the results can be dramatic. Drainage of the fluid accumulated in the pericardial cavity is called pericardiocentesis and is performed by cardiologists and emergency medicine professionals as an acute therapeutic procedure for life-threatening conditions. It was first described in 1653 by Riolanus. The technique and application of pericardiocentesis have evolved over the years. As stated in the ESC position paper related to pericardiocentesis, procedures can be performed from different approaches (subxiphoid, apical, and parasternal) and with different guides (echocardiography guided, fluoroscopy-guided, and electrocardiography guided) [1, 2]. Although pericardiocentesis is performed in the light of theoretical information, it is beneficial for the clinician to see how the procedure is performed.

Almost 4.54 billion people were active internet users as of January 2020, encompassing 59% of the global population [3]. Today, the internet and social media have played a part in the lives of people from all social layers of life. The probability of disseminating misleading information to healthcare consumers is high and can have catastrophic implications [4]. Video sharing channels, which are accessible to everyone, provide visual information to viewers. YouTube is the largest video sharing site, with over 300 hours of video uploaded every minute [5]. Unlike journal articles or textbooks that are heavily reviewed, there is no review process for the videos being uploaded on YouTube. For this reason, research to establish the accuracy and reliability of these educational videos is crucial.

There are some tools that were created for measuring the quality of videos or health-associated internet sites; the DISCERN score, HONcode score, GQS score, RELIABILITY score, and JAMA benchmark instrument are the most used ones of them [6-9].

Aim

In our study, we tried to determine the medical suitability of videos related to pericardiocentesis, which were shared on YouTube, which is easily accessible, free, and popular. Their reliability was determined by using evaluation methods that were suitable for our study, such as HONcode score, GQS score, and RELIABILITY score. We aimed to test and characterize the content and quality of information on pericardiocentesis in YouTube videos in the COVID-19 pandemic period.

Material and methods

We conducted this study to identify the educational properties of pericardiocentesis procedure videos on YouTube. The YouTube (www.YouTube.com) search bar was checked using the keywords ‘pericardiocentesis’, ‘cardiac tamponade’, ‘pericardial effusion’, ‘pericardial effusion drainage’, and ‘pericardial tamponade’. The collection of all videos containing information regarding pericardiocentesis was conducted on 10 March 2020. We did not use any personal Google or YouTube account for the video search. The search yielded 168 videos in total; after applying exclusion criteria, there were 38 remaining videos to examine. Videos were ranked by relevance and screened for information on pericardiocentesis. We collected the following quantitative data: view counts, comments, likes, dislikes, and duration. All videos were viewed and analysed by 2 independent physicians who have sufficient experience in pericardiocentesis procedures (SCE, SK); in the event of a discrepancy, a third reviewer arbitrated the disagreement. Videos were rated by researchers according to HONcode score, GQS score, and RELIABILITY score.

Videos not using English language were not included in the study.

Only one video was considered if the same video was repeated or if the video was found in parts.

Videos with incompatible content were not included in the study.

Animated videos were not included in the study.

Videos shorter than 30 seconds were not included in the study.

Patient or miscellaneous (when identifying data could not be found or determined).

Physician.

Educational channel.

University or Educational Hospital.

Pericardiocentesis Checklist

We prepared a checklist according to pericardiocentesis consensus papers [2, 10, 11]. We prepared 15 questions. We rated the videos according to the questions (no: 0, yes: 1) in this checklist. Videos with a rating of more than 80% of questions were defined as good quality videos (Table I).

Table I

Pericardiocentesis checklist according to consensus reports

Question	Rating
1. Is the patient's position suitable?	0 1
2. Is there a saturation probe?	0 1
3. Is the patient’s arterial blood pressure monitored?	0 1
4. Does the patient have a central venous catheter?	0 1
5. Does the patient have electrocardiographic monitorization?	0 1
6. Was appropriate sterilization performed?	0 1
7. Was a guiding instrument (echocardiography, fluoroscopy, electrocardiography) used during the process?	0 1
8. Is there anyone who assists in the process?	0 1
9. Is sedation used?	0 1
10. Is the needle entry angle appropriate?	0 1
11. Is needle insertion suitable?	0 1
12. Is the needle advanced by aspiration?	0 1
13. Is the coagulation of aspirate fluid checked?	0 1
14. Was the needle was confirmed to be in the pericardial cavity?	0 1
15. Was a sheet placed accordingly guideline and closed?	0 1

Reliability of information score

All videos were analysed with a RELIABILITY score in terms of the reliability and integrity of their information based on a 5-point scale. The RELIABILITY score, which is derived from the DISCERN score, attempts to determine the reliability of the information based on 5 questions [7]. The result can reach a maximum of 5 points by giving 1 point for each question covered in the video. The reliability of information questions can be seen in Table II.

Table II

Reliability of information questions

Question	Rating
1. Are the aims clear and achieved in the video?	0 1
2. Are reliable sources of information used in the video?	0 1
3. Is the information presented balanced and unbiased in the video?	0 1
4. Are additional sources of information listed for patient reference?	0 1
5. Are areas of uncertainty mentioned in the video?	0 1

GQS description

All videos were also rated using a global quality score (GQS) that uses a 5-point scale to rate the overall quality of the video. The GQS score is an assessment to ensure that the quality of information and the reviewer decide how useful a particular video will be for a patient [8]. The GQS questions can be seen in Table III.

Table III

GQS description questions

Question	Rating
1. Poor quality, poor flow of the video, most information missing, not at all useful for patients	0 1
2. Generally poor quality and poor flow, some information listed but many important topics missing, of very limited use to patients	0 1
3. Moderate quality, suboptimal flow, some important information is adequately discussed but others poorly discussed, somewhat useful for patients	0 1
4. Good quality and generally good flow. Most of the relevant information is listed, but some topics not covered, useful for patients	0 1
5. Excellent quality and flow, very useful for patients	0 1

HONcode Score

The HONcode is one of the oldest methods used to present and evaluate health-related information on the internet. Studies have shown that websites that comply with the HONcode principles provide quality health information to patients and healthcare professionals. The HONcode tool, consisting of 8 adapted questions, is used to evaluate the quality of the videos. For each question, the evaluator states whether the principle is followed in the video (1: yes, 0: no), and the video is evaluated by calculating the total HONcode score [6].

The HONcode adherence level was determined by total score: 0–2 means low adherence; 3–5 means medium adherence; and 6–8 means high adherence. HONcode questions can be seen in Table IV.

Table IV

HONcode instrument questions

Question	Rating
1. Any medical or health advice given in the video must come from a qualified health professional unless it is clearly stated that the information does not come from a qualified health source	0 1
2. The information provided in the videos must be designed to support the patient’s self-management but is not meant to replace a patient–physician relationship	0 1
3. The information in the video maintains the right to confidentiality and respect of the individual patient featured	0 1
4. Each video contains references to source data on information presented or contains a specific HTML link to source information	0 1
5. Each video containing claims on the benefits or performance of specific, skills/behaviours, interventions, treatments, products, and so on must be supported by evidence through references or HTML links	0 1
6. The video must provide the viewer with contact information, or contain a website link to more information	0 1
7. Any individual or organization that contributes funds, services, or material in the posted video must be clearly identified in the video or video description	0 1
8. If advertisement supports funding to the video or the video’s developers, it must be clearly stated. Included advertising must be clearly differentiable to the viewer: There should be a clear difference between the advertising material and the educational material in the video	0 1

Ethics

Because of the anonymity of the data, there was no need to obtain ethical approval.

Statistical analysis

For statistical analysis, R software version 4.02 (Vienna, Austria), ‘Hmisc’, ‘rms’ packed was used. Describing the continuous data, we used mean ± standard deviation (for normal distribution), and the median and interquartile range (25–75 IQR) for non-normal distribution. Frequency and percentage were given for categorical data, and the χ2 test was used to compare these parameters. The Shapiro-Wilk test and histogram graphic analysis were used to check the normality of data for quantitative variables.

A correlation test was applied for HONcode, GQS, RELIABILTY scores, and the pericardiocentesis checklist score. The Pearson test was applied if data had a normal distribution, and if not, the Spearman correlation test was applied. The Cronbach-α value was evaluated for inter-rater assessments such as HONcode, GQS, and total pericardiocentesis checklist score. Among 4 upload centre group scores of HONcode, pericardiocentesis checklist, GQS, and RELIABILTY instruments were tested with one-way ANOVA or Kruskal-Wallis (non-normal distribution) for numerical variables; if needed, sub-group analysis with Tukey post hoc test was applied.

With linear regression, the most informative parameter pericardial checklist was taken as a dependent variable to the model as numeric. Independent variables were as follows: HONcode, GQS, likes, dislikes, and video duration. The statistical significance was considered as p-value < 0.05.

Results

A total of 168 videos were examined. After the application of the exclusion criteria, 38 videos were evaluated. A sample size of 38 meets the guidelines for conducting reliability analyses in media content analyses (i.e. at least 50 cases or 10 to 20% of the sample). When looking at the video sources, it was determined that 8 (21.1%) videos originated from a university or research hospital, 6 (15.8%) videos originated from educational sites, 18 (47.4%) videos were shared by individual health workers, and 6 (15.8%) videos could not be identified. The average duration of the videos was 2 ±0.6 minutes, and the average number of views was 16.799 ±4890. The average number of likes was 43 ±12, and the average number of dislikes was 4 ±1. The number of average comments was 2 ±0.7 (Table V).

Table V

Video parameters and scores

Parameter	1	2	3	4	All groups	P-value
Number of videos, n (%)	6 (15.8)	18 (47.4)	6 (15.8)	8 (21.1)
Duration [min]					2 ±0.6
View					16.799 ±4890
Like					43 ±12
Dislike					4 ±1
Comment (number)					2 ±0.7
Pericardiocentesis Checklist Score:
Score (given as mean)	7.5 ±2.0	9.7 ±1.9	12 ±1.4	13.1 ±1.5	10.45 ±2.56	< 0.001
95% confidence interval	(5.32, 9.68)	(8.78, 10.67)	(10.52, 13.48)	(11.83, 14.42)
GQS Score:
Mean	1.33 ±0.51	2.17 ±0.78	3.5 ±0.83	3.38 ±1.30	2.5 ±1.15	< 0.001
95% confidence interval	(0,79, 1.88)	(1.78, 2.56)	(2.62, 4.38)	(2.29, 4.46)	(2.12, 2.88)
HONcode:
Mean	1.17 ±0.98	2.56 ±1.46	4.17 ±1.94	4.63 ±2.13	3.03 ±1.97	0.001
95% confidence interval	(0.13, 2.20)	(1.83, 3.28)	(2.13, 6.20)	(2.84, 6.41)	(2.38, 3.68)
RELIABILITY:
Mean	2.0 ±0	2.44 ±0.85	3.33 ±0.81	3.13 ±0.99	2.66 ±0.90	0.016
95% confidence interval	(2.0, 2.0)	(2.02, 2.87)	(2.48, 4.19)	(2.30, 3.95)	(2.36, 2.96)

Cronbach’s α method was used for interrater reliability analysis. Values for HONcode score and total pericardiocentesis score were found to be 0.917 and 0.953, respectively.

Pericardiocentesis checklist

According to the checklist, which was prepared according to the pericardiocentesis consensus reports, the average score was 10.45 ±2.56. According to sources of videos, the average score of university or research hospital videos was 13.1 ±1.5, the average score of educational site videos was 12 ±1.4, individual health workers videos averaged 9.7 ±1.9, and videos that could not be identified averaged 7.5 ±2.0 (Table IV).

According to the checklist, questions 11 (n = 37, 97.4%) and 9 (n = 35, 92.1%) were the hughest rated ones; on the other hand, questions 4 (n = 10, 26.3%) and 14 (n = 13, 34.2%) were the least rated ones.

Checklist questions and their ratings (n: number of videos, % percentage)

Is the patient’s position suitable? (n = 28, 73.7%)

Is there a saturation probe? (n = 21, 55.3%)

Is the patient’s arterial blood pressure monitored? (n = 30, 78.9%)

Does the patient have a central venous catheter? (n = 10, 26.3%)

Does the patient have electrocardiographic monitoring? (n = 31, 81.6%)

Has appropriate sterilization been performed? (n = 31, 81.6%)

Was a guiding instrument (echocardiography, fluoroscopy) used during the process? (n = 26, 68.4%)

Did anyone assist in the process? (n = 25, 65.8%)

Was sedation used? (n = 35, 92.1%)

Was the needle entry angle appropriate? (n = 32, 84.2%)

Was the needle insertion suitable? (n = 37, 97.4%)

Was the needle advanced by aspiration? (n = 34, 89.5%)

Was the coagulation of aspirate fluid checked? (n = 13, 34.2%)

Was the needle confirmed to be in the pericardial cavity? (n = 16, 42.1%)

Was a sheet placed accordingly and closed? (n = 28, 73.7%)

HONcode

A majority of the videos met the first question criterion, which required medical and health advice to come from a qualified health professional, unless clearly stated otherwise (n = 34, 89.5%). Videos meeting the criterion for the second question, which was providing patient education on pericardiocentesis self-management without the intention to replace the patient-physician relationship, was more than half (n = 25, 65.8%). Also, nearly half of the videos maintained the right to confidentiality and respect of patients, as featured in the third question (n = 18, 47.4%). The number of videos meeting the criterion for the fourth question, which contained references to source data on the information presented, was low (n = 11, 28.9%). The number of videos meeting the fifth criterion, which was requiring claims to be supported by evidence through references or hypertext markup language (HTML) links, was low (n = 7, 18.4%). For the sixth criterion, which asked about additional contact information and/or a website link for further information, was met by one-third of the videos (n = 13, 34.2%). The 2 HONcode questions with the fewest videos in compliance were the seventh question, which identified funders and contributors of information presented in the video or within video descriptions (n = 3, 7.9%) and the eighth question, which only applied to videos supported by funding from advertisers (n = 5, 13.2%).

The mean HONcode score was 3.12 ±1.95 and 2.91 ±1.99 between raters, respectively, and the mean score was 3.03 ±1.97 (Table V).

RELIABILITY score

The reliability of the information questionnaire score was 2.62 ±1.12 and 2.70 ±0.68 between raters, respectively, and the average score was 2.66 ±0.90 (Table V).

GQS Score

The GQS description score was 2.56 ±1.12 and 2.44 ±0.18 between raters, respectively, and the mean score was 2.5 ±1.15 (Table V).

Quality of videos

According to the pericardiocentesis checklist score, the number of videos that responded positively to more than 80% of the questions was 8 (21). Only 21% of the videos were evaluated as compatible with the position paper of ESC according to the level of HONcode adherence (low, medium, and high). The majority of videos across all media categories were rated as low quality (n = 17, 44.7%), 16 (42.1%) were judged to be medium quality, and 5 (13.2) and were judged to be high quality.

According to the GQS score, video qualities were as follows: 8 (21.1%) were poor quality, 13 (34.2%) were generally poor quality and poor flow videos, 9 (23.7%) were moderate quality, 6 (15.8%) were good quality and generally good flow videos, and 2 (5.3%) videos were excellent quality (Table VI).

Table VI

Quality of videos

According to	Poor quality	Good quality	Excellent quality
Pericardiocentesis Checklist Score	30 (79%)	8 (21%)
GQS Score	30 (78.8%)	8 (21.2%)
HONcode	17 (44.7%)	16 (42.1%)	5 (13.2%)
RELIABILITY	31 (81.6%)	7 (18.4%)

Pericardiocentesis checklist scores of videos and changes according to HONcode, RELIABILTY, and video duration are shown in gg plots in Figure 1. The graph shows that the parameters are increasing linearly.

Figure 1

Pericardiocentesis checklist scores of videos and changes according to HONcode, RELIABILITY, and video duration shown in scatterplot

It was observed that there was a statistical difference between GQS Score, HONCODE, and RELIABILTY scores of the videos that were found to be appropriate or not appropriate according to the pericardiocentesis checklist. Mean scores of videos according to pericardiocentesis checklist quality can be seen in Table VII.

Table VII

Mean scores of videos according to Pericardiocentesis Checklist Scores

Parameter	Pericardiocentesis checklist score good or excellent (n = 8, 21%)	Pericardiocentesis checklist score poor (n = 30, 79%)	P-value
GQS Score	4.0 ±0.81	2.16 ±0.93	< 0.001
HONcode	5.29 ±1.7	2.52 ±1.67	< 0.001
RELIABILITY	3.57 ±0.97	2.45 ±0.76	0.002

Upload centre 1 was defined as a reference for comparing other upload centres. Total pericardiocentesis checklist score, which is the main parameter in terms of teaching and training features of the videos was checked according to videos score and parameters. The parameters predicting the total pericardiocentesis checklist score was analysed in a multivariable linear regression model. The university hospital uploads (β-coefficient 3.960, p-value 0.004) were statistically more significant than other upload centres, but other parameters (such as HONcode, reliability, video duration, like, dislike) included in the model were not statistically significant (Table VIII).

Table VIII

Upload centre differences according to scores

Variables	β-coefficient	Confidence interval	P-value
HONcode	0.217	–0.499, 0.933	0.539
RELIABILITY score	0.434	–0.933, 1.801	0.521
Video dration	1.054	–2.058, 4.167	0.493
Like	0.008	–0.006, 0.022	0.256
Dislike	–0.071	–0.212, 0.069	0.305
Upload centre:
2-1	1.490	–0.315, 3.297	0.102
3-1	2.165	–0.552, 4.883	0.114
4-1	3.960	1.389, 6.530	0.004

Discussion

Although invasive procedures can be learned theoretically, the value of visual sources cannot be ignored in terms of consolidating information. In our study, we tried to determine the accuracy of tutorials in the videos shared on YouTube regarding pericardiocentesis attempts. According to the checklist, which was based on the consensus papers of ESC with a maximum score of 15, we found that only 8 (21%) of these videos met the criteria with sufficient information by maintaining a score of more than 12. We determined that 5 (13.2%) of the videos were of suitable quality according to the HONcode score. In the evaluation we made in light of these results, we determined that the quality of videos shared on YouTube about pericardiocentesis education is low. We have shown that these videos may not be suitable for educational purposes.

YouTube is a public portal that creates a platform for everyone to share. This may have beneficial aspects as well as misleading ones. YouTube videos contain great heterogeneity in terms of image quality, primary upload purpose, upload source, and reliability. As we have seen during the COVID-19 pandemic, when online education came to the fore, the reliability of videos uploaded to video sites is of great importance. However, there is no evaluation process before uploading these videos, and everyone can share. It has been shown that the number of views and the amount of likes of the videos are not significant in terms of their reliability [12]. In our study, it was seen that the video-sharing made especially by university hospitals or training hospitals was statistically high compared to individual posts. For this reason, our opinion for healthcare professionals and patients is that they should prefer university hospitals or educational hospital posts in the selection of informative and educational videos. As we have shown in our study, it has been shown in other studies that videos made by professionals of educational institutions are more reliable [13, 14].

In some studies, evaluating YouTube videos in terms of education, it has been observed that YouTube is beneficial for learning and the videos are sufficient [15, 16]. Also, it has been stated in most publications that the video quality is not sufficient [17, 18]. It is noteworthy that publications which include sufficient video quality are containing short and specific procedures videos [15]. It is known that YouTube videos are the most frequently used visual education resource in surgical education, so it is important for users to evaluate videos in terms of reliability by using procedures as well as evaluating scientific publications [19, 20].

There were few references of sharing in the videos. This finding is consistent with previous studies, which have found that online information can offer an unbalanced view with little or poor referencing to scientific data [21]. Evidence in the literature proves that multimedia-oriented learning has advantages in terms of understanding complex temporal and spatial events [22].

With this study, we showed that the training videos uploaded to YouTube are not always educational at the desired level. The lack of training videos on highly reliable cardiology and cardiothoracic surgeons’ websites can lead novices to use videos on YouTube for educational purposes. For this reason, it may be useful to check the educational suitability of videos using special tools such as HONcode, RELIABILITY, and global quality score to ensure that these trainees develop satisfactorily.

There are limitations of the study. First, the lack of a specific YouTube scoring utility and scoring system. After defining the steps of pericardiocentesis according to a reference consensus reports, we tried to overcome this limitation with specific tools such as the global quality score, HONcode, Modified Discern, and JAMA. Scoring systems such DISCERN, HONCODE, etc. are quite general or are geared more towards patients than proceduralists. Other video channels such as Dailymotion were not considered. Our study was snapshot searching of YouTube videos, because of this after study period finished videos might be increased or revised by uploaders.

Conclusions

YouTube contains variable quality videos related to pericardiocentesis. The educational value of these videos up until now is questionable. It might be recommended that healthcare trainers be aware of and adopt the developing technology, and they might make prefer videos uploaded from university hospitals and educational hospitals.

Disclosure

Authors report no conflict of interest.