Analysis of Quality, Usefulness, Reliability, Visibility, and Popularity of Videos about Dental Caries on YouTube: A Cross-sectional Analysis.

Objectives: YouTube is the most popular social media and is widely used to access dental and oral diseases information. Nonetheless, the quality of online health information remains a problem. This study aimed to analyze the quality, usefulness, reliability, visibility, and popularity of YouTube videos about dental caries as a source of information. Materials and
Methods: The study design was a cross-sectional analytical study. Here, 300 videos that were uploaded in September 2019-2020 were screened, and total duration, number of views, number of likes, number dislikes, uploader category (individual users or health professionals), and upload date of the videos were recorded. One hundred videos were analyzed in terms of their quality, usefulness, reliability, visibility, and popularity using the Global Quality Scale, usefulness score, DISCERN, viewing rate, and interaction index, respectively.
Results: Videos were 78% uploaded by individual users. Videos uploaded by health professionals showed high scores in quality, usefulness, and reliability. Videos that were greater than 6 min long exhibited both better quality and higher popularity. Video with good-quality showed high visibility and popularity. In addition, videos with better usefulness and reliability showed high visibility; however, their popularity was low. Conclusions: Information about dental caries on YouTube is limited in quality. YouTube videos have a potentially important role in oral health education. Copyright:

INTRODUCTION

The Internet is been widely used by most of the world’s population.[1] As of January 2020, there were 4.54 billion global Internet users and 175.4 million Internet users are from Indonesia.[2] In addition, social media services are used nearly daily by many people connected to the Internet.[3] Currently, there are 160 million social media users in Indonesia.[2] Compared to other social media services, online video sharing applications, such as Google, Facebook, and YouTube are the most frequently used information source, especially health information.[45] Registered YouTube users can easily share any video without being validated,[6] and 60% of Internet users have searched for health information on YouTube.[7] This makes YouTube video publications vulnerable due to potentially invalid information, which can affect health-related decisions.[6] As many as 75% of patients address health problems based on the results of health information obtained from online media.[5]

YouTube is widely used to access information related to dental and oral diseases. Dental caries are health problems that affect nearly all age groups.[8] In 2018 the average decay missing filling teeth index of permanent teeth at the age of 12 years and over in Indonesia is 7.1, and this includes very high risk of dental caries.[9] If not treated immediately, dental caries will develop and eventually cause infection and pain. In Indonesia, the prevalence of permanent dental caries is 61% in children age 12-year-old, which is higher than in other countries in Southeast Asia.[10] Poor oral conditions can impact the functional, emotional, and social development of children and their families.[11] In addition, a lack of individual knowledge can increase the risk of dental caries.[12] Early health education plays an important role in the prevention of dental caries, and educational media must be in line with current trends in society. The use of technology can make it easier to obtain health information because technology is very prevalent in modern society.[12]

There have been many discussion about the quality of YouTube videos about oral health in other countries.[1314] Although some online health information is of good quality, poor-quality information remains a problem.[15] In Indonesia, the quality of YouTube videos about oral health information has not been investigated thoroughly. Therefore, to provide good oral health information via online video sites, particularly information about dental caries, this study investigates the quality, usefulness, reliability, visibility, and popularity of Indonesian YouTube videos about dental caries as a source of health information.

MATERIALS AND METHODS

The reporting of this study is in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement.[1617] This study was approved by the Research Ethics Committee of the Faculty of Dentistry, Universitas Indonesia, No. 18/Ethical Approval/FKGUI/VIII/2020 with Protocol No. 090190720. YouTube videos about dental caries were acquired using a selection of video filters in the past year. Video searching used the keyword “dental caries,” which were uploaded to YouTube in the last 1 year (September 2019–2020). After searching with the given keyword, as many as 300 videos were screened, and then videos were selected with inclusion criteria using all Indonesian language YouTube videos containing information about dental caries. The exclusion criteria included videos with no sound, videos that discuss dental caries but dominantly discuss other diseases, duplicate videos, advertisements (e.g., oral health promotions and dental clinics), and video conferences or lectures that target a specialized audience.[14]

A total of 100 videos were assessed for complete analysis based on the inclusion and exclusion criteria, the total duration, number of views, likes, dislikes, uploaders, and date of upload information were recorded. In addition, the videos were categorized based on the upload source, that is, independent users or health professionals.[1418] The information in the YouTube videos was assessed using four analysis techniques, which are Global Quality Scale (GQS),[13] Usefulness,[14] reliability based on five questions adapted from the DISCERN questionnaire,[13] and viewer interaction (e.g., number of views, likes, dislikes, and date of upload).

Video quality was investigated using the GQS based on the information of the video as a whole and how useful the video was for patients according to the following criteria: 1 = very poor quality (poor flow, lack of information, nothing useful for patients); 2 = generally poor quality (low level of flow, some information is listed, but there are many important topics, very limited use to patients); 3 = moderate quality (flow is less than ideal, some important information is discussed adequately, other pieces of information are discussed poorly, somewhat useful to patients); 4 = good quality (generally good flow, most of the relevant information is listed, some topics are not addressed, useful to patients); 5 = excellent quality (excellent flow, very useful for patients).[13] Usefulness was assessed whether the information in the videos included definitions, indications, contraindications, advantages, involved procedures, complications, costs, and prognosis. Each item was given one point if the information was mentioned in the video. The point range was 0 to 8, with 0 to 2 points representing poor quality, 3 to 5 points representing moderate quality, 6 to 8 points representing excelling quality.[14]

Reliability was investigated based on five questions adapted from the DISCERN questionnaire to assess the completeness of the information and the reliability of the information source. The questionnaire comprises of five questions.[1] Are the objectives clear and achieved?[2] Are the information sources reliable?[3] Is the presented information balanced and unbiased?[4] Are additional information sources listed for patient reference?[5] Are areas of uncertainty mentioned? Each question was rated as 1 if it was mentioned in the video, with the total score ranging from 0 to 5.[13] Further, viewer interaction was investigated in terms of video features, for example, the number of views, number of likes, number of dislikes, and the upload date, which were calculated to generate a viewing rate to assess visibility and interaction index to assess popularity. The viewing rate and interaction index are expressed as follows:[614] Viewing rate (visibility) = ; Interaction index (popularity) =

A pilot study was conducted to calibrate and analyze the reliability of the observer. Two observers viewed the same videos independently. The reliability measurement was conducted by observing (with both inter- and intra-observer) 20 videos repeatedly at different times. An analysis was then performed using the intraclass correlation coefficient (ICC) with an expected value of >0.80 that indicated good agreement. Statistical analysis was performed using a licensed SPSS Statistical Software IBM SPSS version 28.0 (IBM, Armonk, New York). Here, the inter-observer agreement was calculated using the ICC, and then the variables were analyzed using the Mann–Whitney test for the nonparametric data. The statistical significance was set to P < 0.05 in this statistical analysis.

RESULTS

In total 100 videos were analyzed. The inter- and intraobserver ICC values for all measurement categories were >0.8, which indicates good agreement. Tables 1 and 2 show the characteristics of the sample and the overall performance of the evaluated videos. Most of the assessed videos were produced by independent users (78%; n = 78). The average video duration was 6 min 27 s (ranging from 96 s to 58 min 40 s). The average number of video views was 65,348 (ranging from 10 to 1,114,735). The number of views, likes, dislikes, and duration as upload are related to the interaction index and viewing rate. From the overall analysis, the average usefulness score and GQS of the videos were poor.

Table 1

Sample characteristics

Variable
Source, no (%)
Independent users	78 (78)
Health professionals	22 (22)
Uploaded since (days)
Mean (SD)	200 (117)
Min–max	3–377
Duration (s)
Mean (SD)	387 (374)
Min–max	96–3,520
Views
Mean (SD)	65,348 (187,286)
Min–max	10–1,114,735
Likes
Mean (SD)	595 (1,676)
Min–max	0–13,205
Dislikes
Mean (SD)	57 (177)
Min–max	0–1,036

Table 2

Videos analysis according to different variables

Variable
Interaction index
Mean (SD)	3.47 (6.05)
Min–max	0.0–28.5
Viewing rate
Mean (SD)	26,323 (65,935)
Min–max	8–405,664
Usefulness score (0–8)
Mean (SD)	1.9 (1.4)
Min–max	0–6
DISCERN (1–5)
Mean (SD)	2.3 (0.6)
Min–max	1–4
GQS (1–5)
Mean (SD)	2.2 (0.9)
Min–max	1–5

Table 3 shows that there were significant differences relative to the number of views, viewing rate, likes, interaction index, usefulness score, GQS, and DISCERN in score each uploader category. The number of views of videos with the same keywords uploaded by health professionals is greater than that of videos uploaded by individual users, with a median of 7.619 (P = 0.037). This significantly affects the viewing rate of videos uploaded by health professionals who also have higher values (P = 0.013). The number of likes on videos uploaded by health professionals is also greater than that for video uploaded by individual users, with median of 135 (P = 0.001). However, the interaction index and dislikes values do not show statistically significant differences between the two uploader categories. In the GQS, usefulness score, and DISCERN score assessments, a significant difference was observed in terms of the corresponding values for videos uploaded by health professionals (P = 0.001).

Table 3

Videos classification according to the upload source

	Individual users	Health professional	P Value
	Median (min–max)	Median (min–max)
Views	1,603 (14–581,030)	7,619 (301–1,114,735)	0.037*
Viewing rate	1,004 (11–194,324)	3,888 (8–405,664)	0.013*
Likes	22 (0–3,622)	135 (0–13,025)	0.001*
Interaction index	1.20 (0–28.5)	1.32 (0–27.19)	0.339
Dislikes	2 (0–988)	2 (0–1,036)	0.725
GQS	1 (1–4)	3 (1–5)	0.001*
Usefulness score	1 (0–5)	4 (0–5)	0.001*
DISCERN	2 (1–4)	3 (2–4)	0.001*

*Significance P < 0.05. Mann–Whitney test

Relative to the video duration analysis, Table 4 shows that differences in the interaction index and GQS were observed between the two duration categories, that is, less or greater than 6 min. Videos greater than 6 min show a higher interaction index, with a median of 1.93 (P = 0.001), and the GQS value also shows that durations greater than 6 min have a higher video quality rating, with a median of 2 (P = 0.027). In terms of the GQS, usefulness score, and DISCERN score analyses, Tables 5,6–7 show that there were no significant differences between the number of views, viewing rate, number of likes, and interaction index for GCS, usefulness, and DISCERN score category.

Table 4

Comparison of videos according to duration (min)

	Up to 6:00	More than 6:00	P Value
	Median (min–max)	Median (min–max)
Views	5,136 (15–1,114,735)	1,000 (14–1,001,992)	0.053
Viewing rate	2,564 (35–335,727)	771 (8–405,664)	0.124
Likes	51 (0–13,205)	27 (0–6,351)	0.481
Interaction index	0.88 (0–24.14)	1.93 (0–28.5)	0.001*
Dislikes	2 (0–988)	1 (0–1,036)	0.151
GQS	1 (1–4)	2 (1–5)	0.027*
Usefulness score	1 (0–5)	2 (0–5)	0.134
DISCERN	2 (2–4)	2 (1–4)	0.845

*Significance P< 0.05. Mann–Whitney test

Table 5

Comparison of videos according to global quality score (GQS)

	GQS 1 & 2		GQS 3, 4, & 5		P Value
	Mean (SD)	Median (min–max)	Mean (SD)	Median (min–max)
Views	59,698 (157,810)	2,193 (14–1,001,992)	83,649 (263,180)	1,983 (23–1,114,735)	0.666
Viewing rate	24,958 (59,352)	1,731 (35.8–405,664)	30,646 (84,853)	1,851 (8,83–335,763)	0.578
Likes	489 (1,028)	33 (0–4,915)	934 (2,913)	58 (0–13,205)	0.460
Interaction index	2.87 (5.57)	1.11 (0–28.5)	5.37 (7.16)	3.02 (0–27.19)	0.560

*Significance P < 0.05. Mann–Whitney test

Table 6

Comparison of videos according to usefulness score

	Usefulness score (0–3)		Usefulness score (4–8)		P Value
	Mean (SD)	Median (min–max)	Mean (SD)	Median (min–max)
Views	62,729 (186,491)	1,739 (14–1,114,735)	78,712 (196,195)	5,272 (241–712,397)	0.300
Viewing rate	25,713 (65,759)	1,158 (11–405,664)	29,302 (68,750)	2,459 (8–261,910)	0.124
Likes	574 (1,690)	29 (0–13,205)	701 (1,648)	71 (0–6,351)	0.122
Interaction index	3.47 (6.13)	1.19 (0–28.5)	3.43 (5.79)	1.34 (0–24.5)	0.670

*Significance P < 0.05. Mann–Whitney test

Table 7

Comparison of videos according to DISCERN score

	DISCERN (0–1)		DISCERN (2–5)		P Value
	Mean (SD)	Median (min–max)	Mean (SD)	Median (min–max)
Views	2,513 (4,491)	404 (14–9,230)	68,068 (190,699)	2,193 (15–1,114,735)	0.082
Viewing rate	1,654 (2,974)	218 (68.25–6,112)	27,351 (67,108)	1,923 (8–405,664)	0.082
Likes	19 (25)	9 (1–56)	620 (1,706)	44 (0–13,205)	0.093
Interaction index	6.25 (7.34)	4.22 (0.28–16.28)	3.35 (6.01)	1.19 (0–28.50)	0.471

*Significance P < 0.05. Mann–Whitney test

DISCUSSION

Social media services represent an important form of interaction between Internet users, for example, sharing information, sharing opinions, and creating content. YouTube is a popular and important social media service, which is evident by the many interactions that occur between users through view, like, and dislike activities on YouTube. This characteristic is an important measure that can indicate the visibility and popularity of a YouTube video or channel. In this study, most of videos were uploaded by independent users. In previous studies, the evaluation results also indicated that most assessed videos were produced by independent users.[13] These results show that many YouTube users seek information about dental caries to learn more or to share information with others.

Currently, it is easy to acquire information from social media services. Most patients use the Internet to search for health information, including dental caries. Dental caries can be experienced by people of all ages. Dental caries are a chronic disease that will impact a person’s quality of life if not treated immediately.[1920] YouTube video content can be a useful source of information for the user; however such video content should not be the primary source of health-related information.

In this study, in the different categories of uploader sources, the number of views was greater than the number of likes. A high number of views indicates that the majority of YouTube users engage as passive users, that is, searching for information without engaging in any online community interactions.[21] Although most videos are uploaded by independent users, the results of this study show that the values for the number of views, viewing rates, likes, and interaction index of videos uploaded by health professionals are greater higher than those independent users. The visibility and popularity of videos from health professionals are higher than those of independent users, which shows that YouTube users are more interested in videos from reliable sources.

The GQS, usefulness score, and DISCERN score of videos uploaded by health professionals were high in scores. This indicates that the video quality, usefulness, and reliability uploaded by health professionals were better than that of independent users. However, incompleteness was observed because some videos did not mention the source of presented information and did not provide references to viewers to seek additional information about dental caries. Previous studies found that videos uploaded by individual users have a greater chance of being misleading than videos uploaded by authorities/experts.[2223]

In our video duration analysis, videos that had a duration of up to 6 min had higher visibility; however, their popularity was less than that of videos with a duration of greater than 6 min. This indicates that YouTube users are more interested in watching videos that are not too long in duration. The results of a previous study that observed students learning from video content showed that video durations of no greater than 6 min tended to be watched until to completion.[24] In contrast, videos with a duration of greater than 6 min had a higher GQS score than videos that were 6 min long or less. This means that longer videos have better quality; thus, such videos are more popular than shorter videos. However, difference in usefulness score between longer or shorter durations was apparent. This could be explained by the fact that most YouTube videos about dental caries do not contain complete information. The information described in most videos is about treatment procedures and prognosis. Videos with good quality have higher visibility and popularity. Although videos with better usefulness and reliability have higher visibility, they are lower in terms of popularity, which indicates that YouTube users select videos to watch based on interest regardless of, usefulness, and reliability.[25]

This study however may have several limitations. Comparison with unequal videos between health professionals and individual users may lead to a biased result. Further studies shall consider equal to near equal samples for comparisons. Potentially measurement bias might occur due to the dynamics of statistics in YouTube, for example, views, likes, and dislikes, which constantly changing.[14] The results will also depend on the time of the search and keywords used. When using different keywords, the results will differ. In addition, only one keyword was used; therefore, further research is required to examine additional searches with a greater number of keywords. However, some studies have shown good results despite using only a single keyword.[1426] Although the study topic is interesting, however there are certain methodological issues as YouTube studies information reliability should be interpreted with caution and further rigorous studies should be conducted.[5] Previous reviews described few similarities between YouTube healthcare information study methods.[527] The inconsistency of methods being used might occur due to possible variables that may contribute to popularity of YouTube videos.[27] Nonetheless the importance of this study shall be highlighted because YouTube may be used as a very effective information resource. Previous study analyzed YouTube videos as a source for parents’ education on early childhood caries.[26] In addition, the current study has recent data, unique content, and employed usefulness score to analyze the YouTube videos regarding dental caries. Earlier study emphasized on detection, effects, etiology, and prevention of early childhood caries,[26] whereas the current one focused on definitions, indications, contraindications, advantages, involved procedures, complications, costs, and prognosis of dental caries in all age groups. YouTube, as the most popular video-sharing site in the world, has high potential as a platform for health promotion and education.[28] Therefore, research on YouTube is encouraged to optimize it’s use and impact in its role to improve health.

CONCLUSION

YouTube videos uploaded by health professionals have better quality, usability, reliability, visibility, and popularity than videos uploaded by individual users. However, most YouTube videos about dental caries have been uploaded by individual users, which can be problematic if inaccurate dental health information is provided in the video content. Videos that are longer than 6 min have a higher interaction index and higher video quality.

FINANCIAL SUPPORT AND SPONSORSHIP

This study was supported by Program Penelitian Kolaborasi Indonesia (PPKI) Grant 2021.

CONFLICTS OF INTEREST

The authors declared no conflict of interest.

AUTHORS CONTRIBUTIONS

Conceptualization: A.B. and D.A.M.; data curation: D.W.P. and D.A.M.; formal analysis: D.W.P., Y.L.R. and D.A.M.; investigation: A.B., D.W.P., Y.L.R., N.H. and D.A.M.; methodology: A.B. and D.A.M.; writing—original draft: D.W.P., A.B., and Y.L.R.; writing—review and editing: Y.L.R., N.H. and D.A.M. All authors have read and agreed to the published version of the manuscript.

ETHICAL POLICY AND INSTITUTIONAL REVIEW BOARD STATEMENT

The study was approved by the Ethics Committee of the Faculty of Dentistry, Universitas Indonesia, No. 18/Ethical Approval/FKGUI/VIII/2020 with Protocol No. 090190720.

PATIENT DECLARATION OF CONSENT

Not applicable.

DATA AVAILABILITY STATEMENT

The raw data are available from the authors to any author who wishes to collaborate with us.