Evaluation of the quality of information on the internet about 2019 coronavirus outbreak in relation to orthodontics.

This study aims to evaluate the content of information in three different search engines in terms of orthodontics as the source of information at the current stage of the COVID-19 outbreak. An internet search was conducted on April 10th, 2020, using the most popular search engines: GoogleTM, BingTM, and Yahoo!® with the keyword "coronavirus orthodontics". Top 10 websites were evaluated for each search engine. After excluding duplicates the remaining 23 sites were saved in Microsoft Excel programme and evaluated by two independent researchers (HKO and RSO; both experienced orthodontists) using the modified DISCERN tool and JAMA benchmarks. The websites were also classified as "useful, misleading and news updates". Sixty one percent of the websites were classified as useful, 26% as misleading, and 13% as news updates. Most of the authors of the websites were unknown (35%) and followed by orthodontists (30%). The DISCERN and JAMA scores of the four websites were excellent and their target audience were orthodontists. The average modified DISCERN score of 23 websites was moderate (average score 2,8). Useful websites had a significantly higher number of DISCERN and JAMA scores than the misleading websites (p < 0.05). Most of the information available in three different search engines about orthodontics related to COVID-19 were useful. The most reliable websites belonged to American Association of Orthodontists (AAO), Australian Society of Orthodontists (ASO), and British Orthodontic Society (BOS), and they appeared on the first page of the GoogleTM. © IUPESM and Springer-Verlag GmbH Germany, part of Springer Nature 2021.

Background

Coronavirus disease (COVID-19) was first detected in Wuhan, China in December 2019 [1]. This infectious disease now spreads almost all over the world, and was declared as the “International Concern Emergency Health Condition” by the World Health Organization (WHO) as of February 1, 2020 [2]. The new corona virus was reported to be transmitted through the mouth, nose and eyes [3], putting healthcare workers under great potential risk. So that healthcare workers are in potential risk. While the disease was not initially seen in any healthcare worker, 7% of patients were reported to be healthcare workers after 12 January 2020 [4]. Another recent study reported that 29% of 138 patients with COVID-19 infected pneumonia in Wuhan, China were healthcare workers [5]. CDC (Centers for Disease Control and Prevention) recommends postponing non-mandatory appointments to a later date excepting emergency dental practices to protect staff during the epidemic of COVID-19, and to increase the available hospital capacity. Although emergencies are rare in orthodontic practice, pain and/ or infection due to injury in soft tissue as a result of breakage of brackets and wires, or other orthodontic appliances require emergency treatment. One of the best ways to prevent and slow down infection is to inform individuals well about the COVID-19 virus, the disease it causes, and how it spreads. GoogleTM (www.google.com), BingTM (www.bing.com), and YAHOO!® (www.yahoo.com) are the world's most popular search engines with important healthcare sources. Since the internet is a source where people frequently seek information, the reliability and content of the information contained here is important. No studies have been found evaluating the information available on the internet in the field of orthodontics regarding COVID-19, which may threaten orthodontists and patients undergoing orthodontic treatment. This study aims to evaluate the content of information in three different search engines in terms of orthodontics as the source of information at the current stage of the COVID-19 outbreak.

Methods

The Google Trends application was used to identify popular searches related to coronovirus for orthodontics. Search was filtered as "last 1 month" and "all around world". Using this app, "coronavirus orthodontics" was found to be the most common search term (Google Trends, 2020). An internet search was conducted on April 10th, 2020 using the most popular search engines: GoogleTM (www.Google.com), BingTM (www.bing.com), and Yahoo!® (www.yahoo.com). The top 10 websites were evaluated for each search engine.

After excluding duplicates and those unrelated to orthodontics the remaining 23 websites were listed using the Microsoft Excel programme. Two independent researchers (HKO and RSO; both experienced orthodontists) were evaluated the websites using a modified DISCERN tool [6], and JAMA benchmarks [7]. In addition, they also classified the webistes as “useful, misleading and news updates" according to previous studies [8-11]. The profession of the author and the target audience were also recorded.

Ethical approval has not been obtained as this study is not related to humans or any material collected from humans.

Modified discern tool

A modified DISCERN tool containing 5 questions was used to evaluate the reliability of the videos. Before the evaluation, the researchers were calibrated using the DISCERN handbook. This handbook contains detailed information about the index along with instructions and examples to make the assessment correctly. Web sites were recorded using the five Likert scale (1 = no, 2, 3 = partially, 4, 5 = yes). Score 5 shows the highest reliability. Both researchers started research after thoroughly reviewing the World Health Organization website and reading articles about coronavirus in the literature [12-19]. The researchers were aware of some uncertainties about COVID-19. The source of the outbreak, the incubation period of the virus and the exact number of people infected with the disease are not fully known.

JAMA benchmarks

The JAMA benchmarks evaluate four key features that should be clearly seen on a website:

Authorship: The authors, contributors and their identity information and the institutions they are affiliated with must be specified.

Attribution: All sources and references must be listed, copyright must be specified.

Disclosure: The "ownership" of the website must be clearly and fully disclosed, with sponsorship, advertising, insurance, business support and potential conflicts of interest.

Currency: The date when the content was published and updated must be specified.

Attention should be paid to explicitly state each criterion when evaluating. Assessment is made by giving 1 point to each feature that provides the criterion, and 0 points to those who do not. The lowest score to be obtained from Jama is “0”, and the highest score is 4.

Website analysis

Modified form to adapted to the topic at hand:

Useful: If website contains scientifically correct information about any aspect of the disease related to orthodontics (symptoms, treatment, disease prevention etc.).

Misleading: If site contains information that has not been scientifically proven.

News Update: If site contains up to date information about the current state of the disease, published by news agencies.

Statistical analysis

A statistical software (SPSS version 22.0, SPSS IBM, Turkey) was used to carry out statistical analysis. The agreement between the two researchers was analyzed by kappa coefficient analysis. Independent student t test was used to compare the DISCERN and JAMA scores between useful and misleading information content. Significance level was set at p < 0.05.

Results

Of the 30 websites found, 7 were excluded (duplicates). The authors of the remaining 23 websites were undisclosed (35%), orthodontist (30%), reporter (13%), ASO (9%), AAO (4%), BOS (4%), editor (4%) (Table 1). Kappa coefficient for DISCERN, JAMA, and website analysis was found to be 0.88, 0.93, and 0.92 respectively (p < 0.001). Sixty one percent (n = 14) of the websites were classified as useful, 26% (n = 6) as misleading, and 13% (n = 3) as news update. The average modified DISCERN score of 23 websites was moderate (average score 2.8). The modified DISCERN reliability index score and JAMA benchmarks of four websites (https://www1.aaoinfo.org/covid-19/;https://www.aso.org.au/coronavirus-covid-19-information-aso-members/; https://orthodonticsaustralia.org.au/coronavirus-update-should-i-still-see-my-orthodontist/; https://www.bos.org.uk/News-and-Events/News/) were excellent and their target audience were orthodontists. The comparison of DISCERN tool and JAMA benchmarks between the websites containing useful and misleading information was shown in Table 2. The DISCERN and JAMA scores of the useful websites were significantly higher than those of the misleading websites (p < 0.05) (Table 2). 79% of useful websites' target audience was patients, and 21% was orthodontists. A hundred percent of news update webistes' target audience was patients. 67% of misleading websites' target audience were patients, and 33% were orthodontists.

Table 1

Authors of the webistes according to website type

Website source	Total websites	Useful	Misleading	News
	(n = 23)	(n = 14)	(n = 6)	(n = 3)
Undisclosed	8	4	4	0
Orthodontist	7	6	1	0
Reporter	3	0	0	3
AOS	2	2	0	0
AAO	1	1	0	0
BOS	1	1	0	0
Editor	1	0	1	0

Table 2

Comparison of DISCERN and JAMA scores between Useful and Misleading websites

	Useful (n = 14)Mean SD		Misleading (n = 6)Mean SD		p value
DISCERN score	3,31	1,28	1,7	1,04	0.012*
JAMA score	2,57	1,02	0,17	0,41	0.000*

SD,standart deviation.

*Unpaired student t-test. p < 0.05

Discussion

This study is the first to evaluate information about COVID-19 in the English language in relation to orthodontics. Henzell et al. [20] reported that most orthodontic patients use internet-based social media sites. This study was designed considering the fact that information on the Internet about COVID-19 may be inadequate or incorrect.

Only the top 10 websites from the websites generated by the keyword entered into each search engine were evaluated because internet users were generally reported to look at the first page [21]. The reason Google Search Trends is used to identify keywords is that it helps to find popular keywords. In this study, we used a single keyword. This is because there appeared only one keyword in the Google Trends application.

In this study, the websites containing the most reliable and useful information were belonged to AAO, ASO and BOS. All these websites were on Google, but not among the top 10 websites in the other two search engines. This showed that AAO, ASO, and BOS are prime sources for information for internet users using Google.

Many studies have been conducted on the quality and reliability of internet information on orthodontics [22-28]. Considering the fact that it may be beneficial to use more than one tool to objectively evaluate websites, the quality of internet information, in our study, was measured using the DISCERN tool and JAMA benchmarks. In a systematic review, Reynders and Isaia [29] concluded that social media is an essential resource for exchanging information on a wide variety of orthodontic topics. However, the reliability of the internet information on orthodontics is variable in the literature [22-28]. Studies evaluating the quality of information on the internet about 'lingual orthodontics' [22, 23], 'orthodontic extraction' [24], 'orthodontic practice' [25], 'orthodontic pain' [26], 'orthodontic brackets' [27], and 'adult orthodontics' [28] have been previously conducted. The tools, study designs, and statistical methods used in those studies were differ. This difference has made it difficult to compare the results of this study with the results of other studies.

Similar to the findings of the study by Khatri et al. [11] used Modified DISCERN index for reliability and medical information and content index (MICI) tools with the keywords “2019 novel coronavirus”, and “Wuhan virus” and YouTubeTM search engine, and found that the useful websites had a significantly higher number of modified DISCERN scores than the misleading group. Khatri et al. also reported that of the 72 videos included, only 2.7% were misleading videos. However, in this study, misleading websites were much higher than this rate (26%). Therefore, individuals should be aware of that there could be an information pollution on the internet about COVID-19 in relation to orthodontics. According to our study, the fact that most of the websites contain useful information indicates that patients and dentists who are internet users can access reliable information about dental procedures from the internet. However, there is still much unknown about this virus and we do not know what we will encounter in the future. N95 masks, special eye protectors and protective clothes recommended to dentists may become equipments which are routinely used by dentists in the future.

This study had some limitations. First, the content of websites in other languages ​​were not evaluated because the search was only in English. Therefore, the results of this study cannot be generalized. Second, the internet is a dynamic platform. Different websites can be added or removed for searches made at different times. Since some situations related to COVID-19 are still uncertain, the search may need to be repeated when the information is updated. In addition, considering internet users looking at the first page, the inclusion of only the top 10 websites out of millions of results is another limitation of this study. Social media sites and YouTubeTM are also frequently used by internet users. For this reason, the reliability of information about COVID-19 in terms of orthodontics should be investigated on those platforms as well.

Conclusion

Most of the information available in three different search engines about orthodontics related to COVID-19 were useful. The most reliable websites were belonged to AAO, ASO, and BOS, and on the first page of the GoogleTM search engine. We recommend these websites for information about COVID-19 for orthodontics.