Factors affecting the quality and reliability of online health information.

BACKGROUND: Internet represents a relevant source of information, but reliability of data that can be obtained by the web is still an unsolved issue. Non-reliable online information may have a relevance, especially in taking decisions related to health problems. Uncertainties on the quality of online health data may have a negative impact on health-related choices of citizens.
OBJECTIVE: This work consisted in a cross-sectional literature review of published papers on online health information. The two main research objectives consisted in the analysis of trends in the use of health web sites and in the quality assessment and reliability levels of web medical sites.
METHODS: Literature research was made using four digital reference databases, namely PubMed, British Medical Journal, Biomed, and CINAHL. Entries used were "trustworthy of medical information online," "survey to evaluate medical information online," "medical information online," and "habits of web-based health information users". Analysis included only papers published in English. The Newcastle Ottawa Scale was used to conduct quality checks of selected works.
RESULTS: Literature analysis using the above entries resulted in 212 studies. Twenty-four articles in line with study objectives, and user characteristics were selected. People more prone to use the internet for obtaining health information were females, younger people, scholars, and employees. Reliability of different online health sites is an issue taken into account by the majority of people using the internet for obtaining health information and physician assistance could help people to surf more safe health web sites.
CONCLUSIONS: Limited health information and/or web literacy can cause misunderstandings in evaluating medical data found in the web. An appropriate education plan and evaluation tools could enhance user skills and bring to a more cautious analysis of health information found in the web.

Introduction

Online health is defined as the use of digital technologies for healthcare, and should enhance the efficiency of medical care, and bring to more precise and personalized medical interventions.[1] It also involves the use of communication and information techniques to address the medical challenges faced by patients or to find possible solutions for treating specific medical issues. Before 1999, the term e-health was intended to define everything virtually related with computers in medicine, whereas today this term is used in a broader sense covering quite extensively the so-called “internet medicine”.[2] This is probably due to the improvement in computer technologies and in the possibility to access the internet easily.[3]

The technological progress has allowed easy access to health information available in the web, but it has increased the possibility to find wrong or not reliable health information. Hence, the problem of the availability of malicious medical web data is becoming a vital issue.[4,5] The majority of web users are not proficient with medical terms and this might lead to misinterpretation of online data. For instance, damage taken by the no-vax movement on the internet was caused because of vaccine deception which enabled the general public to significantly reduce vaccinations. A consequent increase of diseases formerly kept under control thanks to large scale vaccinations was observed.[6] This has happened due to false propaganda with well-tailored stories which did not reflect the reality and therefore misled the general public. Scientists and scholars stressed the need for the wide range of web users looking for medical information to be more suspicious of considering everything they read on the web.[7,8] Some initiatives were already proposed to evaluate web reliability issues[9] and recommendations to web developers to realize quality health sites are available.[10] Among them, Health on the Net (HON) should be mentioned. This is a nonprofit charity organization providing an online calculator tool enabling page administrators to assess the quality of the contents of their sites.8-11 If a WEB site respects all the eight HON principles (authoritative, complementarily, privacy, attribution, justifiability, transparency, financial disclosure, and advertising policy), a HON recognition seal is awarded with one-year validation.[12,13]

DISCERN is another project handled by the University of Oxford allowing web users to provide feedbacks.[12,14] Moreover, the Journal of the American Medical Association (JAMA) has proposed a tool to provide web site score based on four criteria of authorship, data source, commercial interest, and currency.[12,15,16] This test was used to evaluate randomly 35 web sites to check quality and readability of overactive bladder (OAB) sites.[11] Three sites only satisfied JAMA criteria and six disclosed authorship contribution. The simplified measure of gobbledygook (SMOG) and the flesh-Kinkaid index are other tools to address language issues of web texts.17,18 Deep behavior analysis, web user hallmarks, and safety perseverance issues are still needed to be addressed.

In the present study, we have reviewed the state of the art of web user hallmarks and if health information contained in web sites were reliable. Analysis of online health information was done to assess the awareness of the possibility to encounter fraudulent data among both users and experts.

Methods

Study selection

Four literature databases including Pub Med, British Medical Journal, Biomed, and Cumulative Index to Nursing and Allied Health Literature (CINAHL) were used for identifying papers for our research. Articles published between 2003 to 2019 were included (Figure 1). Entry terms used were “trustworthy of medical information online”, “survey to evaluate medical information online,” “medical information online,” and “habits of WEB-based health information users”.

Figure 1.

Number of documents identified from individual library.

Inclusion criteria

Article evaluation was done by a two-phase screening method. In the first phase, title and abstract were extracted to identify possible relevant articles, and the screening of individual abstract was done in the second phase. All materials and studies, published before 2003 (the year which is considered as the beginning of the digital era) were excluded. Before 2003 the public did not use often digital platforms for obtaining medical information.[19] Papers published in non-peer-reviewed journals, reviews, and articles not covering characteristics of health information users were also excluded.

Study selection and quality check

Study search has identified 212 studies, and 35 articles were excluded due to duplication. The remaining 177 papers were further screened to identify outcomes of interest, full text, English language, and humans. After this further screening, 55 papers were selected for full-text reading by co-authors of this study. Each reviewer received an equal number of papers for quality assessment by the Newcastle-Ottawa Scale (NOS).[20] The quality of each study was defined as poor (0–4), moderate (5–6), and good (7–9). The score was assigned based on the following filters: comparability and outcome of individual papers (Figure 2). Thirty-one papers were excluded because of quality issues (i.e., NOS < 7), and the remaining 24 were included for a further qualitative synthesis (see Appendix 1 for details).

Figure 2.

Search and article evaluation procedure.

Aims and objectives

The present study had two main objectives, namely the identification of trends in health WEB sites use (RO1), and the level of trust and overall WEB site quality (RO2). Table 1 summarizes the data derived from literature with some extra information when it was available.[21-45]

Table 1.

Adopted study characteristics of online health quality papers reviewed.

Author	Article type	Method	Sample	Principal findings	Limitations
Ayantude21	Original research	Questionnaire	663	42% of the interviewees had used the Internet for medical information.	Not identified
Pletneva36	Case study	Survey	524	Provide insight into medical knowledge after a consultation with a physician. WEBsite quality is being a great concern.	The sample is not representative of the overall medical and health information.
Powell22	Case study	Survey	792	Women resulted in seeking healthinformation online more often thanmen did. Obstacles encountered are inaccuracy of the information and the absence of human interaction.	The authors were not able to calculate the response rate. The sample may not represent the overall population online.
Pletneva30	Case study	Survey	384	Most of the users committed to retrieving health information online were graduated or post-graduated.	The sample is not representative of the overall medical and health- information user community.
Fergie23	Case study	Survey	34	Social media WEBsites are used as a health information source because peers’ support reassured them. Reliability was a common concern.	The sample was small and not heterogeneous.
Chauduri26	Case study	Survey	403	Health care providers and pharmacists are the most trusted source by older adults, and the internet considered one of the least reliable sources.	Most of the respondents are white, the response rate is low.
Katz29	Original research	Questionnaire	225	Most of them looked for online health information and likely receive useful medical links by their physicians	Not identified
Fiksdal31	Clinical trial	Interview	3 focus groups with 5–6 member	The WEB was deemed useful for minor health issues. The high quantity of search results has made the users feel confused and has described as one of the main reasons to stop the search process.	Sample limited to inhabitants of Olmsted County, MN. Only highly educated individuals.
Beck34	Case study	Survey	1052	Health-related information online seeking was more likely among women, executives, and managers. 29.9% saw the Internet as a replacement for the doctor’s consultation.	Not identified
Orgaz-Molina32	Case study	Survey	175	Patients turned to the internet mainly to understand the information given by a physician or to be more prepared before a clinical visit.	The larger sample requested to confirm the findings of this study.
Sadah35	Original research	Comparative Analysis	276 medical keywords	In the case of drug reviews and health forums, women are more likely to have internet access	Classification of ethnicity and gender is not perfect.
Lussiez33	Case study	Survey	337	60% of patients are ready to perform WEB searches before a clinic visit. It has been highly observed in cancer patients, younger individuals, and scholars.	Small sample size and not perfectly reflect the trends of all online population.
Benetoli24	Clinical trial	Interview	36	Facebook, Wikipedia, and YouTube were widely used among participants because of their simplicity of use and emotional support they provided.	Participants have recruited only from the Sydney metropolitan area, and the study is not generalizable to all the population.
Paige27	Case study	Survey	829	Assessment of user awareness of e-health resources, information seeking skills, and engagement abilities.	Not identify the social nature of e-health
Jacobs28	Original research	Questionnaire	365	90% of Dutch women do consider internet information is reliable during or before pregnancy, and among 76.2% are suggested the need for reliable WEBsites by caregivers	A large percentage of individuals not satisfied with the information given on the internet.
Leitner40	Perspective	Viewpoint	2 WEBsites	Wikipedia is the easiest to use the source, but it is not as accurate as of the NCI WEBsite.	Not defined
De Groot41	Original research	Statistical analysis	100 WEBsites about pancreatic cancer.	The average overall quality was poor compared with that of the NCI WEBsite.	Evaluation of English WEBsites only done
Alsaiari42	Original research	Experiment	35 WEBsites about adult kidney cancer.	The average DISCERN score was 42 out of 80, only 15 sites had the HON code seal, and only 3 met all the JAMA criteria. In many cases, the readability level was too high to understand by everyone.	Non-consideration of other search engines and poor assessment multimedia contents.
Arif43	Case study	Comparative analysis	188 WEBsites about breast cancer treatments.	Only 13 WEBsites had the HON code seal. Google showed first the most readable sites instead of those with the HON code seal.	Only one search query and not in other languages.
Sakun39	Original research	Scoring based test	222	Quality of health sites can influence the users when they have trust in the usefulness of the particular site	Limited sample and not addressed the cross-cultural values
Pauer44	Case study	Survey	693 WEBsites about rare diseases.	None of the WEBsites met all the quality criteria, and the overall quality of sites about rare diseases has considered insufficient.	Only WEBsites in the German language. they have not been considered the information available on social media.
Sillence45	Original research	Staged model trust development	15 menopausal women.	Most of the participants declared information they read had influenced their decisions	Data was not generalizable to all kinds of Internet users.
Dutta-Bergman37	Original research	Independent sample tests	2636 interviewees.	Participants who considered medical universities as the most trustworthy sources were younger, more educated and had higher salaries	The study did not consider all the health information sources available on the Internet.
Zulman38	Case study	Survey	1450 older adults.	The internet was perceived as a trustworthy source, and those with a higher level of trust reported that information had changed their behaviour.	Self-reported data could have generated biases. All the respondents came from a single territory.

Results

RO1: Trends in health web site use

Analysis has been considered in sequential order, from the earliest to the most recent articles to evaluate changes in web user habits over time. A survey on 800 UK patients has reported that 42% of patients were looking for online health information in everyday life.[21] Another study on NHS website user’s motivations and individual habits estimated that 50% of the users visit a website before consulting a doctor.[22] The reason for this was that they felt access with online health information quicker and easier and thanks to the web they felt more confident on dealing with medical data. The trend of sharing personal health problems on social media platforms can be found in teenagers.[23] In countries like Australia, famous social media platforms including Facebook, Wikipedia, and YouTube are used to gain medical knowledge. In particular, most participants are highly active on Facebook, and some of them were also members of personal disease groups, follow health related pages.[24] Issues of reliability and trust could be a big concern on these platforms while dealing with online health information.[25]

Older adults are not much interested to have access to online health sources and they preferred to have direct doctor visits. This trend was identified through study of 403 older respondents.[26] Physicians, and pharmacists are more trusted sources than online information, and it is a proven that older adults are less vulnerable to web-health.[27] On the other hand, preliminary results of online usage for pregnancy check in Netherlands women have found that 50% of them made their decisions through the internet.[28] Oncology patients are using the internet to get information about their disease and were happy to receive useful website links through their physicians.[29] Most of the real-time users consulting online health information were post-graduated or scholars. This suggests that people with a higher level of education are more interested in being e-patients.[29,30] To sum-up older adults are more prone to consider physician suggestions rather than information they get from the internet.

In some cases, principal motivations behind turning to the internet were either to integrate information provided by physicians or enhance knowledge before consulting a physician. A large amount of search outcomes makes users confused, and the internet could be useful for minor health issues but not in detail.[31] This is proven in a study concluding that medical information search is a widespread reality among dermatological patients and is not associated with a poor relationship with the physician.[32] In contrast, outcomes from thoracic surgery patients on internet search mentioned that cancer patients who were younger, with high income and better education are more prone to get information from hospital websites.[33] These studies indicate that the internet represents a highly consulted and trusted source after dermatologists and personal physicians.

A survey on 1,052 respondents indicated the habit of seeking online health information involved more often women, and people with higher incomes.[34] A few papers have classified socio-demographic attributes of individuals who habitually search for online health information. Women were more likely to access the internet than men, primarily for drug review websites and health forums.[35] Comparison of results from both studies revealed that people are interested to search online medical information to gain better knowledge of disease symptoms, although doctors remain the most reliable sources. In 2010, the HON foundation tried to identify motivation behind experts.[36] The main purpose of internet research was to get more information compared with those obtained by physicians about diseases, drug side effects, safety, and ability. Site quality was identified as a main obstacle to claim web health site as a good quality one.

RO2: Level of trust on overall web site quality

Online site selection should be done based on the level of trust and quality. Three studies highlighted user knowledge assessment on the reliability of health sites.[39-41] In 2010, Wikipedia accounted for expertise suggestions on musculoskeletal tumor surgery.[40] Considering the needs of web information on pancreatic cancer, a study highlighted the poor quality of online information on pancreatic cancers.[41] Moreover, two studies on 35 kidney cancer websites, and 188 breast cancer websites have found that only 12.5% of sites fulfilled the HON requirements.[42,43] Rare diseases sites, although of good quality, did not meet the JAMA quality criteria.[44] A study on 15 menopausal women to identify the trust model applied on the search of information on hormone replacement therapy has shown that these women preferred websites with a better visual appeal and design.[45] A comparison of the results of the above studies indicates that sites with better visual appearance and good quality are those preferred for conducting an online search for health information.

Discussion

The present study was focused on the identification of the characteristics of subjects consulting health web sites and to define if these users have an awareness of problem of the quality of information contained in these sites. At the beginning of this century, the three most trusted sources of medical information were doctors, universities, and government/governmental agencies sites. This situation remained constant for a few years. Subsequent studies have evaluated the importance of availability of online health data, and reported that females were more frequent readers than man of health web sites.[23,26-28,30]

More recent studies have confirmed that 85% women are more likely to use the internet than men and reported that only 22% of web users over 65 years in the US are interested in health-related topics available in the web.[46] More recently, patients are adapting themselves to the digital era and are becoming “e-patients”, more prone to get online health information. The large amount of information available on the internet makes people more aware of the possibility to get health data from this repository.[31] Patients with chronic diseases or disabilities are the most active members.[47] Users with a higher level of education are more prone to explore the internet to get information on health topics. Another study has reported that among users of internet health information, the 82% of them were students. Students acknowledged that the internet could have been a potential source of bias, given that “anyone can create a blog or something of the like and publish anything they think worthwhile”.

Older adults do not represent common users of the internet for obtaining medical information and they prefer to address medical questions to their physicians.[48] An opportunity living in the digital era, is to use available technology to gather reliable information for old patients taking into account that they are more exposed compared to young people to chronic diseases.[49] An instrument which can serve as a better guideline to improve presentation of health information on the internet was proposed.[50] This to avoid data inaccuracy, disorientation, and the absence of human interaction.

Health information quality (HIQ) analysis can be realized based on several criteria: 1) Categorization of the typology of topics discussed; 2) Classification using JAMA score; 3) Presence of HON code seal; 4) Readability using FK and SMOG. In line with these studies, the DISCERN toolkit can be used to judge the quality of health information by analyzing reliable published material, and the LIDA instrument allows design evaluation and analysis of accessible health information contents.[51,52] The understanding of health information requires a high school education. It should be mentioned that the more reliable web sites do not appear on the top list sites mentioned by the three search engines such as Google, Bing, and Yahoo. This issue should be considered by the web designers, or concerned authorities to guarantee the provision of quality health information.

The quality criteria for health sites must follow the HON codes. The issues related to the target audience, site development, information nature, and presentation style should be identified by the respondents.[58] This point should consider not only the site development, but also should explain quality issues in particular for health-related contents. Collaboration between health schools and/or health professionals and reviewers is required to a web site editor, and these initiatives could enhance transparency and accountability of health web sites.

Some patients claim that internet access is necessary before or after a clinical visit.[31,33] Information features such as quality seal, author relationship with either an university or a hospital, and data from medical databases (e.g., PubMed) could solve in part the limitations of information quality.[35] Analysis of the papers included in the present review has shown that several research groups are directing their efforts to establish quality issues and social consequences of health web sites. Most of the published work on this topic comes from Northern European countries,[53-55] and only two studies were identified in Italy.[56,57] In these investigations the 37% of respondents confirmed the use of the web for obtaining health information,[56] with young people, females, and chronic disease patients being the main readers.[56] The development of appropriate educational programs in pharmacies can contribute to promote the culture of extraction of high-quality information from the web. It would be also desirable that health experts advise the general public to enhance awareness on the appropriate use of the internet for medical purposes.[57]

From the analysis of the literature quoted in the present work, we have observed that overall there is not enough awareness of the problem of quality of web health information among the internet users. Some authors have proposed that healthcare professionals should be involved to guide patients in choosing better quality online health information.[43] Most healthcare providers are exposing their doubts on the risks of an independent search of health information on the web. To avoid these problems, health professionals should adapt their competence to patient needs and to represent a guide for them in a secure internet world.[29] However, a problem emerging from some studies is that a number of doctors encounter difficulties in getting online information, and there is the risk that they lack enough awareness of the problem of reliability of health data available on the internet.

Reasonable and attentive use of health information available on the internet could contribute to the improve preventive medicine strategies, and can guide people in moving towards a personalized approach to their health problems.[59,60] In view of this, responsible efforts for assessing the quality of the amount of online global health information should be addressed.[61] Education on the quality of health web sites by governments or education institutions is necessary and could contribute to the development of new strategies in the field of preventive and personalized medicine. What we need is an alliance between institutions, health professionals, and citizens for reaching the goal of a mature and responsible use of the internet as a source of health information.

Getting quality online health information could represent a considerable approach also for users with problems with internet access. It is important to create awareness for older individuals to be better documented about their health conditions by using an information store simple and quick to access.[59,62] The problem of reliability of health information on the web is still unsolved, and misinterpretation remains the main problem for users with not enough health skills. In this context, experts and institutions should favor the creation and spreading of an education plan, for both common internet users and healthcare professionals. An useful evaluation system must be addressed and internet users should take into account their capabilities to control the false information. The internet could be without a doubt a valid support to our health, but only if it is properly used.

Peer review

This manuscript was reviewed by reviewers who have chosen to remain anonymous.

	Selection				Comparability		Outcome
Study	A representative of the exposed cohort	Selection of nonexposed cohort	Ascertainment of exposure	Demonstration that the outcome of interest was not present at the start of the study	Adjust for the most critical risk factors	Adjust for other risk factors	Assessment of outcome	Follow-up length	The loss to follow-up rate	Total
21	1	1	0	1	1	1	1	1	1	8
22	1	0	1	0	1	1	1	1	1	7
23	1	1	1	1	1	1	1	1	1	9
26	1	1	1	1	0	1	1	1	1	8
27	1	1	1	1	1	0	1	1	1	7
28	1	1	1	0	1	1	1	1	1	8
29	1	1	1	1	1	1	1	1	1	9
30	1	1	1	0	1	0	1	1	1	7
31	1	1	1	1	1	1	1	1	1	9
33	1	1	1	1	1	1	1	0	1	8
32	1	1	1	1	1	1	1	1	1	9
34	1	1	1	1	1	1	1	1	1	9
35	1	1	1	1	0	0	1	1	1	7
36	1	1	1	1	1	1	1	1	1	9
24	1	1	1	1	1	1	1	1	1	9
40	1	1	1	1	0	1	1	1	1	8
41	1	1	1	1	1	1	1	1	0	8
37	1	1	1	1	1	1	1	1	1	9
38	1	1	1	1	1	1	1	1	1	9
42	1	1	1	1	1	1	1	1	1	9
43	1	1	1	0	1	0	1	1	1	7
44	1	1	1	1	1	1	1	0	1	8
45	1	0	1	1	1	0	1	1	1	7
39	1	1	1	1	1	1	1	1	1	9