An Evaluation of the Source and Content of Kienböck's Disease Information on the Internet.

Introduction  The use of the internet for health-related information continues to increase. Because of its decentralized structure, information contained within the World Wide Web is not regulated. The purpose of the present study is to evaluate the type and quality of information on the internet regarding Kienböck's disease. We hypothesized that the information available on the World Wide Web would be of good informational value. Materials and Methods  The search phrase "Kienböck's disease" was entered into the five most commonly used internet search engines. The top 49 nonsponsored Web sites identified by each search engine were collected. Each unique Web site was evaluated for authorship and content, and an informational score ranging from 0 to 100 points was assigned. Each site was reviewed by two fellowship-trained hand surgeons. Results  The informational mean score for the sites was 45.5 out of a maximum of 100 points. Thirty-one (63%) of the Web sites evaluated were authored by an academic institution or a physician. Twelve (24%) of the sites were commercial sites or sold commercial products. The remaining 6 Web sites (12%) were noninformational, provided unconventional information, or had lay authorship. The average informational score on the academic or physician authored Web sites was 54 out of 100 points, compared with 38 out of 100 for the remainder of the sites. This difference was statistically significant. Conclusion  While the majority of the Web sites evaluated were authored by academic institutions or physicians, the informational value contained within is of limited completeness. More than one quarter of the Web sites were commercial in nature. There remains significant room for improvement in the completeness of information available for common hand conditions in the internet. Society of Indian Hand & Microsurgeons. All rights reserved. Thieme Medical and Scientific Publishers Pvt. Ltd., A-12, 2nd Floor, Sector 2, Noida-201301 UP, India.

Introduction

The internet as a largely unregulated repository of data provides access to a massive amount of information from a multitude of sources. Health-related information is not an exception to internet searches despite the view that physicians are the most trusted source of information for health-related topics. 1 With increasing availability and accessibility of technology, it is not surprising that a recent cross-sectional study by Swoboda et al found that the most frequent initial source patients use to research health information is the internet. 2 With these changing trends in health information seeking, it is important for physicians to be aware of the general quality of health internet information so that we may, in turn, guide patients toward more credible educational sources and to better contribute to the quality of internet information.

Osteonecrosis of the lunate, Kienböck’s disease, was first described in 1910 by Austrian radiologist Robert Kienböck. 3 Patients typically present with dorsal wrist pain and limitation of wrist motion. Recent studies on Kienböck’s disease have assessed current surgeon preferences and approach to managing the disease. 4 5 Hand surgeons have made many advances in our understanding of Kienböck’s disease since its first description but the exact etiology, natural history, and recommended treatment options continue to evolve. 6 7 We wondered if either of these conditions would cause a difference in the information quality as opposed to a common condition such as carpal tunnel syndrome. 8 9 Past studies have evaluated how internet-based information has changed health care and others have specifically looked at the quality of information on various orthopaedic conditions. 2 10 However, we found no similar assessment that focused on the completeness of internet information on Kienböck’s disease. The purpose of this study was to investigate the information available on the internet for Kienböck’s disease. We hypothesized that the informational content on the internet would reasonably represent our current accepted understanding of this condition.

Materials and Methods

The search phrase “Kienböck’s disease” was used to mimic how patients would search the internet for information about the diagnosis. This phrase was entered into the five most commonly used search engines at the time of this investigation (Google [ www.google.com ], Bing [ www.bing.com ], Yahoo [ www.yahoo.com ], Ask [ www.ask.com ], and AOL Search [ www.aol.com ]). 7 The first 49 unsponsored results were identified and used to create a master roster of Web sites to be evaluated for informational content. A previously published search engine protocol evaluation was utilized and applied to each Web site. 8

Each Web site was reviewed by two independent, fellowship-trained hand surgeons based on evaluation guidelines previously published. 8 The Web sites were reviewed for information on the categories of Authorship, Content, Disease Summary, Treatment Options, Pathogenesis, Complications, and Results. Any variation in assessment was resolved with a third independent evaluator making a final decision. A “Total Informational Score” was then determined based on a scale of 0 to 100.

Authorship

The author of the Web site information was evaluated based on several categories: (1) academic indicated a stated affiliation with a research organization or university; (2) physician indicated the author or authors were individual or group-practice physicians who were not affiliated with a university or research organization or whose affiliation was not stated on the web page; (3) nonphysician care provider indicated chiropractors, physical and occupational therapists, acupuncturists, and other alternative medical providers; (4) commercial site indicated the author represented a commercial website without an interest in a specific commercial product (typically, the stated purpose of these websites was to provide medical information); (5) commercial product indicated an author or authors who were marketing a commercial product for evaluation or treatment of Kienböck’s disease; (6) lay indicated individuals or organizations who did not belong to any of the previous categories and who maintained a noncommercial website for providing information about Kienböck’s disease; or (7) unidentified indicated the author was not specified.

Content

The nature of the information regarding evaluation, treatment, pathogenesis, and prevention of Kienböck’s on each site was separated into one of four categories: (1) conventional indicated the site was dedicated to providing information consistent with conventional knowledge as outlined in textbooks and orthopaedic literature; (2) unconventional indicated the site provided alternative information in addition to conventional knowledge without secondary commercial gains; (3) misleading indicated the site offered unconventional information with secondary commercial gains; or (4) noninformational indicated the site was without patient-related information.

Informational Value

Disease summary (maximum, 30 points): Three points were awarded to each of the following 10 factors when mentioned: pain, weakness, stiffness, collapse or development of arthritis, anatomy of the carpal bones, classification, decreased strength on physical examination, decreased motion on physical examination, disease progression, and diagnosis with X-ray and magnetic resonance imaging.

Treatment options (maximum, 20 points): Five points were awarded to each of the following treatment options when given: splinting/casting, oral anti-inflammatory medications, activity modification, and surgery.

Pathogenesis (maximum, 20 points): Five points were awarded for each of the following etiologies mentioned: trauma, hypercoagulable state, ulnar variance, mechanical overload on the lunate, and idiopathic.

Complications of treatment (maximum, 15 points): The 7.5 points were awarded for each of the following categories mentioned: complications of nonoperative treatment (such as progression of disease, side effects of oral anti-inflammatory medication) and complications of operative treatment (such as progression of disease, stiffness, infection, or nerve injury). A single mention of a complication in each category was sufficient to earn a full 7.5 points.

Results of treatment (maximum, 15 points): The 7.5 points were awarded when the results of nonoperative treatment were described and 7.5 points were awarded when the results of operative treatment were described. A single mention of results in each category was sufficient to earn a full 7.5 points.

Statistical analysis was performed using the Student’s t -test for continuous variables. Interobserver reliability was calculated by comparing agreement between site type and overall informational score.

Results

The mean informational score for the sites was 46 out of a maximum of 100 points (46%). Analyzed by specific score contents, the mean informational values were as follows: Disease summary , 16.7 out of 30 maximum points (59%); Treatment options , 10.7 out of 20 maximum points (54%); Pathogenesis , 10.7 out of 20 maximum points (54%); Complications of treatment , 2.2 out of 15 maximum points (14%); and Results of treatment , 4.3 out of 15 maximum points (29%).

Regarding authorship, 31 (63%) of the Web sites evaluated were authored by an academic institution or a physician. Twelve (24%) of the sites were commercial sites or sold commercial products. The remaining 6 Web sites (12%) were noninformational, provided unconventional information, or had lay authorship. The average informational score on the academic or physician authored Web sites was 54.2 out of 100 points, compared with 38.2 out of 100 for the remainder of the sites. This difference was statistically significant ( p = 0.03).

Regarding interobserver reliability, the reviewers agreed on the authorship in 46 of 49 of the sites (94%), and on the content in 45 of 49 of the sites (92%). The difference in overall mean informational score between observers was 3%, indicating a high degree of agreement.

Discussion

Our review of the first 10 sites from five most commonly used internet search engines returned 49 unique URL addresses. Thirteen of the sites were either commercial or sold commercial products. The mean informational value score was 45.5 points for the websites evaluated. From these data, we conclude that the information available on the internet regarding Kienböck’s disease is of below average completeness. These findings are similar to other studies that evaluated internet information specific to orthopaedic diseases. Regardless of the platform used, whether it was YouTube videos, online images, or Web sites, in general, the information quality was found to be poor. 11 12 13 14 15 16 While the internet contains orthopaedic information that is easily accessible, its accuracy and reliability are not monitored. Cassidy and Baker evaluated several review articles published since 2010 examining the quality and/or readability of online orthopaedic information concluding the information to be of poor quality. 17 The information reliability and accuracy for orthopaedic sports medicine, foot and ankle, pediatric orthopaedics, and hand surgery diagnoses have been investigated and results demonstrate a similar inferior quality of information content. 14 15 16 Cassidy and Baker described three common instruments to measure online quality including The Journal of the American Medical Association (JAMA) benchmark criteria, the DISCERN criteria, and the Health On the Net Code (HONcode), but unfortunately, these are not universally utilized. 17 18 19 20

A previous study looked at trapeziometacarpal arthritis Web sites for readability and quality with the Flesch–Kincaid grade level and the HONcode site evaluation, respectively. 21 In 2013, only 3 of 60 Web sites contained a HONcode. Some of the sites may have had appropriate information to qualify for HON certification, yet few designers took the extra steps to obtain and postcertification. Similar to others, the authors found the available internet-based health information is of poor quality, predominantly posted by physician authors and hard to read. Along the same lines, another study evaluated the quality of pediatric orthopaedic information on the internet and found that Web sites with a higher HON score had a positive correlation with a higher content score custom developed by the authors. 16 A similar trend was noted in the opposite direction, where low HON scoring Web sites also had low content scores. Starman et al evaluated online information quality of the top 10 common sports medicine diagnoses and found that Web sites with a HON seal had a significantly higher HON score and content score compared with those without an HON seal. 15 The HON code of conduct may be a valuable tool in internet information quality standardization.

One of the limitations of this study is the measurement scale used. While it is effective in determining overall completeness and relevance of information available, it is not a representation of overall accuracy. If all factors as delineated by our methods were addressed, the Web site would obtain a score of 100. This scoring system did not deduct for extraneous and possibly incorrect pieces of information. Thus, a Web site with misleading or wrong information could still receive a high score. A Web site that had fully factual information but did not address all factors would have received a lower score due to its lack of completeness. However, this evaluation method was chosen due to its emphasis on completeness while still requiring a basic level of accuracy. If patients need to cross-reference multiple Web sites to find what they are looking for, they may find the task time consuming and frustrating. Additionally, since specific criteria for each category had been identified, it can be assumed there was a basic level of accuracy required to obtain full points for a category.

There is no doubt that in this information age, the internet is critical in providing information regarding medical conditions and treatments to patients. The majority of the Web sites at the top of the search lists were authored by physicians or academic institutions. These Web sites had significantly higher scores than those that were commercial or lay sites. Based on our completeness scores, there still remains significant room for improvement in the quality and completeness of information available for hand conditions on the internet. The findings demonstrate that due to a lack of completeness, internet information cannot be considered a replacement for in-person office visits. With the knowledge that Web sites are incomplete, web owners will be able to make the necessary modifications to ensure that a more through overview is provided to health information seeker.