Quality and readability of online information on idiopathic subglottic stenosis.

OBJECTIVE: Idiopathic subglottic stenosis (ISS) is a chronic condition characterized by disease recurrence and multiple surgeries. These frustrated patients may utilize the internet to research their condition. The aim of this study was to determine the quality and readability of online ISS information.
METHODS: "Idiopathic subglottic stenosis" was entered into Google. The first 50 websites that met inclusion criteria were extracted. The DISCERN instrument, Flesch Reading Ease Score (FRES), and Flesch-Kincaid Grade Level (FKGL) assessed the quality and readability, respectively. Means, SDs, Pearson correlation coefficients, and two-tailed Student's t-test were calculated.
RESULTS: The 50 websites consisted of 17 patient-targeted and 33 professional-targeted websites, plus 30 major and 20 minor websites. The overall DISCERN, FRES, and FKGL scores were 2.81 ± 0.99, 27.75 ± 15.27, and 13.65 ± 2.79, respectively (mean ± SD). Patient-targeted websites had significantly lower quality (DISCERN [P < .00]) but were easier to read (lower FKGL [P < .00], higher FRES [P < .00]) than professional-targeted websites. Minor websites had a significantly lower quality (DISCERN [P < 0.00]) but were easier to read (lower FKGL [P < .00], higher FRES [P < .00]) than major websites. There was a positive correlation between overall quality and difficulty in readability.
CONCLUSION: The quality of online ISS information was suboptimal. Resources were too difficult to comprehend and readability scores were above AMA and NIH recommendations. Improved online information is required to properly educate this patient population. LEVEL OF EVIDENCE: Level 4.

INTRODUCTION

Idiopathic subglottic stenosis (ISS) is a rare disease that comprises 20% of subglottic stenosis cases and affects 1:400 000 persons per year. , ISS tends to affect perimenopausal Caucasian women, however, its etiology is currently unknown. , , The leading hypothesis depicts ISS as aberrant mucosal inflammation secondary to a triggering event. These triggers could include abnormal estrogen and progesterone signaling, gastroesophageal reflux disease, severe coughing, and mycobacterium species. , , , As the etiology of ISS is unknown, treatments only address the stenosis and not the source of the disease. This result in disease recurrence and multiple surgeries.

Since ISS is a chronic and frustrating condition, these patients often do their own research. Patients may consult the internet to better understand their health condition, and for some conditions such as breast cancer, this has significantly increased over time. , In 2013, 35% of U.S. adults indicated that they have utilized the internet to diagnose themselves. In 2018, 80% of people on social media platforms used them to gather health information. Unfortunately, online resources tend to be of low quality and are written at an advanced readability level. , , , Readability is defined as the reading level a person must have to comprehend written information. With 43 million Americans having low literacy levels, it is essential to follow the readability recommendations of the American Medical Association (AMA) and the National Institutes of Health (NIH) to write health information at a fourth to sixth grade level. ,

An appropriate readability level is required for patients to improve health literacy, which is defined as having the ability to read, understand, and communicate important medical and health information. Health literacy is a stronger predictor of health outcomes than a person's age, race, income, and education level. Previous research has shown that low health literacy can lead to suboptimal adherence to preoperative medications and poor comprehension of discharge instructions. Unfortunately, low health literacy has been documented in otolaryngology, as over one‐third of patients undergoing a total laryngectomy met the criteria for low health literacy. ISS is a chronic, relapsing condition, where patients have the luxury of time to do their own research. Consequently, the goal of this study was to evaluate the quality and readability of online patient education information on ISS. The hypothesis was that websites are written at a higher reading level than recommended and that the quality of information is suboptimal.

MATERIALS AND METHODS

This project was exempt by the Research Ethics Board of the University of British Columbia because the study relies exclusively on information that is publicly available. An unconstrained Google search of “idiopathic subglottic stenosis” was done to simulate a patient search. This search was conducted on June 17, 2020 in Toronto, Canada using a non‐academic Wi‐Fi server to represent patients without academic credentials. The first 50 websites were included from the preliminary search. Websites were excluded and replaced if they were irrelevant sites, duplicate sites, broken links, advertisements, non‐text media (eg, videos and PowerPoint presentations), or focused on pediatric subglottic stenosis. Websites not written in English were excluded. The top 50 websites that met inclusion criteria were categorized as patient targeted vs professional targeted, and major vs minor. Patient‐targeted websites are written in layman's terms to provide patients with education and support. These included clinic pamphlets, patient stories, blogs, and so forth. Professional‐targeted websites are written to communicate with professionals and included peer‐reviewed journal articles, treatment guidelines, and medical specialty‐specific newsletters. When peer‐reviewed journal article abstracts were encountered but the full‐articles required payment for access, only the abstract was analyzed to better replicate a patient search. Major websites included resources from established academic institutions, articles published in peer‐reviewed journals, and from large national organizations. Minor websites included all remaining websites such as blogs and social media posts. Previous literature has shown that 95% of traffic on Google is on the first page of 10 hits. As a result, a subgroup analysis comparing the top 10 hits with the total 50 hits was conducted.

This data collection, categorization, and subsequent quality and readability evaluation and data analysis were completed by a medical student at the University of British Columbia. This medical student was trained with the DISCERN instrument through the use of general instructions, a quick reference guide, and a list of terms. The medical student was closely supervised by the senior author, as per the good practice guidelines from the DISCERN authors. The use of the DISCERN and readability metrics have been used in peer‐reviewed publications on the quality and readability of online resources both in and outside of Otolaryngology—Head and Neck Surgery. , , , , , , , ,

Quality assessment: DISCERN Instrument

The DISCERN Instrument was developed by experts in the United Kingdom to provide consumers of health information with a tool to assess the quality of written information regarding treatment choices. This tool consists of 16 questions rated on a 5‐point scale, where the first 15 questions each represent a quality criterion. These questions are organized into three categories: (a) publication reliability (Q #1‐8), (b) treatment information quality (Q #9‐15), and (c) overall publication rating (Q #16). The question rating scores are used to generate a mean DISCERN value on a 5‐point scale which reflects the overall quality of the website text.

Readability assessment: FRES and FKGL

Readability was assessed using the Flesch Reading Ease Score (FRES), and the Flesch‐Kincaid Grade Level (FKGL) score. These readability tests assigned their scores using the number of syllables in words and the number of words in sentences. The FRES is an index that rates the text on a 100‐point scale, where a higher score indicates that the text is easier to understand. In contrast, the FKGL rates the text using U.S. grade levels, such that a higher score indicates that the text is more difficult to read. These scores were calculated by copying the website text (omitting references) into Microsoft Word 2019 for Mac and using the spelling and grammar tool. A criticism of the FRES and FKGL scores is that both can be inflated by a medical term with many syllables. To address this concern, “Idiopathic subglottic stenosis” was replaced in the text by ISS and the FRES and FKGL scores were re‐calculated. The FRES and FKGL formulas used by Microsoft Word 2019 can be found below. where:

ASL = average sentence length (the number of words divided by the number of sentences).

ASW = average number of syllables per word (the number of syllables divided by the number of words).

Statistical analysis

Statistical analyses were performed using the data analysis tool on Microsoft Excel 2019 for Mac. This included calculating the overall means, standard deviations and Pearson correlation coefficients for DISCERN, FRES and FKGL scores. The patient targeted and professional targeted website scores, the minor and major websites scores, FRES and FKGL scores for websites with ISS included or excluded, and scores from the first 10 websites vs all 50 were compared using a two‐tailed unpaired Students' t‐test assuming unequal variances. An a priori significance level (P < .05) was used.

RESULTS

The search yielded about 123 000 results. The first 50 websites were included, but upon further review, seven were excluded due to non‐text media, broken links, irrelevance, and website duplication. These websites were replaced by the next websites in the original search list. The resulting 50 websites consisted of 17 patient targeted websites and 33 professional targeted websites, as well as 30 major websites and 20 minor websites.

The mean and SD values for each DISCERN quality criterion across all 50 websites are listed in Table 1. The overall DISCERN, FRES and FKGL means and standard deviations including the term “idiopathic subglottic stenosis” were 2.81 ± 0.99, 27.75 ± 15.27, and 13.65 ± 2.79, respectively (Table 2, Figure 1). This indicates that search results were of low quality and required high reading levels. To ensure that the readability scores were not inflated by a medical term with many syllables, “idiopathic subglottic stenosis” was removed from all website text, replaced with ISS, and the readability scores were recalculated. When FRES and FKGL scores were compared to the original scores, there was no statistically significant difference.

TABLE 1

DISCERN quality criteria for health information

Quality criterion	Mean score ± SD
1. Are the aims clear?	2.82 ± 1.84
2. Does it achieve its aims?	4.52 ± 0.74
3. Is it relevant?	3.62 ± 1.19
4. Is it clear what sources of information were used to compile the publication (other than the author or producer)?	3.2 ± 1.82
5. Is it clear when the information used or reported in the publication was produced?	3.66 ± 1.69
6. Is it balanced and unbiased?	3.32 ± 1.27
7. Does it provide details of additional sources of support and information?	3.38 ± 1.66
8. Does it refer to areas of uncertainty?	3.44 ± 1.51
9. Does it describe how each treatment works?	2.88 ± 1.44
10. Does it describe the benefits of each treatment?	2.5 ± 1.18
11. Does it describe the risks of each treatment?	2.14 ± 1.26
12. Does it describe what would happen if no treatment is used?	1.16 ± 0.55
13. Does it describe how the treatment choices affect overall quality of life?	2.16 ± 1.08
14. Is it clear that there may be more than one possible treatment choice?	2.94 ± 1.41
15. Does it provide support for shared decision‐making?	1.6 ± 1.16
16. Based on the answers to all of the above questions, rate the overall quality of the publication as a source of information about treatment choices.	2.78 ± 1.17
Overall mean DISCERN score	2.81 ± 0.99

TABLE 2

Mean readability and quality scores for all 50 included websites with or without ISS

Measurement tool	+ISS	−ISS	P value
Mean score ± SD
DISCERN	2.81 ± 0.99	NA	NA
FRES	27.75 ± 15.27	31.47 ± 15.75	.23
FKGL	13.65 ± 2.79	12.71 ± 3.03	.11

Abbreviations: FKGL, Flesch–Kincaid Grade Level; FRES, Flesch Reading Ease Score; +ISS, idiopathic subglottic stenosis text included in article; −ISS, idiopathic subglottic stenosis text removed from article.

FIGURE 1

A, DISCERN website quality scores for 10 websites, 50 websites, and all patient‐targeted, professional‐targeted, major and minor websites. B, FRES readability scores for websites with and without the ISS term, and for 10 websites, 50 websites, and all patient‐targeted, professional‐targeted, major and minor websites. C, FKGL readability scores for websites with and without the ISS term, and for 10 websites, 50 websites, and all patient‐targeted, professional‐targeted, major and minor websites. FRES: Flesch Reading Ease Score; ISS, Idiopathic subglottic stenosis. *P < .05 when comparing patient targeted to professional targeted websites. **P < .05 when comparing major to minor websites

The Pearson correlation coefficients for DISCERN and FRES, DISCERN and FKGL, and FRES and FKGL for these 50 websites were—0.349 (P = .013), 0.317 (P = .025), and −0.934 (P = 4.15 × 10−23) respectively. When comparing the DISCERN, FRES, and FKGL values for patient and professional‐targeted websites, the latter had significantly higher DISCERN (P = 1.27 × 10−6) and FKGL values (P = 1.18 × 10−4), and a significantly lower FRES value (P = 1.12 × 10−5; Table 3, Figure 1). Similarly, when comparing DISCERN, FRES, and FKGL values for minor and major websites, the latter had significantly higher DISCERN (P = 5.33 × 10−9) and FKGL (P = 2.92 × 10−4) values, and a significantly lower FRES value (P = 7.17 × 10−5; Table 4, Figure 1).

TABLE 3

Comparison of readability and quality scores for patient‐ and professional‐targeted websites

	Patient	Professional	P value
Websites, n (%)	17 (34%)	33 (66%)	NA
Mean score ± SD
DISCERN	2.00 ± 0.64	3.23 ± 0.88	*1.27 × 10−6
FRES	42.02 ± 14.47	20.40 ± 9.41	*1.12 × 10−5
FKGL	11.41 ± 2.71	14.80 ± 2.05	*1.18 × 10−4

Abbreviations: FKGL, Flesch–Kincaid Grade Level; FRES, Flesch Reading Ease Score.

P < .05.

TABLE 4

Comparison of readability and quality scores for major and minor websites

	Major	Minor	P Value
Websites, n (%)	30 (60%)	20 (40%)	NA
Mean score ± SD
DISCERN	3.39 ± 0.74	1.95 ± 0.65	*5.33 × 10−9
FRES	20.87 ± 11.87	38.06 ± 14.14	*7.17 × 10−5
FKGL	14.79 ± 2.32	11.93 ± 2.59	*2.92 × 10−4

Abbreviations: FKGL, Flesch–Kincaid Grade Level; FRES, Flesch Reading Ease Score.

P < .05.

In subgroup analysis, the first 10 websites were analyzed and compared to results for all 50. This yielded DISCERN, FRES, and FKGL scores of 2.97 ± 1.00, 32.61 ± 15.76, and 13.05 ± 2.66, respectively. These scores were not significantly different from scores for all 50 websites (Table 5).

TABLE 5

Mean readability and quality scores for first 10 websites and all 50 included websites

Measurement tool	10 Websites	50 Websites	P value
Mean score ± SD
DISCERN	2.97 ± 1.00	2.81 ± 0.99	.67
FRES	32.61 ± 15.76	27.75 ± 15.27	.41
FKGL	13.05 ± 2.66	13.65 ± 2.79	.55

Abbreviations: FKGL, Flesch–Kincaid Grade Level; FRES, Flesch Reading Ease Score; +ISS, idiopathic subglottic stenosis text included in article; −ISS, idiopathic subglottic stenosis text removed from article.

Titles and hyperlinks for the 50 included websites are listed in Table 6.

TABLE 6

Titles and hyperlinks for the 50 included patient and professional targeted websites

Professional targeted websites
Multidisciplinary care of idiopathic subglottic stenosis—Mayo Clinic (https://mayocl.in/3ae39Y6)	Idiopathic subglottic stenosis: techniques and results—PubMed (https://bit.ly/3e8M2Ih)
Comparing results of three treatments for idiopathic subglottic stenosis—PCORI (https://bit.ly/3wVKOss)	Comparative treatment outcomes for patients with idiopathic subglottic stenosis—JAMA Network (https://bit.ly/2P6RkeX)
The role of inflammatory cytokines in the development of idiopathic subglottic stenosis—Translation Cancer Research (https://bit.ly/32eiNyb)	Idiopathic subglottic stenosis: Factors affecting outcome after single‐stage repair—Annals Thoracic Surgery (https://bit.ly/3duaZPo)
Treatment options in idiopathic subglottic stenosis: protocol for a prospective international multicenter pragmatic trial, Protocol—BMJ Open (https://bit.ly/3mLbCad)	Idiopathic subglottic tracheal stenosis due to gastroesophageal reflux: Still a challenge—European Respiratory Journal (https://erj.ersjournals.com/content/54/suppl_63/PA4353)
Molecular analysis of idiopathic subglottic stenosis for Mycobacterium species—Laryngoscope (https://bit.ly/2Q3jfgn)	Adult Idiopathic subglottic stenosis: A diagnosis and therapeutic challenge—ResearchGate (https://bit.ly/3eazIY2)
Idiopathic subglottic stenosis revisited—SAGE Journals (https://bit.ly/3a6AXpZ)	Subglottic stenosis in adults—Medscape (https://bit.ly/3x4wq1e)
Predictors of recurrence after surgical treatment of idiopathic progressive subglottic stenosis—ACTA Otorhinolaryngologica Italica (https://bit.ly/3eeWrSy)	Gastroesophageal reflux characteristics and patterns in patients with idiopathic subglottic stenosis—Hindawi (https://www.hindawi.com/journals/grp/2018/8563697/)
Endoscopic Treatment of Idiopathic Subglottic Stenosis: A Systematic Review—Frontiers in Surgery (https://bit.ly/3am7k3Z)	Subglottic stenosis—Mayo Clinic, Clinical Trials (https://mayocl.in/3aqVANO)
Laryngotracheal microbiota in adult laryngotracheal stenosis—ASM Journals (https://msphere.asm.org/content/4/3/e00211‐19)	Outcomes after cricotracheal resection for idiopathic subglottic stenosis—Wiley Online Library (https://bit.ly/3dzJuUr)
An interdisciplinary approach to the management of idiopathic subglottic stenosis in pregnancy—Scientific Open Access Journals (http://www.scientificoajournals.org/jgo.1021.php)	Idiopathic subglottic tracheal stenosis, an unusual cause of dyspnea during pregnancy—Proceedings of UCLA Healthcare 2016 (https://bit.ly/32nUy0r)
A novel technique for laryngotracheal reconstruction for idiopathic subglottic stenosis—Annals Thoracic Surgery (https://bit.ly/3x8YZKJ)	Idiopathic subglottic tracheal stenosis: An unusual cause of dyspnea on exertion—ATS Journals 2020 (https://bit.ly/3ttqHQe)
Idiopathic subglottic stenosis: a review—Prime PubMed (https://bit.ly/2QCwDrm)	Idiopathic subglottic stenosis: A Familial Predisposition—Science Direct (https://bit.ly/3uRhYre)
Subglottic stenosis—University of Iowa Health Care (https://bit.ly/3alKWI1)	Idiopathic subglottic stenosis an epidemiological single‐center study—Springer Link (https://bit.ly/3suEW61)
Idiopathic tracheal stenosis: A clinicopathologic study of 63 cases and comparison of the pathology with chondromalacia—The American Journal of Surgical Pathology (https://bit.ly/3uYCCFX)	Single‐stage subchordal resection and reconstruction of idiopathic laryngotracheal stenosis in a male patient—JTCVS (https://bit.ly/3dzKYOr)
Subglottic stenosis—UCI Health Voice & Swallowing Center (https://bit.ly/3akcf5k)	Surgical management of idiopathic subglottic tracheal stenosis—European Journal of Cardio‐Thoracic Surgery (https://bit.ly/3dtFy7u)
Treatment alternatives in iSGS—UCSF Clinical Trials (clinicaltrials.ucsf.edu)	Idiopathic subglottic stenosis is a reflux mediated disease—ENT Today (https://bit.ly/3gkbv4b)
Idiopathic subglottic stenosis—Laryngopedia (https://bit.ly/3uYcgUv)
Patient targeted websites
Idiopathic subglottic stenosis—Winchester Hospital (https://bit.ly/3xeTpGW)	Idiopathic subglottic stenosis—National Organization for Rare Disorders (https://bit.ly/3ty9Zza)
Idiopathic subglottic stenosis—ENT Columbia (https://bit.ly/3tud7fw)	Idiopathic subglottic stenosis—Health Engine (https://bit.ly/3x35rmw)
Idiopathic subglottic stenosis treatment NYC—Mount Sinai (https://bit.ly/32pjTY0)	Idiopathic subglottic stenosis—The North American Airway Collaborative (https://bit.ly/3akjeey)
Subglottic stenosis—Wikipedia (https://bit.ly/2QIkzVS)	Subglottic stenosis—Baylor College of Medicine (https://bit.ly/3x2nkC3)
Idiopathic subglottic stenosis—Massachusetts General Hospital (https://bit.ly/2QA8Zfr)	Subglottic stenosis patient gets her breath back—Cleveland Clinic (https://cle.clinic/2QA9x4Z)
Not all noisy breathing is asthma—Richard Gallagher (https://bit.ly/3du4mwt)	Idiopathic subglottic stenosis archives—VVUMC Reporter (https://bit.ly/3x4TLQb)
Idiopathic subglottic stenosis—The Free Dictionary by Farlex (https://bit.ly/3akj6M6)	Conditions we treat: Subglottic stenosis—Johns Hopkins Medicine (https://bit.ly/2QzQAPC)
Comments for case 15—Washington University (http://gamma.wustl.edu/old/vq015te367.html)	Idiopathic subglottic stenosis—Pinterest (https://bit.ly/3agd8vS)
Dana's story—Temple Health (https://bit.ly/3xcMcqz)

DISCUSSION

With the advent of the internet came an immense amount of information that is accessible to patients diagnosed with various diseases, including rare conditions such as ISS. The internet, however, is not completely peer‐reviewed by medical experts and no one regulates the information available. This study is the first to systematically evaluate the quality and readability of online information for ISS. Our results confirmed the hypothesis that online information on ISS is written at a higher reading level (college freshman level) than is recommended for the average patient and that the quality of information is suboptimal.

In terms of readability, the FRES and FKGL scores for these ISS websites did not meet the AMA or NIH readability recommendations of 60 to 70, and fourth to sixth grade levels respectively. , , , Included in these scores were patient education materials from hospitals and academic institutions which also exceeded the recommendations. This study's results are supported by a cross‐sectional analysis of 502 major articles from top U.S. children's hospitals, pediatric otolaryngology fellowships and the American Academy of Otolaryngology—Head and Neck Surgery. This analysis calculated a mean readability grade level of the 10th grade, which was lower than our mean FKGL, but still exceeded the AMA and NIH recommendations. One of the better online resources for ISS was the North American National Airway Collaborative “ISS—A Rough Guide for Beginners” which was of moderate quality and had a seventh grade readability level. These readability concerns may be less of an issue for the ISS community as a recent study in JAMA determined that 88% of ISS patients had at least some college education. However, the quality of these ISS websites remains a concern. The mean quality of all included websites was determined to be low to moderate (2.81 ± 0.99) by the DISCERN instrument. This is in agreement with multiple studies that used DISCERN to evaluate the quality of websites focused on thyroplasty, vocal fold nodules, in‐office vocal fold injections, and the treatment of swallowing disorders. , , ,

When the readability and quality scores were correlated, a weak positive correlation with FKGL and a weak negative correlation with FRES emerged. This means that as the reading difficulty of a website decreased, its quality decreased as well. Minor websites were significantly easier to read and of significantly lower quality than major websites. Likewise, patient‐targeted websites were significantly easier to read and of significantly lower quality than professional‐targeted websites. Ensuring that patient education information is both easy to read and of high quality can be challenging.

Unfortunately, there is no gold standard measurement tool on how to evaluate a health‐related website. A British Medical Journal review article assessed the criteria used to evaluate health‐related websites and 29 different rating tools were identified. For instance, the Journal of the AMA Benchmark Criteria (authorship, attribution, and disclosure) was created 2 years prior to the DISCERN instrument. This tool can be used to ensure website information is reliable and authors are accountable. Additionally, the Health On the Net code (HONcode) seal is an initiative created in 1995 where health‐related webpages, applications and social network pages can apply for certification. This certification indicates that the website meets the HONcode principles for reliability and quality. In addition to these tools, new instruments have been developed including the Patient Educational Video Assessment Tool (PEVAT), which can evaluate the accessibility, reliability, and quality of audiovisual materials.

Similarly, additional readability tools are available including the CDC Clear Communication Index and the following reading grade level (RGL) tools; Raygor estimate, SMOG, Coleman‐Liau, Fry, FORCAST, and Gunning Fog. The CDC Clear communication index provides users with a tool to assess the clarity of public messages and materials. The RGL tools, like FKGL, indicate the United States reading grade level required to read a website effectively. These indices and RGLs could and have been used in combination through software such as the Readability Studio Professional Edition, Version 2019 (Oleander Software Ltd., Vandalia, Ohio, USA) to generate a better understanding of website readability.

We chose the FRES, FGKL, and DISCERN tools because we felt these tools complemented each other. FKGL and FRES have been validated for use for online resources and in the biomedical context. , , Both tools are easily accessible in Microsoft Word. FKGL and FRES are objective and quantitative tools, thus we added the qualitative DISCERN instrument. DISCERN is a validated, reliable scale for assessing quality of patient education around medical treatment. , , DISCERN has been evaluated against several other tools , , and was found to be easy to use and widely accepted in other medical specialities. Three previous peer‐reviewed publications of the senior author also used these three instruments. , ,

Google was chosen as the search engine as it owns about 91% of the search engine market share. Other search engines, like Yahoo and Bing, may have yielded slightly different results. Fifty websites were included because quality tends to decrease after 50 hits. Since critics argue that 91% of search engine traffic goes to the first page of search results (ie, 10 hits), a subgroup analysis compared the first 10 hits with all 50 websites. Analysis revealed that the scores were not significantly different (Table 5, Figure 1). Four previous publications by the senior author also used 50 websites with similar methodology. , , ,

Despite the current state of ISS website quality and readability, there are benefits associated with online resources. For instance, a large online community of patients diagnosed with ISS belongs to a Facebook group called “Living with Idiopathic Subglottic Stenosis.” This group was started in 2009 by a patient diagnosed with ISS and has about 2600 members internationally. A thematic analysis of the communications revealed three main themes: (a) information sharing; (b) emotional support, expression, and experience sharing, and (c) community building. This forum allows patients with a rare, chronic disorder to receive emotional support, share experiences, and learn. However, these online support groups are the perfect environment for misinformation. Therefore, to maximize the benefits of online health information and social media platforms the quality and readability of online health information for ISS needs to be improved.

Strategies for improving online information for ISS include appraising, referring, creating, and optimizing ISS‐related websites. First, surgeons should welcome patients to present online information and be more willing to critically appraise this information with their patients. , , Surgeons should also refer patients to reliable internet resources. One large U.S. health system was successful in connecting patients to reliable content by linking patients to clinician recommended resources through their electronic medical record system. Creating patient‐friendly resources is another option. The American Laryngological Association recently launched a patient education section on their website, providing short print outs of readable information on common laryngological conditions and procedures. To improve the impact of these new resources, their location in Google search results should be optimized with the expert help of a Web Developer.

There were limitations in this study. A single reviewer, a medical student, used the DISCERN instrument to evaluate the websites under the close supervision of the senior author. A single reviewer is permitted under the good practice recommendations by DISCERN authors as the tool was designed to provide objectivity to quality ratings such that a non‐expert may be able to assess online health information. This methodology was also used in three peer‐reviewed manuscripts published by the senior author and in publications on various medical topics including second trimester ultrasound examination, gender‐affirming hormone treatments, and rhinoplasty. , , , , , However, to reduce rating bias, it may have been more ideal to have two independent reviewers extract data with the DISCERN tool. The assessor was not blinded to the category of the website, which could introduce bias in the DISCERN assessment. DISCERN cannot be used to assess the scientific quality or accuracy of a publication because this would involve fact checking against other sources. It is also unable to score audiovisual materials. DISCERN was created to assess treatment choices. Our search term, ISS, was not treatment focused; instead, it searched for all information relevant to ISS. Therefore, the DISCERN score may be low due to Section 2 of the instrument being non‐applicable. Similarly, the FKGL and FRES readability assessments have limitations. For instance, they omit communication factors, like audiovisuals, that contribute to clarity and comprehension. This study excluded non‐English websites. To accommodate our multilingual patient population, material in different languages should be considered. Google Translate can also be used to translate websites into different languages. These limitations should be addressed in future studies.

CONCLUSION

ISS is a chronic, relapsing condition that is frustrating for patients. As a result, patients may be motivated to research their health condition on the Internet. Unfortunately, the Internet is not completely peer reviewed by medical experts and the information is not regulated. The available online information on ISS was of low quality and written at a reading level that surpasses the average patient. When patients look for higher quality information, they are unlikely to understand it because reading level increases with quality. It is essential that otolaryngologists inform their patients of this insufficiency and advocate for improved online patient education resources.

FUNDING AND CONFLICT OF INTEREST

The authors have no funding or conflicts of interest to disclose.