Pyoderma gangrenosum: Readability and quality of online health resources.

Dear Editors,

Pyoderma gangrenosum (PG) is a chronic, often debilitating neutrophilic dermatosis that disproportionately affects women. In the United States, its prevalence in women is two times higher than in men (Xu et al., 2020). Patients are increasingly turning to the Internet for health information (Fox, 2011), but there is a paucity of data on accessibility of online PG health information. Herein, we characterize the readability and quality of online PG patient resources.

An online search using the term “pyoderma gangrenosum” was conducted on February 5, 2021, on three search engines: Google, Yahoo, and Bing. The first 50 results from each site were evaluated (Fig. 1). Duplicate webpages, research articles, advertisements, blog posts, subscription-based sites, and dictionary entries were excluded. Readability was assessed using six established readability metrics: Flesch Kinkaid reading ease, Flesch Kincaid grade level, Gunning Fog score, Simple Measure of Gobbledygook index, Coleman–Liau index, and Automated Readability index. Quality was assessed using the Journal of the American Medical Association (JAMA) benchmark score (four criteria) and the modified Discern instrument (15 items). For the Discern instrument, two reviewers independently scored webpages. Discrepancies within one point between reviewers on the Discern instrument were averaged, and discrepancies of two points or greater were discussed until consensus was reached. The correlation between average readability level and Discern quality score was assessed using the Pearson correlation coefficient, and a p-value ≤ .05 was considered statistically significant.

Fig. 1

Schematic depicting selection and assessment of top searched patient education websites on pyoderma gangrenosum. JAMA, Journal of the American Medical Association; SMOG, Simple Measure of Gobbledygook index.

Twenty-five sites met the inclusion criteria. The average readability grade level was 12.6 (over the 12th grade level), ranging from 9.8 to 18.5 (Table 1). Sites with the easiest readability are Mayo Clinic (9.8), AAO EyeWiki (10.2), and JAMA Dermatology Patient page (10.4). Half of the websites (13 of 25 [52%]) required a college-level reading ability.

Table 1

Most searched pyoderma gangrenosum educational website characteristics (N = 25)

Website name	Average readability grade level*
Mayo Clinic	9.8
American Academy of Ophthalmology EyeWiki	10.2
JAMA Dermatology Patient Page	10.4
Epiphany Dermatology	10.7
UK National Health Service	10.8
EmuAid	10.9
DermNetNZ	11.1
National Institutes of Health Genetic and Rare Diseases Information Center	11.4
British Skin Foundation	11.6
Legs Matter Coalition	11.7
HealthPrep	11.7
Wikipedia	11.8
Diseases Pictures	12.9
Beaver Valley Foot Clinic	12.9
National Organization for Rare Disorders	13
VeryWellHealth	13
Beth Israel Lahey Health Winchester Hospital	13.2
Prime Health Channel	13.3
Drugs.com	13.7
WoundCareCenters.org	13.8
Medical Point	14.2
WoundEducators	14.7
University of Utah Health	14.7
Merck Manual Consumer Version	15
Cleveland Clinic	18.5
Website characteristics	n (%)
Author name provided	10 (40.0)
Author degree (n = 10): MD/MD and PhD/other/unknown	3 (30.0)/1 (10)/2 (20.0)/3 (30.0)
Author is a dermatologist (n = 10)	4 (40.0)
Year written/modified was noted	21 (84.0)
Overall website readability and quality scores†	Mean ± standard deviation (range)
Flesch Kincaid reading ease	38.7 ± 11.8 (4.6–56)
Flesch Kincaid grade level	11.7 ± 2.2 (8.7–18.6)
Gunning Fog score	14.1 ± 2.5 (10.1–21.2)
Simple Measure of Gobbledygook index	10.9 ± 1.8 (8.6–16.4)
Coleman–Liau index	15.2 ± 1.5 (12.2–18.4)
Automated Readability index	11.1 ± 2.4 (7.5–18)
Average readability grade level*	12.6 ± 2.0 (9.8–18.5)
JAMA benchmark total criteria score‡	1.8 ± 1.0 (0–3)
Total modified Discern score§	36.1 ± 6.9 (21–46)
Website modified Discern score categories	n (%)
Excellent (63–75)	0 (0)
Good (51–62)	0 (0)
Fair (39–50)	9 (36)
Poor (27–38)	13 (52)
Very Poor (15–26)	3 (12)

JAMA, Journal of the American Medical Association

Average readability grade level is average of Flesch Kincaid grade level, Gunning Fog score, Simple Measure of Gobbledygook index, Coleman–Liau index, and automated readability index scores.

Flesch Kincaid reading ease is scored between 0 and 100, with a higher score indicating easier readability. Flesch Kincaid grade level, Gunning Fog score, Simple Measure of Gobbledygook index, Coleman–Liau index, and the automated readability index calculate an educational grade level likely required to comprehend a given text.

The JAMA benchmark uses four core standards to evaluate the quality of websites: authorship, attribution (e.g., references), disclosure (e.g., mention of potential conflicts of interest), and currency (e.g., whether website is dated), with a maximum score of 4.

The modified Discern instrument is comprised of 15 questions and analyzes reliability and overall quality of written health information, with a maximum score of 75 (Weil et al., 2014).

Quality scores were variable. The average modified Discern score was 36.1 (maximum 75), which is categorized as poor. The websites with the highest scores were the National Organization for Rare Disease (50) and Legs Matter Coalition (49.5), with readability grade levels of 13 and 11.7, respectively. There was no significant correlation between readability and Discern quality score (r = –0.32; p = .12). The average JAMA benchmark score was 1.76 (range, 0–3). No websites met all four criteria. The majority (21 of 25 [84%]) included a publication or modification date, but only 40% provided authorship information.

Online patient resources for PG, a disease that predominantly affects women, fail to meet recommended reading levels. The difficult readability of online PG resources parallels findings in similar studies on other dermatologic topics, such as psoriasis (Radonjic and Treanor, 2019). In addition, a recent study found similar readability gaps in PG online health resources (Kirchner et al., 2021). Quality across websites in our study was variable; however, the average Discern quality score was poor.

A recent survey of patients with PG found that nearly one-third had an educational level of a high school diploma equivalent or less (Hobbs et al., 2021), highlighting that patients with PG may have difficulty comprehending more than half of the websites in this study. Articles written at a higher grade level were not associated with higher content quality. Therefore, online educational material should focus on improving readability and endeavor to do so without sacrificing quality.

Online patient PG resources need improved readability and quality. Dermatologists can help address this gap by vetting online patient health resources and, when contributing to online PG content, prioritizing patient accessibility.