A Multimetric Readability Analysis of Online Patient Educational Materials for Submental Fat Reduction.

BACKGROUND: Patients often utilize the Internet to seek information related to their care. This study assesses the readability of online patient educational materials for submental fat reduction.
METHODS: Patient educational materials from the 12 most popular websites related to submental fat reduction were downloaded and assessed for readability grade level using 10 unique scales.
RESULTS: Analysis of the 12 most popular websites (and corresponding 47 articles) revealed that patient educational materials were written, on average, at an 11th grade reading level. The Flesch Reading Ease score was 48.9 (range 39.8-59.2), representing a "difficult" level of reading. Mean readability grade levels (range 9-13th grade for individual websites) were as follows: Coleman-Liau, 11.1; Flesch-Kincaid, 10.8; FORCAST, 10.8; Fry Graph, 10.1; Gunning Fog, 12.7; New Dale-Chall, 10.1; New Fog Count, 11.8; Simple Measure of Gobbledygook, 11.7; Raygor, 6.7. No website was at the 6th grade reading level for patient educational materials recommended by the American Medical Association and National Institutes of Health.
CONCLUSIONS: Online patient educational materials for submental fat reduction are written well above the recommended reading level. Recognition of disparities in health literacy is necessary to enable patients to make informed decisions and become active participants in their own care. LEVEL OF EVIDENCE V: This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors   www.springer.com/00266.

Introduction

Digital health is an increasingly relevant in the post-pandemic world [1]. Due to social distancing norms and nationwide lockdowns, the COVID-19 pandemic has led to a surge in patients’ use of the Internet to obtain information related to their healthcare [2]. Online interest in cosmetic procedures, as illustrated with Google searches, have increased to levels higher than those prior to the pandemic.[3-5]

With the growing influence of the Internet on patient decision-making, it is important for online patient educational materials to be written in a manner easily understood by the average patient. Health literacy is an important predictor of patient outcomes, given its association with greater patient compliance, decreased complication rates, and overall lower healthcare costs [6-10]. Nearly half of adults in the United States have low or limited proficiency in basic language and health literacy, with the average American adult having a 7–8th grade reading level [11, 12]. The American Medical Association (AMA) and National Institutes of Health (NIH) recommend that healthcare information be presented at no higher than a 6–7th grade reading level. [6, 13]

Despite this, patient education materials from the American Society of Plastic Surgeons and American Society for Aesthetic Plastic Surgery websites are consistently rated at a significantly higher reading level and reading difficultly when compared to information from other health care related websites [14]. Articles from the American Academy of Facial Plastic and Reconstructive Surgery site were found to be written at an average of a 12th grade reading level when analyzed with 10 different readability scales [15]. In light of the increasing relevance of digital health and the necessity of comprehensible patient educational materials, our objective was to evaluate readability of the most accessed websites for submental fat reduction and relate these findings to the AMA and NIH patient education reading level recommendations. We chose to examine submental fat reduction as the study material given the long-term popularity of submental liposuction and the recent growth in popularity of non-invasive alternative treatments of submental fat.

Methods

An Internet search query for terms “double chin treatment,” “chin fat reduction,” “nonsurgical chin fat removal,” “submental fat reduction,” and “chin liposuction” were conducted using Google (Google, Inc., Mountain View, CA) on May 20, 2021. To avoid inadvertent bias (e.g., search results being influenced by previous browsing history), the search query was performed on an Incognito browser and location filters, advertisements, and sponsored results were disabled. As patients are most likely to access webpages only from the first page of search results, the first 12 unique websites were screened [16]. All articles containing patient educational materials on submental fat reduction within one click of each parent website were included in our analysis [17]. Each article was then pasted into a Microsoft Word (Microsoft Corp., Redmond, WA) document and converted into plain text format. Photographs, figures, references, and links were removed. Readability of each article was analyzed using the Readability Studio professional edition (Oleander Software, Ltd., Vandalia, OH).

Ten readability tests were performed to evaluate different aspects of readability difficulty: Coleman-Liau Index, Flesch-Kincaid Grade Level, Flesch Reading Ease test, FORCAST scale, Fry Readability Graph, Gunning Fog Index, New Dale-Chall Readability Formula, New Fog Count, Simple Measure of Gobbledygook (SMOG) test, and Raygor Readability Estimate Graph (Table 1). The Flesch Reading Ease test generates a value between 0 and 100, with lower numbers representing a greater difficulty of readability. The Flesch-Kincaid Grade Level uses the same core measurements of word and sentence length as the Flesch Reading Ease test to produce a grade level score, which represents the lowest grade level needed to understand any given text. In addition, the Coleman-Liau Index, Fry Readability Graph, Gunning Fog Index, New Fog Count, SMOG test, and Raygor Readability Estimate Graph all utilize word and syllable count to estimate grade level [15]. The FORCAST scale uses only a vocabulary element to calculate grade level, analyzing the number of single-syllable words in a 150-word sample [18]. Lastly, the New Dale-Chall Readability Formula utilizes a list of 3000 words commonly understood by 4th grade American students to calculate grade level, with words outside of this list considered to be difficult to understand [19]. Readability analysis was performed for all articles, followed by analysis of source websites to quantify differences in readability.

Table 1

Readability test definitions

Readability test	Variables	Formula
Coleman-Liau index	Average number of letters per 100 words (L) and average number of sentences per 100 words (S)	(0.0588 × L)—(0.296 × S)—15.8
Flesch-Kincaid grade level	Average number of syllables per word (SY) and average number of words per sentence (W)	(0.39 × W) + (11.8 × SY)—15.59
Flesch reading ease test	Average number of syllables (B), average number of words per sentence (W), and average number of sentences (S)	206.835—(84.6 × (B/W))—(1.015*(W/S))
FORCAST scale	Number of single-syllable words in a 150-word sample (SS)	20—(SS/10)
Fry Readability Graph	Average number of sentences and syllables per 100 words	(1) Select a 100-word excerpt from the passage, (2) count the number of sentences, (3) count the number of syllables, and (4) find the intersection of the points on the chart
Gunning Fog index	Number of sentences (S), number of words (W), number of words with three or more syllables (C)	0.4 × (W/S + ((C/W) × 100))
New Dale-Chall readability formula	Average number of words per sentence (AW) and percent unfamiliar words (%U)	NDC = (0.0496 × (W/S)) + (0.1579 × (U/W)) + 3.6365
New Fog count	Number of complex words (C), number of easy words (E), number of sentences (S)	NFC = (((E + (3 × C))/S)—3)/2
Simple measure of Gobbledygook test	Average number of words with 3 or more syllables (C) and average number of sentences (S)	1.043 × √ (C × (30/S)) + 3.1291
Raygor readability estimate graph	Average number of sentences and long (six or more characters) words per 100 words	(1) Select a 100-word excerpt from the passage, (2) count the number of sentences, estimated to the nearest tenth, (3) count the number of words that are six or more letters, and (4) find the intersection of the points on the chart

Results

Forty-seven articles related to submental fat reduction from 12 websites were downloaded and analyzed for readability level (Table 2). Treatments for submental fat reduction included liposuction, cryolipolysis, liposculpture, injectable chemical adipocytolysis with deoxycholic acid, and radiofrequency-assisted contouring. All articles had an overall average 11th grade reading level. The average Flesch Reading Ease score, graded from 0 being the most difficult to 100 being the least difficult, was 48.9 and thus classified as “difficult.” From highest to lowest, the mean scores for each of the remaining readability tests was as follows: Gunning Fog Index, 12.7; New Fog Count, 11.8; SMOG, 11.7; Coleman-Liau Index, 11.1; Flesch-Kincaid Grade Level, 10.8; FORCAST, 10.8; Fry Readability Graph, 10.1; New Dale-Chall Readability Formula, 10.1; and Raygor Readability Estimate Graph, 6.7 (Fig. 1). Stratified by parent website, articles produced mean readability grades ranging from 9th (finesseplasticsurgery.com) to 13th grade, or first year of college (dermatologymohsinstitute.com). The Flesch Reading Ease score for websites ranged from 39.8 or “difficult” (coolsculpting.com), representing the hardest to read, to 59.2 or “fairly difficult” (zlmedspa.com), representing the easiest to read (Fig. 2).

Table 2

Websites containing patient educational materials for submental fat reduction, in order of appearance on Google search query.

Website	Organization	Number of articles
mykybella.com	Allergan	6
drkarenhorton.com	Dr. Karen Horton	9
finesseplasticsurgery.com	Finesse plastic surgery	2
dermatologymohsinstitute.com	Dermatology and mohs surgery institute	2
theatlantic.com	The atlantic monthly group	1
healthline.com	Healthline media	5
premierdermatologypartners.com	Premier dermatology Partners	2
zlmedspa.com	Larson plastic surgery	2
thedermgrouppartners.com	The dermatology group	2
medicalnewstoday.com	Healthline media	4
coolsculpting.com	AbbVie	3
goodhousekeeping.com	Hearst lifestyle and design group	9
	Total	47

Fig. 1

Box and whiskers plot comparing readability tests. Middle point: median; box 25–75% (percentile range); whiskers: non-outlier range

Fig. 2

Mean readability stratified by website

Discussion

To our knowledge, this is the first study analyzing readability of online patient educational materials for submental fat reduction. We utilized ten different scales to assess various aspects of readability, such as word complexity and sentence length. None of the 12 parent websites with articles relating to submental fat reduction was at or under the 6th grade reading level for patient educational materials as recommended by the AMA and NIH. Instead, we found that the most popular websites related to submental fat reduction contain articles written, on average, at an 11th grade reading level. As the accessibility and use of online patient educational materials continues to increase, recognition of health literacy needs will be necessary in order to empower patients to become active participants in their own care, make informed decisions, and improve health outcomes.

Online readability analyses have been performed for various breast reconstructive procedures, [16, 17, 20–22] as well as for cosmetic procedures including breast augmentation, abdominoplasty, rhinoplasty, and botulinum toxin injections. [23-25]. Despite the wide array of materials analyzed, all studies have come to the same conclusion: readability of online patient educational materials is on average at a much higher level than appropriate for the average patient. Patient educational materials on websites curated by established plastic surgical societies, universities, private practices, and media publishing groups ranged in readability level from 10 to 14.7, which is 4–8.7 grades higher than the recommended 6th grade reading level. [14, 15, 21, 26–28] No single study found patient educational materials to be at an acceptable reading level. Thus, we strongly encourage the development of targeted interventions to improve the readability of online patient educational materials in plastic surgery across all platforms. Such interventions may include simplification of vocabulary and syntax, the use of shorter sentences, elimination of unnecessary medical jargon, and implementation of simple definitions for potentially confusing terms.

In addition, it is imperative for physicians to consider the increased risk of low health literacy in non-native English speakers. The AMA and NIH fail to factor in non-English speaking immigrants or those for whom English is a second language when calculating literacy rates. However, it is estimated that 44.9% of those with limited English proficiency reported low health literacy as compared to just 13.8% of English speakers. One study found that limited English proficiency may carry greater health risk than low health literacy, though important racial and ethnic variations may exist [29]. Therefore, in addition to adjusting sentence length and complexity, it is paramount that official sources also provide readily-available translated resources.

Readability of patient educational materials is particularly important in aesthetic surgery. Aggressive advertising tactics pose a unique challenge for aesthetic surgery patients compared to those seeking reconstructive surgery, which further underscores the importance of improving readability of cosmetic patient educational materials [30]. Various studies have investigated the highest level of education among cosmetic surgery patients, with mixed results. Multiple studies have found that the average cosmetic surgery patient has significantly fewer years of education than general population controls [31-33], however Schlessinger et al. and Zahirodden et al. found that a majority of private practice cosmetic surgery patients (66.9% and 73%, respectively) were at least college-educated [34, 35]. These differences may be due to a variety of variables including operations studied, socioeconomic, age, and demographic differences. Nevertheless, it is important that patient education materials be geared to the level of the reading audience.

Until recently, treatments for submental fat excess were limited to surgical procedures such as liposuction or direct fat excision. There has been recent growth in the popularity of nonsurgical procedures in fat reduction, which may lower the barrier for patients to access alternative noninvasive treatments for submental fat [36, 37]. According to the Aesthetic Plastic Surgery National Databank Statistics for 2018, liposuction was the second most common surgical procedure, and nonsurgical fat reduction was the third most common nonsurgical procedures in the United States [38]. In addition, most patients considering plastic surgery first search for information on Google over seeking advice from friends or their primary care physician. [39, 40] Montemurro et al. estimated that 95% of patients used the Internet to find information prior to seeking an aesthetic surgery consultation [41]. This highlights the importance of accessible online informational resources as a significant population of patients use the Internet to learn about medical procedures and treatments.

The COVID-19 pandemic has highlighted digital access as a fundamental driver of health outcomes and a social determinant of health [2, 42]. Thus, it is important that online patient educational materials convey information easily understood by the average patient. The differential weighting of parameters by each of the readability scales explains the variability in our results, with each test producing a 10th grade or higher reading level, with the exception of the Raygor Readability Estimate Graph. This was the only test suggestively aligning with the recommended 6th grade reading level.

One notable shortcoming of the readability tests used in this study is that short medical terminology may be more unfamiliar to a reader than a word with more syllables. For example, a patient may be more familiar with the multisyllabic term “abdominoplasty” if it is related to their condition or treatment, rather than the term “flap” or a monosyllabic medical abbreviation such as “GERD.” Utilization of a combination of several readability scales increases the validity of our results in comparison to the estimate provided by one specific readability test. Our study was limited by the exclusion of videos, images, and figures from analysis, as the readability tests employed are only able to analyze text. However, complementary use of multimedia and visual aids in patient educational materials has shown to be beneficial in increasing patient comprehension and satisfaction, especially for those with low-literacy. [43-46] Pictures closely linked to written or spoken text can markedly increase comprehension, attention to and recall of health education information when compared to texts alone [43]. Delp and Jones found that mean correct recall of information was 85% with infographics and 14% without [47]. Similarly, Lehmann found that patients receiving wound-care instructions with cartoons were able to answer questions correctly 46% of the time three days later, compared to only 6% of patients who received only written instructions [48]. Thus, patient comprehension may be actually higher than what is estimated by text-based readability scales. Moreover, recommendations from the AMA and NIH encourage the use of non-written patient education materials in the form of graphic illustrations, audio, video, and other supplementary materials. [6, 7] Further implementation of a scale or machine learning algorithm that can analyze images, and multimedia may help guide physicians and health educators to construct more readable patient education materials.

Our results underscore the need to develop readable health materials for all seekers of health information, regardless of educational status, presumed reading level, age, or socioeconomic background. Accordingly, we urge physicians and healthcare systems to combat digital health inequities by collaborating with community-based, state, and federal organizations to improve health literacy and mitigate disparities in this increasingly digital age of healthcare.

Conclusion

Patients are increasingly turning to the Internet as a source of medical information. In this study, we assessed the readability of the most accessed websites related to submental fat reduction. Use of ten different validated readability scales demonstrated low readability of each website, with estimated reading grade levels higher than those recommended by the AMA and NIH. Our findings suggest that there are significant barriers to accessibility of health information, disproportionately affecting patients with lower health literacy. Recognition of these health literacy disparities is necessary to enable patients to make informed decisions and empower them to become active participants in their own care.